Skip to content

Commit 985f7e5

Browse files
committed
doc: update note on how to use core images
1 parent f971d06 commit 985f7e5

File tree

2 files changed

+30
-1
lines changed

2 files changed

+30
-1
lines changed

docs/content/advanced/_index.en.md

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -366,4 +366,32 @@ You can control the backends that are built by setting the `GRPC_BACKENDS` envir
366366
make GRPC_BACKENDS=backend-assets/grpc/llama-cpp build
367367
```
368368

369-
By default, all the backends are built.
369+
By default, all the backends are built.
370+
371+
### Extra backends
372+
373+
LocalAI can be extended with extra backends. The backends are implemented as `gRPC` services and can be written in any language. The container images that are built and published on [quay.io](https://quay.io/repository/go-skynet/local-ai?tab=tags) contain a set of images split in core and extra. By default Images bring all the dependencies and backends supported by LocalAI (we call those `extra` images). The `-core` images instead bring only the strictly necessary dependencies to run LocalAI without only a core set of backends.
374+
375+
If you wish to build a custom container image with extra backends, you can use the core images and build only the backends you are interested into. For instance, to use the diffusers backend:
376+
377+
```Dockerfile
378+
FROM quay.io/go-skynet/local-ai:master-ffmpeg-core
379+
380+
RUN PATH=$PATH:/opt/conda/bin make -C backend/python/diffusers
381+
```
382+
383+
Remember also to set the `EXTERNAL_GRPC_BACKENDS` environment variable (or `--external-grpc-backends` as CLI flag) to point to the backends you are using (`EXTERNAL_GRPC_BACKENDS="backend_name:/path/to/backend"`), for example with diffusers:
384+
385+
```Dockerfile
386+
FROM quay.io/go-skynet/local-ai:master-ffmpeg-core
387+
388+
RUN PATH=$PATH:/opt/conda/bin make -C backend/python/diffusers
389+
390+
ENV EXTERNAL_GRPC_BACKENDS="diffusers:/build/backend/python/diffusers/run.sh"
391+
```
392+
393+
{{% notice note %}}
394+
395+
You can specify remote external backends or path to local files. The syntax is `backend-name:/path/to/backend` or `backend-name:host:port`.
396+
397+
{{% /notice %}}

docs/content/getting_started/_index.en.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,7 @@ You can control LocalAI with command line arguments, to specify a binding addres
178178
| --watchdog-busy-timeout value | $WATCHDOG_BUSY_TIMEOUT | 5m | Watchdog timeout. This will restart the backend if it crashes. |
179179
| --watchdog-idle-timeout value | $WATCHDOG_IDLE_TIMEOUT | 15m | Watchdog idle timeout. This will restart the backend if it crashes. |
180180
| --preload-backend-only | $PRELOAD_BACKEND_ONLY | false | If set, the api is NOT launched, and only the preloaded models / backends are started. This is intended for multi-node setups. |
181+
| --external-grpc-backends | EXTERNAL_GRPC_BACKENDS | none | Comma separated list of external gRPC backends to use. Format: `name:host:port` or `name:/path/to/file` |
181182

182183
### Container images
183184

0 commit comments

Comments
 (0)