Skip to content

Commit 26d6f11

Browse files
mudlermudler
authored
Merge branch 'master' into update/docs
2 parents ae63e21 + 09e5d90 commit 26d6f11

File tree

26 files changed

+586
-150
lines changed

26 files changed

+586
-150
lines changed

‎.github/workflows/image-pr.yml‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
---
2+
name: 'build container images tests'
3+
4+
on:
5+
pull_request:
6+
7+
concurrency:
8+
group: ci-${{ github.head_ref || github.ref }}-${{ github.repository }}
9+
cancel-in-progress: true
10+
11+
jobs:
12+
extras-image-build:
13+
uses: ./.github/workflows/image_build.yml
14+
with:
15+
tag-latest: ${{ matrix.tag-latest }}
16+
tag-suffix: ${{ matrix.tag-suffix }}
17+
ffmpeg: ${{ matrix.ffmpeg }}
18+
image-type: ${{ matrix.image-type }}
19+
build-type: ${{ matrix.build-type }}
20+
cuda-major-version: ${{ matrix.cuda-major-version }}
21+
cuda-minor-version: ${{ matrix.cuda-minor-version }}
22+
platforms: ${{ matrix.platforms }}
23+
runs-on: ${{ matrix.runs-on }}
24+
secrets:
25+
dockerUsername: ${{ secrets.DOCKERHUB_USERNAME }}
26+
dockerPassword: ${{ secrets.DOCKERHUB_PASSWORD }}
27+
quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
28+
quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
29+
strategy:
30+
# Pushing with all jobs in parallel
31+
# eats the bandwidth of all the nodes
32+
max-parallel: ${{ github.event_name != 'pull_request' && 2 || 4 }}
33+
matrix:
34+
include:
35+
- build-type: ''
36+
platforms: 'linux/amd64'
37+
tag-latest: 'false'
38+
tag-suffix: '-ffmpeg'
39+
ffmpeg: 'true'
40+
image-type: 'extras'
41+
runs-on: 'arc-runner-set'
42+
- build-type: 'cublas'
43+
cuda-major-version: "12"
44+
cuda-minor-version: "1"
45+
platforms: 'linux/amd64'
46+
tag-latest: 'false'
47+
tag-suffix: '-cublas-cuda12-ffmpeg'
48+
ffmpeg: 'true'
49+
image-type: 'extras'
50+
runs-on: 'arc-runner-set'
51+
core-image-build:
52+
uses: ./.github/workflows/image_build.yml
53+
with:
54+
tag-latest: ${{ matrix.tag-latest }}
55+
tag-suffix: ${{ matrix.tag-suffix }}
56+
ffmpeg: ${{ matrix.ffmpeg }}
57+
image-type: ${{ matrix.image-type }}
58+
build-type: ${{ matrix.build-type }}
59+
cuda-major-version: ${{ matrix.cuda-major-version }}
60+
cuda-minor-version: ${{ matrix.cuda-minor-version }}
61+
platforms: ${{ matrix.platforms }}
62+
runs-on: ${{ matrix.runs-on }}
63+
secrets:
64+
dockerUsername: ${{ secrets.DOCKERHUB_USERNAME }}
65+
dockerPassword: ${{ secrets.DOCKERHUB_PASSWORD }}
66+
quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
67+
quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
68+
strategy:
69+
matrix:
70+
include:
71+
- build-type: ''
72+
platforms: 'linux/amd64'
73+
tag-latest: 'false'
74+
tag-suffix: '-ffmpeg-core'
75+
ffmpeg: 'true'
76+
image-type: 'core'
77+
runs-on: 'ubuntu-latest'
78+
- build-type: 'cublas'
79+
cuda-major-version: "12"
80+
cuda-minor-version: "1"
81+
platforms: 'linux/amd64'
82+
tag-latest: 'false'
83+
tag-suffix: '-cublas-cuda12-ffmpeg-core'
84+
ffmpeg: 'true'
85+
image-type: 'core'
86+
runs-on: 'ubuntu-latest'

‎.github/workflows/image.yml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
name: 'build container images'
33

44
on:
5-
pull_request:
65
push:
76
branches:
87
- master

‎.github/workflows/release.yaml‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,22 @@ jobs:
3434
sudo apt-get update
3535
sudo apt-get install build-essential ffmpeg
3636
37+
- name: Cache grpc
38+
id: cache-grpc
39+
uses: actions/cache@v3
40+
with:
41+
path: grpc
42+
key: ${{ runner.os }}-grpc
43+
- name: Build grpc
44+
if: steps.cache-grpc.outputs.cache-hit != 'true'
45+
run: |
3746
git clone --recurse-submodules -b v1.58.0 --depth 1 --shallow-submodules https://github.com/grpc/grpc && \
38-
cd grpc && mkdir -p cmake/build && cd cmake/build && cmake -DgRPC_INSTALL=ON \
39-
-DgRPC_BUILD_TESTS=OFF \
40-
../.. && sudo make -j12 install
47+
cd grpc && mkdir -p cmake/build && cd cmake/build && cmake -DgRPC_INSTALL=ON \
48+
-DgRPC_BUILD_TESTS=OFF \
49+
../.. && sudo make -j12
50+
- name: Install gRPC
51+
run: |
52+
cd grpc && cd cmake/build && sudo make -j12 install
4153
4254
- name: Build
4355
id: build

‎.github/workflows/test.yml‎

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -86,11 +86,22 @@ jobs:
8686
sudo cp -rfv sources/go-piper/piper-phonemize/pi/lib/. /usr/lib/ && \
8787
# Pre-build stable diffusion before we install a newer version of abseil (not compatible with stablediffusion-ncn)
8888
GO_TAGS="stablediffusion tts" GRPC_BACKENDS=backend-assets/grpc/stablediffusion make build
89-
89+
- name: Cache grpc
90+
id: cache-grpc
91+
uses: actions/cache@v3
92+
with:
93+
path: grpc
94+
key: ${{ runner.os }}-grpc
95+
- name: Build grpc
96+
if: steps.cache-grpc.outputs.cache-hit != 'true'
97+
run: |
9098
git clone --recurse-submodules -b v1.58.0 --depth 1 --shallow-submodules https://github.com/grpc/grpc && \
91-
cd grpc && mkdir -p cmake/build && cd cmake/build && cmake -DgRPC_INSTALL=ON \
92-
-DgRPC_BUILD_TESTS=OFF \
93-
../.. && sudo make -j12 install
99+
cd grpc && mkdir -p cmake/build && cd cmake/build && cmake -DgRPC_INSTALL=ON \
100+
-DgRPC_BUILD_TESTS=OFF \
101+
../.. && sudo make -j12
102+
- name: Install gRPC
103+
run: |
104+
cd grpc && cd cmake/build && sudo make -j12 install
94105
- name: Test
95106
run: |
96107
GO_TAGS="stablediffusion tts" make test

‎api/api.go‎

Lines changed: 2 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ import (
55
"errors"
66
"fmt"
77
"os"
8-
"path/filepath"
98
"strings"
109

1110
config "github.com/go-skynet/LocalAI/api/config"
@@ -17,7 +16,7 @@ import (
1716
"github.com/go-skynet/LocalAI/metrics"
1817
"github.com/go-skynet/LocalAI/pkg/assets"
1918
"github.com/go-skynet/LocalAI/pkg/model"
20-
"github.com/go-skynet/LocalAI/pkg/utils"
19+
"github.com/go-skynet/LocalAI/pkg/startup"
2120

2221
"github.com/gofiber/fiber/v2"
2322
"github.com/gofiber/fiber/v2/middleware/cors"
@@ -38,25 +37,7 @@ func Startup(opts ...options.AppOption) (*options.Option, *config.ConfigLoader,
3837
log.Info().Msgf("Starting LocalAI using %d threads, with models path: %s", options.Threads, options.Loader.ModelPath)
3938
log.Info().Msgf("LocalAI version: %s", internal.PrintableVersion())
4039

41-
modelPath := options.Loader.ModelPath
42-
if len(options.ModelsURL) > 0 {
43-
for _, url := range options.ModelsURL {
44-
if utils.LooksLikeURL(url) {
45-
// md5 of model name
46-
md5Name := utils.MD5(url)
47-
48-
// check if file exists
49-
if _, err := os.Stat(filepath.Join(modelPath, md5Name)); errors.Is(err, os.ErrNotExist) {
50-
err := utils.DownloadFile(url, filepath.Join(modelPath, md5Name)+".yaml", "", func(fileName, current, total string, percent float64) {
51-
utils.DisplayDownloadFunction(fileName, current, total, percent)
52-
})
53-
if err != nil {
54-
log.Error().Msgf("error loading model: %s", err.Error())
55-
}
56-
}
57-
}
58-
}
59-
}
40+
startup.PreloadModelsConfigurations(options.Loader.ModelPath, options.ModelsURL...)
6041

6142
cl := config.NewConfigLoader()
6243
if err := cl.LoadConfigs(options.Loader.ModelPath); err != nil {

‎api/api_test.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,9 +16,9 @@ import (
1616
. "github.com/go-skynet/LocalAI/api"
1717
"github.com/go-skynet/LocalAI/api/options"
1818
"github.com/go-skynet/LocalAI/metrics"
19+
"github.com/go-skynet/LocalAI/pkg/downloader"
1920
"github.com/go-skynet/LocalAI/pkg/gallery"
2021
"github.com/go-skynet/LocalAI/pkg/model"
21-
"github.com/go-skynet/LocalAI/pkg/utils"
2222
"github.com/gofiber/fiber/v2"
2323
. "github.com/onsi/ginkgo/v2"
2424
. "github.com/onsi/gomega"
@@ -61,7 +61,7 @@ func getModelStatus(url string) (response map[string]interface{}) {
6161
}
6262

6363
func getModels(url string) (response []gallery.GalleryModel) {
64-
utils.GetURI(url, func(url string, i []byte) error {
64+
downloader.GetURI(url, func(url string, i []byte) error {
6565
// Unmarshal YAML data into a struct
6666
return json.Unmarshal(i, &response)
6767
})

‎api/config/config.go‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ import (
99
"strings"
1010
"sync"
1111

12+
"github.com/go-skynet/LocalAI/pkg/downloader"
1213
"github.com/go-skynet/LocalAI/pkg/utils"
1314
"github.com/rs/zerolog/log"
1415
"gopkg.in/yaml.v3"
@@ -300,21 +301,21 @@ func (cm *ConfigLoader) Preload(modelPath string) error {
300301
// Create file path
301302
filePath := filepath.Join(modelPath, file.Filename)
302303

303-
if err := utils.DownloadFile(file.URI, filePath, file.SHA256, status); err != nil {
304+
if err := downloader.DownloadFile(file.URI, filePath, file.SHA256, status); err != nil {
304305
return err
305306
}
306307
}
307308

308309
modelURL := config.PredictionOptions.Model
309-
modelURL = utils.ConvertURL(modelURL)
310+
modelURL = downloader.ConvertURL(modelURL)
310311

311-
if utils.LooksLikeURL(modelURL) {
312+
if downloader.LooksLikeURL(modelURL) {
312313
// md5 of model name
313314
md5Name := utils.MD5(modelURL)
314315

315316
// check if file exists
316317
if _, err := os.Stat(filepath.Join(modelPath, md5Name)); errors.Is(err, os.ErrNotExist) {
317-
err := utils.DownloadFile(modelURL, filepath.Join(modelPath, md5Name), "", status)
318+
err := downloader.DownloadFile(modelURL, filepath.Join(modelPath, md5Name), "", status)
318319
if err != nil {
319320
return err
320321
}

‎docs/content/advanced/_index.en.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ weight = 6
99

1010
In order to define default prompts, model parameters (such as custom default `top_p` or `top_k`), LocalAI can be configured to serve user-defined models with a set of default parameters and templates.
1111

12-
You can create multiple `yaml` files in the models path or either specify a single YAML configuration file.
12+
In order to configure a model, you can create multiple `yaml` files in the models path or either specify a single YAML configuration file.
1313
Consider the following `models` folder in the `example/chatbot-ui`:
1414

1515
```
@@ -96,6 +96,12 @@ Specifying a `config-file` via CLI allows to declare models in a single file as
9696

9797
See also [chatbot-ui](https://github.com/go-skynet/LocalAI/tree/master/examples/chatbot-ui) as an example on how to use config files.
9898

99+
It is possible to specify a full URL or a short-hand URL to a YAML model configuration file and use it on start with local-ai, for example to use phi-2:
100+
101+
```
102+
local-ai github://mudler/LocalAI/examples/configurations/phi-2.yaml@master
103+
```
104+
99105
### Full config model file reference
100106

101107
```yaml

‎docs/content/build/_index.en.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,14 @@ make GRPC_BACKENDS=backend-assets/grpc/llama-cpp build
235235

236236
By default, all the backends are built.
237237

238+
### Specific llama.cpp version
239+
240+
To build with a specific version of llama.cpp, set `CPPLLAMA_VERSION` to the tag or wanted sha:
241+
242+
```
243+
CPPLLAMA_VERSION=<sha> make build
244+
```
245+
238246
### Windows compatibility
239247

240248
Make sure to give enough resources to the running container. See https://github.com/go-skynet/LocalAI/issues/2

‎docs/content/features/GPU-acceleration.md‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,19 @@ This section contains instruction on how to use LocalAI with GPU acceleration.
1515
For accelleration for AMD or Metal HW there are no specific container images, see the [build]({{%relref "build/#acceleration" %}})
1616
{{% /notice %}}
1717

18-
### CUDA
18+
### CUDA(NVIDIA) acceleration
1919

2020
Requirement: nvidia-container-toolkit (installation instructions [1](https://www.server-world.info/en/note?os=Ubuntu_22.04&p=nvidia&f=2) [2](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html))
2121

22-
To use CUDA, use the images with the `cublas` tag.
22+
To check what CUDA version do you need, you can either run `nvidia-smi` or `nvcc --version`.
23+
24+
Alternatively, you can also check nvidia-smi with docker:
25+
26+
```
27+
docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi
28+
```
29+
30+
To use CUDA, use the images with the `cublas` tag, for example.
2331

2432
The image list is on [quay](https://quay.io/repository/go-skynet/local-ai?tab=tags):
2533

0 commit comments

Comments
 (0)