Skip to content

Commit b7383ab

Browse files
Aisukomudler
Aisuko
authored and
mudler
committed
feat(conda): Add the seperate conda env for VALL-E X (#1147)
**Description** This PR is related to #1117 **Notes for Reviewers** * The gRPC server cannot start up ``` (ttsvalle) @Aisuko ➜ /workspaces/LocalAI (feat/vall-e-x) $ /opt/conda/envs/ttsvalle/bin/python /workspaces/LocalAI/extra/grpc/vall-e-x/ttsvalle.py Traceback (most recent call last): File "/workspaces/LocalAI/extra/grpc/vall-e-x/ttsvalle.py", line 14, in <module> from utils.generation import SAMPLE_RATE, generate_audio, preload_models ModuleNotFoundError: No module named 'utils' ``` The installation steps follow https://github.com/Plachtaa/VALL-E-X#-installation below: * Under the `ttsvalle` conda env ``` git clone https://github.com/Plachtaa/VALL-E-X.git cd VALL-E-X pip install -r requirements.txt ``` **[Signed commits](../CONTRIBUTING.md#signing-off-on-commits-developer-certificate-of-origin)** - [x] Yes, I signed my commits. <!-- Thank you for contributing to LocalAI! Contributing Conventions ------------------------- The draft above helps to give a quick overview of your PR. Remember to remove this comment and to at least: 1. Include descriptive PR titles with [<component-name>] prepended. We use [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/). 2. Build and test your changes before submitting a PR (`make build`). 3. Sign your commits 4. **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below). 5. **X/Twitter handle:** we announce bigger features on X/Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! By following the community's contribution conventions upfront, the review process will be accelerated and your PR merged more quickly. If no one reviews your PR within a few days, please @-mention @mudler. --> Signed-off-by: GitHub <[email protected]> Signed-off-by: Ettore Di Giacinto <[email protected]>
1 parent e73cce5 commit b7383ab

File tree

7 files changed

+173
-7
lines changed

7 files changed

+173
-7
lines changed

‎Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ ARG TARGETARCH
1414
ARG TARGETVARIANT
1515

1616
ENV BUILD_TYPE=${BUILD_TYPE}
17-
ENV EXTERNAL_GRPC_BACKENDS="huggingface-embeddings:/build/extra/grpc/huggingface/run.sh,autogptq:/build/extra/grpc/autogptq/run.sh,bark:/build/extra/grpc/bark/run.sh,diffusers:/build/extra/grpc/diffusers/run.sh,exllama:/build/extra/grpc/exllama/exllama.py,vall-e-x:/build/extra/grpc/vall-e-x/ttsvalle.py,vllm:/build/extra/grpc/vllm/run.sh"
17+
ENV EXTERNAL_GRPC_BACKENDS="huggingface-embeddings:/build/extra/grpc/huggingface/run.sh,autogptq:/build/extra/grpc/autogptq/run.sh,bark:/build/extra/grpc/bark/run.sh,diffusers:/build/extra/grpc/diffusers/run.sh,exllama:/build/extra/grpc/exllama/exllama.py,vall-e-x:/build/extra/grpc/vall-e-x/run.sh,vllm:/build/extra/grpc/vllm/run.sh"
1818
ENV GALLERIES='[{"name":"model-gallery", "url":"github:go-skynet/model-gallery/index.yaml"}, {"url": "github:go-skynet/model-gallery/huggingface.yaml","name":"huggingface"}]'
1919
ARG GO_TAGS="stablediffusion tts"
2020

‎Makefile‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -415,6 +415,8 @@ prepare-extra-conda-environments:
415415
$(MAKE) -C extra/grpc/diffusers
416416
$(MAKE) -C extra/grpc/vllm
417417
$(MAKE) -C extra/grpc/huggingface
418+
$(MAKE) -C extra/grpc/vall-e-x
419+
418420

419421
backend-assets/grpc:
420422
mkdir -p backend-assets/grpc

‎extra/grpc/vall-e-x/Makefile‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
.PONY: ttsvalle
2+
ttsvalle:
3+
@echo "Creating virtual environment..."
4+
@conda env create --name ttsvalle --file ttsvalle.yml
5+
@echo "Virtual environment created."
6+
7+
.PONY: run
8+
run:
9+
@echo "Running ttsvalle..."
10+
bash run.sh
11+
@echo "ttsvalle run."

‎extra/grpc/vall-e-x/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Creating a separate environment for the ttsvalle project
2+
3+
```
4+
make ttsvalle
5+
```

‎extra/grpc/vall-e-x/run.sh‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
##
2+
## A bash script wrapper that runs the ttsvalle server with conda
3+
4+
# Activate conda environment
5+
source activate ttsvalle
6+
7+
# get the directory where the bash script is located
8+
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
9+
10+
python $DIR/ttvalle.py

‎extra/grpc/vall-e-x/ttsvalle.py‎

Lines changed: 43 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,15 @@
11
#!/usr/bin/env python3
2-
import grpc
2+
33
from concurrent import futures
4-
import time
5-
import backend_pb2
6-
import backend_pb2_grpc
74
import argparse
85
import signal
96
import sys
107
import os
11-
from pathlib import Path
8+
import time
9+
import backend_pb2
10+
import backend_pb2_grpc
11+
12+
import grpc
1213

1314
from utils.generation import SAMPLE_RATE, generate_audio, preload_models
1415
from scipy.io.wavfile import write as write_wav
@@ -21,9 +22,34 @@
2122

2223
# Implement the BackendServicer class with the service methods
2324
class BackendServicer(backend_pb2_grpc.BackendServicer):
25+
"""
26+
gRPC servicer for backend services.
27+
"""
2428
def Health(self, request, context):
29+
"""
30+
Health check service.
31+
32+
Args:
33+
request: A backend_pb2.HealthRequest instance.
34+
context: A grpc.ServicerContext instance.
35+
36+
Returns:
37+
A backend_pb2.Reply instance with message "OK".
38+
"""
2539
return backend_pb2.Reply(message=bytes("OK", 'utf-8'))
40+
2641
def LoadModel(self, request, context):
42+
"""
43+
Load model service.
44+
45+
Args:
46+
request: A backend_pb2.LoadModelRequest instance.
47+
context: A grpc.ServicerContext instance.
48+
49+
Returns:
50+
A backend_pb2.Result instance with message "Model loaded successfully" and success=True if successful.
51+
A backend_pb2.Result instance with success=False and error message if unsuccessful.
52+
"""
2753
model_name = request.Model
2854
try:
2955
print("Preparing models, please wait", file=sys.stderr)
@@ -49,6 +75,17 @@ def LoadModel(self, request, context):
4975
return backend_pb2.Result(message="Model loaded successfully", success=True)
5076

5177
def TTS(self, request, context):
78+
"""
79+
Text-to-speech service.
80+
81+
Args:
82+
request: A backend_pb2.TTSRequest instance.
83+
context: A grpc.ServicerContext instance.
84+
85+
Returns:
86+
A backend_pb2.Result instance with success=True if successful.
87+
A backend_pb2.Result instance with success=False and error message if unsuccessful.
88+
"""
5289
model = request.model
5390
print(request, file=sys.stderr)
5491
try:
@@ -97,4 +134,4 @@ def signal_handler(sig, frame):
97134
)
98135
args = parser.parse_args()
99136

100-
serve(args.addr)
137+
serve(args.addr)

‎extra/grpc/vall-e-x/ttsvalle.yml‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
name: ttsvalle
2+
channels:
3+
- defaults
4+
dependencies:
5+
- _libgcc_mutex=0.1=main
6+
- _openmp_mutex=5.1=1_gnu
7+
- bzip2=1.0.8=h7b6447c_0
8+
- ca-certificates=2023.08.22=h06a4308_0
9+
- ld_impl_linux-64=2.38=h1181459_1
10+
- libffi=3.4.4=h6a678d5_0
11+
- libgcc-ng=11.2.0=h1234567_1
12+
- libgomp=11.2.0=h1234567_1
13+
- libstdcxx-ng=11.2.0=h1234567_1
14+
- libuuid=1.41.5=h5eee18b_0
15+
- ncurses=6.4=h6a678d5_0
16+
- openssl=3.0.11=h7f8727e_2
17+
- pip=23.2.1=py310h06a4308_0
18+
- python=3.10.13=h955ad1f_0
19+
- readline=8.2=h5eee18b_0
20+
- setuptools=68.0.0=py310h06a4308_0
21+
- sqlite=3.41.2=h5eee18b_0
22+
- tk=8.6.12=h1ccaba5_0
23+
- tzdata=2023c=h04d1e81_0
24+
- wheel=0.41.2=py310h06a4308_0
25+
- xz=5.4.2=h5eee18b_0
26+
- zlib=1.2.13=h5eee18b_0
27+
- pip:
28+
- aiofiles==23.2.1
29+
- altair==5.1.2
30+
- annotated-types==0.6.0
31+
- anyio==3.7.1
32+
- click==8.1.7
33+
- cn2an==0.5.22
34+
- cython==3.0.3
35+
- einops==0.7.0
36+
- encodec==0.1.1
37+
- eng-to-ipa==0.0.2
38+
- fastapi==0.103.2
39+
- ffmpeg-python==0.2.0
40+
- ffmpy==0.3.1
41+
- fsspec==2023.9.2
42+
- future==0.18.3
43+
- gradio==3.47.1
44+
- gradio-client==0.6.0
45+
- grpcio==1.59.0
46+
- h11==0.14.0
47+
- httpcore==0.18.0
48+
- httpx==0.25.0
49+
- huggingface-hub==0.17.3
50+
- importlib-resources==6.1.0
51+
- inflect==7.0.0
52+
- jieba==0.42.1
53+
- langid==1.1.6
54+
- llvmlite==0.41.0
55+
- more-itertools==10.1.0
56+
- nltk==3.8.1
57+
- numba==0.58.0
58+
- numpy==1.25.2
59+
- nvidia-cublas-cu12==12.1.3.1
60+
- nvidia-cuda-cupti-cu12==12.1.105
61+
- nvidia-cuda-nvrtc-cu12==12.1.105
62+
- nvidia-cuda-runtime-cu12==12.1.105
63+
- nvidia-cudnn-cu12==8.9.2.26
64+
- nvidia-cufft-cu12==11.0.2.54
65+
- nvidia-curand-cu12==10.3.2.106
66+
- nvidia-cusolver-cu12==11.4.5.107
67+
- nvidia-cusparse-cu12==12.1.0.106
68+
- nvidia-nccl-cu12==2.18.1
69+
- nvidia-nvjitlink-cu12==12.2.140
70+
- nvidia-nvtx-cu12==12.1.105
71+
- openai-whisper==20230306
72+
- orjson==3.9.7
73+
- proces==0.1.7
74+
- protobuf==4.24.4
75+
- pydantic==2.4.2
76+
- pydantic-core==2.10.1
77+
- pydub==0.25.1
78+
- pyopenjtalk-prebuilt==0.3.0
79+
- pypinyin==0.49.0
80+
- python-multipart==0.0.6
81+
- regex==2023.10.3
82+
- safetensors==0.4.0
83+
- semantic-version==2.10.0
84+
- soundfile==0.12.1
85+
- starlette==0.27.0
86+
- sudachidict-core==20230927
87+
- sudachipy==0.6.7
88+
- tokenizers==0.14.1
89+
- toolz==0.12.0
90+
- torch==2.1.0
91+
- torchaudio==2.1.0
92+
- torchvision==0.16.0
93+
- tqdm==4.66.1
94+
- transformers==4.34.0
95+
- triton==2.1.0
96+
- unidecode==1.3.7
97+
- uvicorn==0.23.2
98+
- vocos==0.0.3
99+
- websockets==11.0.3
100+
- wget==3.2
101+
prefix: /opt/conda/envs/ttsvalle

0 commit comments

Comments
 (0)