Skip to content

Commit 896610a

Browse files
Aisukomudler
Aisuko
authored and
mudler
committed
[Extra backend] Add seperate environment for ttsbark (#1141)
**Description** This PR relates to #1117 **Notes for Reviewers** Same to the latest PR: * The code is also changed, but only the order of the import package parts. And some code comments are also added. * Add a configuration of the `conda` environment * Add a simple test case for testing if the service can be startup in current `conda` environment. It is succeed in VSCode, but the it is not out of box on terminal. So, it is hard to say the test case really useful. **[Signed commits](../CONTRIBUTING.md#signing-off-on-commits-developer-certificate-of-origin)** - [x] Yes, I signed my commits. <!-- Thank you for contributing to LocalAI! Contributing Conventions ------------------------- The draft above helps to give a quick overview of your PR. Remember to remove this comment and to at least: 1. Include descriptive PR titles with [<component-name>] prepended. We use [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/). 2. Build and test your changes before submitting a PR (`make build`). 3. Sign your commits 4. **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below). 5. **X/Twitter handle:** we announce bigger features on X/Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! By following the community's contribution conventions upfront, the review process will be accelerated and your PR merged more quickly. If no one reviews your PR within a few days, please @-mention @mudler. --> Signed-off-by: GitHub <[email protected]> Signed-off-by: Ettore Di Giacinto <[email protected]>
1 parent d222fe3 commit 896610a

File tree

5 files changed

+164
-7
lines changed

5 files changed

+164
-7
lines changed

‎extra/grpc/bark/Makefile‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
.PONY: ttsbark
2+
ttsbark:
3+
@echo "Creating virtual environment..."
4+
@conda env create --name ttsbark --file ttsbark.yml
5+
@echo "Virtual environment created."

‎extra/grpc/bark/README.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Creating a separate environment for ttsbark project
2+
3+
```
4+
make ttsbark
5+
```
6+
7+
# Testing the gRPC server
8+
9+
```
10+
<The path of your python interpreter> -m unittest test_ttsbark.py
11+
```
12+
13+
For example
14+
```
15+
/opt/conda/envs/bark/bin/python -m unittest extra/grpc/bark/test_ttsbark.py
16+
``````

‎extra/grpc/bark/test_ttsbark.py‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
import unittest
2+
import subprocess
3+
import time
4+
import backend_pb2
5+
import backend_pb2_grpc
6+
7+
import grpc
8+
9+
class TestBackendServicer(unittest.TestCase):
10+
"""
11+
TestBackendServicer is the class that tests the gRPC service
12+
"""
13+
def setUp(self):
14+
self.service = subprocess.Popen(["python3", "ttsbark.py", "--addr", "localhost:50051"])
15+
16+
def tearDown(self) -> None:
17+
self.service.terminate()
18+
self.service.wait()
19+
20+
def test_server_startup(self):
21+
time.sleep(2)
22+
try:
23+
self.setUp()
24+
with grpc.insecure_channel("localhost:50051") as channel:
25+
stub = backend_pb2_grpc.BackendStub(channel)
26+
response = stub.Health(backend_pb2.HealthMessage())
27+
self.assertEqual(response.message, b'OK')
28+
except Exception as err:
29+
print(err)
30+
self.fail("Server failed to start")
31+
finally:
32+
self.tearDown()

‎extra/grpc/bark/ttsbark.py‎

Lines changed: 15 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,33 @@
1+
"""
2+
This is the extra gRPC server of LocalAI
3+
"""
4+
15
#!/usr/bin/env python3
2-
import grpc
36
from concurrent import futures
47
import time
5-
import backend_pb2
6-
import backend_pb2_grpc
78
import argparse
89
import signal
910
import sys
1011
import os
11-
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
12-
from pathlib import Path
13-
from bark import SAMPLE_RATE, generate_audio, preload_models
1412
from scipy.io.wavfile import write as write_wav
1513

14+
import backend_pb2
15+
import backend_pb2_grpc
16+
from bark import SAMPLE_RATE, generate_audio, preload_models
17+
18+
import grpc
19+
20+
1621
_ONE_DAY_IN_SECONDS = 60 * 60 * 24
1722

1823
# If MAX_WORKERS are specified in the environment use it, otherwise default to 1
1924
MAX_WORKERS = int(os.environ.get('PYTHON_GRPC_MAX_WORKERS', '1'))
2025

2126
# Implement the BackendServicer class with the service methods
2227
class BackendServicer(backend_pb2_grpc.BackendServicer):
28+
"""
29+
BackendServicer is the class that implements the gRPC service
30+
"""
2331
def Health(self, request, context):
2432
return backend_pb2.Reply(message=bytes("OK", 'utf-8'))
2533
def LoadModel(self, request, context):
@@ -83,4 +91,4 @@ def signal_handler(sig, frame):
8391
)
8492
args = parser.parse_args()
8593

86-
serve(args.addr)
94+
serve(args.addr)

‎extra/grpc/bark/ttsbark.yml‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
name: bark
2+
channels:
3+
- defaults
4+
dependencies:
5+
- _libgcc_mutex=0.1=main
6+
- _openmp_mutex=5.1=1_gnu
7+
- bzip2=1.0.8=h7b6447c_0
8+
- ca-certificates=2023.08.22=h06a4308_0
9+
- ld_impl_linux-64=2.38=h1181459_1
10+
- libffi=3.4.4=h6a678d5_0
11+
- libgcc-ng=11.2.0=h1234567_1
12+
- libgomp=11.2.0=h1234567_1
13+
- libstdcxx-ng=11.2.0=h1234567_1
14+
- libuuid=1.41.5=h5eee18b_0
15+
- ncurses=6.4=h6a678d5_0
16+
- openssl=3.0.11=h7f8727e_2
17+
- pip=23.2.1=py311h06a4308_0
18+
- python=3.11.5=h955ad1f_0
19+
- readline=8.2=h5eee18b_0
20+
- setuptools=68.0.0=py311h06a4308_0
21+
- sqlite=3.41.2=h5eee18b_0
22+
- tk=8.6.12=h1ccaba5_0
23+
- wheel=0.41.2=py311h06a4308_0
24+
- xz=5.4.2=h5eee18b_0
25+
- zlib=1.2.13=h5eee18b_0
26+
- pip:
27+
- accelerate==0.23.0
28+
- aiohttp==3.8.5
29+
- aiosignal==1.3.1
30+
- async-timeout==4.0.3
31+
- attrs==23.1.0
32+
- bark==0.1.5
33+
- boto3==1.28.61
34+
- botocore==1.31.61
35+
- certifi==2023.7.22
36+
- charset-normalizer==3.3.0
37+
- datasets==2.14.5
38+
- dill==0.3.7
39+
- einops==0.7.0
40+
- encodec==0.1.1
41+
- filelock==3.12.4
42+
- frozenlist==1.4.0
43+
- fsspec==2023.6.0
44+
- funcy==2.0
45+
- grpcio==1.59.0
46+
- huggingface-hub==0.16.4
47+
- idna==3.4
48+
- jinja2==3.1.2
49+
- jmespath==1.0.1
50+
- markupsafe==2.1.3
51+
- mpmath==1.3.0
52+
- multidict==6.0.4
53+
- multiprocess==0.70.15
54+
- networkx==3.1
55+
- numpy==1.26.0
56+
- nvidia-cublas-cu12==12.1.3.1
57+
- nvidia-cuda-cupti-cu12==12.1.105
58+
- nvidia-cuda-nvrtc-cu12==12.1.105
59+
- nvidia-cuda-runtime-cu12==12.1.105
60+
- nvidia-cudnn-cu12==8.9.2.26
61+
- nvidia-cufft-cu12==11.0.2.54
62+
- nvidia-curand-cu12==10.3.2.106
63+
- nvidia-cusolver-cu12==11.4.5.107
64+
- nvidia-cusparse-cu12==12.1.0.106
65+
- nvidia-nccl-cu12==2.18.1
66+
- nvidia-nvjitlink-cu12==12.2.140
67+
- nvidia-nvtx-cu12==12.1.105
68+
- packaging==23.2
69+
- pandas==2.1.1
70+
- peft==0.5.0
71+
- protobuf==4.24.4
72+
- psutil==5.9.5
73+
- pyarrow==13.0.0
74+
- python-dateutil==2.8.2
75+
- pytz==2023.3.post1
76+
- pyyaml==6.0.1
77+
- regex==2023.10.3
78+
- requests==2.31.0
79+
- rouge==1.0.1
80+
- s3transfer==0.7.0
81+
- safetensors==0.3.3
82+
- scipy==1.11.3
83+
- six==1.16.0
84+
- sympy==1.12
85+
- tokenizers==0.14.0
86+
- torch==2.1.0
87+
- torchaudio==2.1.0
88+
- tqdm==4.66.1
89+
- transformers==4.34.0
90+
- triton==2.1.0
91+
- typing-extensions==4.8.0
92+
- tzdata==2023.3
93+
- urllib3==1.26.17
94+
- xxhash==3.4.1
95+
- yarl==1.9.2
96+
prefix: /opt/conda/envs/bark

0 commit comments

Comments
 (0)