image size excessive

[szaimen]

Hi, on v2.0.0 the image size is excessive with 36.9 GB of compressed size on quay.io.

See

Thus I would like to use the v2.0.0-ffmpeg-core image. But I guess I would need to download the python dependencies manually. How would I do that? I could not find instructions on this.

Thanks for your help!

---

Edit:

Apparently these are the main differences that cause the image size difference:

LocalAI/Dockerfile

Lines 162 to 194 in adfa7aa

	RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
	PATH=$PATH:/opt/conda/bin make -C backend/python/autogptq \
	; fi
	RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
	PATH=$PATH:/opt/conda/bin make -C backend/python/bark \
	; fi
	RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
	PATH=$PATH:/opt/conda/bin make -C backend/python/diffusers \
	; fi
	RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
	PATH=$PATH:/opt/conda/bin make -C backend/python/vllm \
	; fi
	RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
	PATH=$PATH:/opt/conda/bin make -C backend/python/sentencetransformers \
	; fi
	RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
	PATH=$PATH:/opt/conda/bin make -C backend/python/transformers \
	; fi
	RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
	PATH=$PATH:/opt/conda/bin make -C backend/python/vall-e-x \
	; fi
	RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
	PATH=$PATH:/opt/conda/bin make -C backend/python/exllama \
	; fi
	RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
	PATH=$PATH:/opt/conda/bin make -C backend/python/exllama2 \
	; fi
	RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
	PATH=$PATH:/opt/conda/bin make -C backend/python/petals \
	; fi
	RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
	PATH=$PATH:/opt/conda/bin make -C backend/python/transformers-musicgen \
	; fi

[mudler]

indeed we have -core images which don't bring the extra python deps (with the cost of course, that the python-based backends won't be available) exactly for this reason. See also: https://localai.io/basics/getting_started/#container-images

[szaimen]

indeed we have -core images which don't bring the extra python deps (with the cost of course, that the python-based backends won't be available) exactly for this reason. See also: localai.io/basics/getting_started/#container-images

Thanks! But how would I add the python-based backends manually? I could not find docs on this or is it the commands of the Dockerfile that I found?

[mudler]

indeed we have -core images which don't bring the extra python deps (with the cost of course, that the python-based backends won't be available) exactly for this reason. See also: localai.io/basics/getting_started/#container-images

Thanks! But how would I add the python-based backends manually? I could not find docs on this or is it the commands of the Dockerfile that I found?

It really depends on how you are deploying LocalAI - if you are reusing LocalAI images and base yours on top you can just run the make commands to prepare the dependencies for the backend you wish to use like in the Dockerfile (e.g. PATH=$PATH:/opt/conda/bin make -C backend/python/diffusers ). Alternatively I was thinking to push separate images for each backend, but my guess is that would put even more stress on our CI.

[szaimen]

It really depends on how you are deploying LocalAI - if you are reusing LocalAI images and base yours on top you can just run the make commands to prepare the dependencies for the backend you wish to use like in the Dockerfile (e.g. PATH=$PATH:/opt/conda/bin make -C backend/python/diffusers ).

Okay thanks!

[mudler]

It really depends on how you are deploying LocalAI - if you are reusing LocalAI images and base yours on top you can just run the make commands to prepare the dependencies for the backend you wish to use like in the Dockerfile (e.g. PATH=$PATH:/opt/conda/bin make -C backend/python/diffusers ).

Okay thanks!

thanks for highlighting this actually - I think docs needs a bit of love around that area especially now that we support more backends this is something everyone most likely will bump into, will try to cover it in the next docs updates I have flying around here.