[Usage]: Can not use vLLM in multiprocessing

[likaixin2000]

Your current environment

vllm==0.8.5.post1

How would you like to use vllm

I want to launch several processes to do vllm inference in parallel on multiple GPUs using multiprocessing.Pool, but this error occurs:

AssertionError: daemonic processes are not allowed to have children

Is my use case not supported anymore? It worked fine with earlier versions.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

Can you show your code?

[yinghai]

yeah are you launching your vllm process with something

p = Process(target=start_vllm, args=(args,), daemon=True)

?
if this then you will have this issue.

[likaixin2000]

Yes, I was using the following pattern and it worked well in some previous versions:

with multiprocessing.Pool():
    model = LLM(...)

I managed to get it working by launching the processes manually and setting daemon to False:

p = Process(target=start_vllm, args=(args,), daemon=False)