[Bug]: AttributeError: 'TokenizeChatRequest' object has no attribute 'mm_processor_kwargs'

[emirhanKural]

Issue with OpenAI Tokenization Endpoint, v0.7.3

Hello,

I am encountering an error when using OpenAI's tokenization endpoint.

• It works correctly when the input is a string.
• However, when I provide a message, it returns an error.

Could you help me resolve this issue?

Thanks!

vllm serve llm --tensor-parallel-size 2 --max-num-seqs 16 --gpu-memory-utilization 0.99

cURL :

curl -X POST \ 'http://localhost:8000/tokenize' \ --header 'Accept: */*' \ --header 'Content-Type: application/json' \ --data-raw '{ "model": "llm", "messages": [ { "role": "user", "content": "beşi beş kuruştan beş yumurta kaç kuruş eder? detaylı açıkla." } ] } '

🐛 Describe the bug

ERROR: Exception in ASGI application Traceback (most recent call last): File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi result = await app( # type: ignore[func-returns-value] File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ return await self.app(scope, receive, send) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/applications.py", line 123, in __call__ await self.middleware_stack(scope, receive, send) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/middleware/errors.py", line 186, in __call__ raise exc File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/middleware/errors.py", line 164, in __call__ await self.app(scope, receive, _send) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/middleware/cors.py", line 85, in __call__ await self.app(scope, receive, send) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/middleware/exceptions.py", line 65, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/routing.py", line 756, in __call__ await self.middleware_stack(scope, receive, send) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/routing.py", line 776, in app await route.handle(scope, receive, send) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/routing.py", line 297, in handle await self.app(scope, receive, send) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/routing.py", line 77, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/starlette/routing.py", line 72, in app response = await func(request) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/fastapi/routing.py", line 278, in app raw_response = await run_endpoint_function( File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/fastapi/routing.py", line 191, in run_endpoint_function return await dependant.call(**values) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/vllm/entrypoints/utils.py", line 56, in wrapper return handler_task.result() File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/vllm/entrypoints/openai/api_server.py", line 361, in tokenize generator = await handler.create_tokenize(request, raw_request) File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/vllm/entrypoints/openai/serving_tokenization.py", line 71, in create_tokenize ) = await self._preprocess_chat( File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/vllm/entrypoints/openai/serving_engine.py", line 454, in _preprocess_chat if request.mm_processor_kwargs is not None: File "/home/103801/venv_vectoryllm_deneme/lib64/python3.9/site-packages/pydantic/main.py", line 891, in __getattr__ raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}') AttributeError: 'TokenizeChatRequest' object has no attribute 'mm_processor_kwargs'

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

It should have been fixed by #13644 already, can you pull the latest code?

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!