Support Anthropic API /v1/messages Endpoint #22627

LiuLi1998 · 2025-08-11T08:20:21Z

Relate issue: #21313
This PR adds support for the Anthropic /v1/messages REST API endpoint to the vLLM FastAPI server.

Support /v1/messages API
Compatibale with all existed tool call parser in OpenAI API

gemini-code-assist

Code Review

This pull request introduces support for the Anthropic Messages API by adding a new API server, protocol definitions, and a serving layer for format conversion. The implementation is based on the existing OpenAI-compatible server. My review has identified several critical and high-severity issues, including a potential NoneType access error, incorrect Pydantic model usage that could lead to validation errors, a risk of generating duplicate tool call IDs, and another case of incorrect attribute access on a Pydantic model that would cause a runtime error. I have provided specific code suggestions to address these issues and ensure the stability and correctness of the new endpoint.

gemini-code-assist · 2025-08-11T08:21:59Z

vllm/entrypoints/anthropic/api_server.py

When handler is None, messages(raw_request) is also None. Calling create_error_response on a None object will raise an AttributeError, causing an unhandled exception and a 500 server error. You should construct an ErrorResponse directly to ensure a proper error is returned. You will need to import ErrorResponse from vllm.entrypoints.openai.protocol and HTTPStatus from http.

Suggested change

return messages(raw_request).create_error_response(

message="The model does not support Chat Completions API")

return ErrorResponse(message="The model does not support Chat Completions API",

type="model_not_found",

code=HTTPStatus.NOT_FOUND.value)

gemini-code-assist · 2025-08-11T08:21:59Z

vllm/entrypoints/anthropic/serving_messages.py

anthropic_request.tool_choice is a Pydantic model instance, not a dictionary. Accessing its attributes should be done with dot notation (e.g., .name). Using .get("name") will result in an AttributeError at runtime.

Suggested change

"name": anthropic_request.tool_choice.get("name")

"name": anthropic_request.tool_choice.name

gemini-code-assist · 2025-08-11T08:21:59Z

vllm/entrypoints/anthropic/protocol.py

The id field is defined as a required field. The model_post_init method, which attempts to set a default value, is called after Pydantic's validation. If id is not provided during initialization, a ValidationError will be raised before model_post_init can execute. To correctly provide a default value for an optional field, you should use default_factory in the field definition and remove the model_post_init method.

Suggested change

id: str

id: str = Field(default_factory=lambda: f"msg_{int(time.time() * 1000)}")

gemini-code-assist · 2025-08-11T08:21:59Z

vllm/entrypoints/anthropic/serving_messages.py

Using int(time.time()) to generate tool call IDs is not safe as it can produce duplicate IDs for tool calls created in the same second. This can lead to incorrect behavior when matching tool calls to their results. It's better to use a UUID-based approach for uniqueness. You can use random_tool_call_id from vllm.entrypoints.chat_utils for this, which needs to be imported.

Suggested change

"id": block.id or f"call_{int(time.time())}",

"id": block.id or random_tool_call_id(),

gemini-code-assist · 2025-08-11T08:22:00Z

vllm/entrypoints/anthropic/serving_messages.py

A text AnthropicContentBlock is created even if generator.choices[0].message.content is None. This can lead to an invalid content block, as the Anthropic API requires the text field for text blocks. When serialized with exclude_none=True, this would result in an invalid content block. You should only create the text content block if there is content available.

Suggested change

content: List[AnthropicContentBlock] = [

AnthropicContentBlock(

type="text",

text=generator.choices[0].message.content

)

]

content: List[AnthropicContentBlock] = []

if generator.choices[0].message.content:

content.append(

AnthropicContentBlock(

type="text",

text=generator.choices[0].message.content))

github-actions · 2025-08-11T08:43:51Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

mgoin · 2025-08-11T19:09:07Z

Exciting! Make sure to add some unit tests before ready and I wonder if we can also include a smoke test that claude code/some other application can communicate with the API correctly

njhill · 2025-08-11T21:13:10Z

Thanks @LiuLi1998! Would you also be willing to help with ongoing support/maintenance of the API?

LiuLi1998 · 2025-08-12T02:41:43Z

Exciting! Make sure to add some unit tests before ready and I wonder if we can also include a smoke test that claude code/some other application can communicate with the API correctly

Thanks for the input! I agree — I’ll add some tests soon to make sure everything works as expected.

LiuLi1998 · 2025-08-12T02:43:32Z

Thanks @LiuLi1998! Would you also be willing to help with ongoing support/maintenance of the API?

Definitely! I’m glad to take part in the support/maintenance of the API

Signed-off-by: liuli <[email protected]>

LiuLi1998 · 2025-08-13T09:00:30Z

Exciting! Make sure to add some unit tests before ready and I wonder if we can also include a smoke test that claude code/some other application can communicate with the API correctly

I've added initial tests.I'm not entirely sure if the current approach follows best practices or covers everything needed—would really appreciate your feedback on improvements or any other cases

Signed-off-by: liuli <[email protected]>

mergify · 2025-08-18T05:20:45Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LiuLi1998.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

# Conflicts: # tests/utils.py

LiuLi1998 · 2025-08-20T06:37:48Z

@mgoin While adding tests, I triggered the CI and encountered the following error:
ModuleNotFoundError: No module named 'anthropic'.
Could someone advise how to add the required dependency to the project's requirements? Should I add anthropic to requirements.txt or requirements-test.txt (or another file)? Any guidance on the correct procedure would be appreciated!

Signed-off-by: liuli <[email protected]>

mergify · 2025-08-23T03:02:17Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LiuLi1998.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

requirements/common.txt

Signed-off-by: liuli <[email protected]>

LiuLi1998 · 2025-10-22T07:22:46Z

The failing test looks related. Might need python3 for CI env?

[2025-10-21T16:33:42Z] ERROR entrypoints/anthropic/test_messages.py::test_simple_messages - FileNotFoundError: [Errno 2] No such file or directory: 'python -m'

Already fixed, thx for help

Signed-off-by: liuli <[email protected]>

…ic_v2

Signed-off-by: liuli <[email protected]>

Signed-off-by: liuli <[email protected]> Co-authored-by: liuli <[email protected]> Co-authored-by: Cyrus Leung <[email protected]> Co-authored-by: Michael Goin <[email protected]> Signed-off-by: jorgentrondsen <[email protected]>

Signed-off-by: liuli <[email protected]> Co-authored-by: liuli <[email protected]> Co-authored-by: Cyrus Leung <[email protected]> Co-authored-by: Michael Goin <[email protected]>

Signed-off-by: liuli <[email protected]> Co-authored-by: liuli <[email protected]> Co-authored-by: Cyrus Leung <[email protected]> Co-authored-by: Michael Goin <[email protected]> Signed-off-by: Alberto Perdomo <[email protected]>

shoted · 2025-10-24T03:22:36Z

how to support both openai and anthropic api

tlipoca9 · 2025-10-24T07:36:33Z

how to support both openai and anthropic api

@LiuLi1998 +1, is it possible? I also want it

Signed-off-by: liuli <[email protected]> Co-authored-by: liuli <[email protected]> Co-authored-by: Cyrus Leung <[email protected]> Co-authored-by: Michael Goin <[email protected]>

Signed-off-by: liuli <[email protected]> Co-authored-by: liuli <[email protected]> Co-authored-by: Cyrus Leung <[email protected]> Co-authored-by: Michael Goin <[email protected]> Signed-off-by: 0xrushi <[email protected]>

LiuLi1998 · 2025-10-27T03:09:30Z

how to support both openai and anthropic api

@LiuLi1998 +1, is it possible? I also want it

Currently, the OpenAI and Anthropic APIs are separate api servers and cannot be used at the same time.

shoted · 2025-10-28T07:03:26Z

how to support both openai and anthropic api如何同时支持 OpenAI 和 Anthropic API

@LiuLi1998 +1, is it possible? I also want it+1，有可能吗？我也想要

Currently, the OpenAI and Anthropic APIs are separate api servers and cannot be used at the same time.目前，OpenAI 和 Anthropic API 是独立的 api 服务器，不能同时使用。

Is there a plan to support them simultaneously?

youkaichao · 2025-10-29T14:53:38Z

vllm/entrypoints/anthropic/serving_messages.py

+        See https://docs.anthropic.com/en/api/messages
+        for the API specification. This API mimics the Anthropic messages API.
+        """
+        logger.debug("Received messages request %s", request.model_dump_json())


would this be super slow? it calls request.model_dump_json() unconditionally.

this will only work when VLLM_LOGGING_LEVEL=DEBUG

youkaichao · 2025-10-29T14:57:31Z

vllm/entrypoints/anthropic/api_server.py

+        return JSONResponse(content=generator.model_dump())
+
+    elif isinstance(generator, AnthropicMessagesResponse):
+        logger.debug(


similar problem, unconditional call of generator.model_dump(exclude_none=True)

youkaichao · 2025-10-29T15:09:15Z

vllm/entrypoints/anthropic/api_server.py

+
+
+@router.post(
+    "/v1/messages",


it seems anthropic API only has a new /v1/messages endpoint, why not merge it with the openai server? like serving both v1/chat/completions and /v1/messages endpoints together.

I think they are two different protocols, and It's possible to merge them together for functional compatibility, but I think it could lead to semantic confusion.

I don't think Kaichao is saying to merge them into one endpoint, just hosting them side-by-side. So when you run vllm serve you get /v1/completions, /v1/chat/completions, /v1/messages, etc. I agree this would be optimal for user ease

I don't think Kaichao is saying to merge them into one endpoint, just hosting them side-by-side. So when you run vllm serve you get /v1/completions, /v1/chat/completions, /v1/messages, etc. I agree this would be optimal for user ease

I agree it's the most user-friendly solution.

bbartels · 2025-10-31T15:07:00Z

@youkaichao @mgoin @shoted Raised #27882 to add /v1/messages to openai api_server

shoted · 2025-11-04T02:15:46Z

@youkaichao @mgoin @shoted Raised #27882 to add /v1/messages to openai api_server

nice， bro

Signed-off-by: liuli <[email protected]> Co-authored-by: liuli <[email protected]> Co-authored-by: Cyrus Leung <[email protected]> Co-authored-by: Michael Goin <[email protected]>

LiuLi1998 requested a review from aarnphm as a code owner August 11, 2025 08:20

mergify bot added the frontend label Aug 11, 2025

gemini-code-assist bot reviewed Aug 11, 2025

View reviewed changes

mgoin self-requested a review August 11, 2025 19:09

LiuLi1998 requested review from DarkLight1337, robertgshaw2-redhat and simon-mo as code owners August 13, 2025 08:36

Liliu1997 added 2 commits August 13, 2025 16:37

support anthropic endpoint

20f42bd

Signed-off-by: liuli <[email protected]>

add antropic api test

82851a3

Signed-off-by: liuli <[email protected]>

LiuLi1998 force-pushed the dev/antropic_v2 branch from 4465c0f to 82851a3 Compare August 13, 2025 08:38

add antropic unit test

34f17fc

Signed-off-by: liuli <[email protected]>

add spdx-header

0697fe8

Signed-off-by: liuli <[email protected]>

mgoin changed the title ~~Support Anthropic API Endponit~~ Support Anthropic API /v1/messages Endpoint Aug 13, 2025

Liliu1997 added 2 commits August 14, 2025 13:34

stream bug fix

9b3fc78

Signed-off-by: liuli <[email protected]>

python lint fix

d5b9860

Signed-off-by: liuli <[email protected]>

mergify bot added the needs-rebase label Aug 18, 2025

Merge remote-tracking branch 'origin/main'

8f54993

# Conflicts: # tests/utils.py

mergify bot removed the needs-rebase label Aug 20, 2025

some pylint fix

78b1601

Signed-off-by: liuli <[email protected]>

mergify bot added the ci/build label Aug 20, 2025

mergify bot added the needs-rebase label Aug 23, 2025

mgoin reviewed Oct 21, 2025

View reviewed changes

requirements/common.txt Outdated Show resolved Hide resolved

unit-test fix

78b6608

Signed-off-by: liuli <[email protected]>

auto-merge was automatically disabled October 22, 2025 07:11
Head branch was pushed to by a user without write access

merge master

904fa56

Signed-off-by: liuli <[email protected]>

LiuLi1998 force-pushed the dev/antropic_v2 branch from fa40146 to 904fa56 Compare October 22, 2025 07:15

Liliu1997 added 5 commits October 22, 2025 15:45

unit-test fix

1f8c940

Signed-off-by: liuli <[email protected]>

fix unit test

fc96a55

Signed-off-by: liuli <[email protected]>

Merge remote-tracking branch 'origin/dev/antropic_v2' into dev/antrop…

26816e4

…ic_v2

fix unit test

037f0d1

Signed-off-by: liuli <[email protected]>

fix unit test

ddc9645

Signed-off-by: liuli <[email protected]>

simon-mo merged commit c9461e0 into vllm-project:main Oct 22, 2025
87 checks passed

youkaichao reviewed Oct 29, 2025

View reviewed changes

mgoin mentioned this pull request Nov 4, 2025

[Feature]: Support Anthropic API /v1/messages endpoint #21313

Closed

1 task

-        return messages(raw_request).create_error_response(
-            message="The model does not support Chat Completions API")
+        return ErrorResponse(message="The model does not support Chat Completions API",
+                             type="model_not_found",
+                             code=HTTPStatus.NOT_FOUND.value)

	"name": anthropic_request.tool_choice.get("name")
	"name": anthropic_request.tool_choice.name

	id: str
	id: str = Field(default_factory=lambda: f"msg_{int(time.time() * 1000)}")

	"id": block.id or f"call_{int(time.time())}",
	"id": block.id or random_tool_call_id(),

-        content: List[AnthropicContentBlock] = [
-            AnthropicContentBlock(
-                type="text",
-                text=generator.choices[0].message.content
-            )
-        ]
+        content: List[AnthropicContentBlock] = []
+        if generator.choices[0].message.content:
+            content.append(
+                AnthropicContentBlock(
+                    type="text",
+                    text=generator.choices[0].message.content))



		@router.post(
		"/v1/messages",

Uh oh!

Support Anthropic API /v1/messages Endpoint #22627

Support Anthropic API /v1/messages Endpoint #22627

Uh oh!

Conversation

LiuLi1998 commented Aug 11, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Aug 11, 2025

Uh oh!

mgoin commented Aug 11, 2025

Uh oh!

njhill commented Aug 11, 2025

Uh oh!

LiuLi1998 commented Aug 12, 2025

Uh oh!

LiuLi1998 commented Aug 12, 2025

Uh oh!

LiuLi1998 commented Aug 13, 2025

Uh oh!

mergify bot commented Aug 18, 2025

Uh oh!

LiuLi1998 commented Aug 20, 2025

Uh oh!

mergify bot commented Aug 23, 2025

Uh oh!

Uh oh!

LiuLi1998 commented Oct 22, 2025

Uh oh!

Uh oh!

shoted commented Oct 24, 2025

Uh oh!

tlipoca9 commented Oct 24, 2025

Uh oh!

LiuLi1998 commented Oct 27, 2025

Uh oh!

shoted commented Oct 28, 2025

Uh oh!

youkaichao Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

LiuLi1998 Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

youkaichao Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

youkaichao Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

LiuLi1998 Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

mgoin Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

LiuLi1998 Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

bbartels commented Oct 31, 2025

Uh oh!

LiuLi1998 commented Aug 11, 2025 •

edited by github-actions bot

Loading