-
-
Notifications
You must be signed in to change notification settings - Fork 11.4k
[Bugfix] reasoning_parser parameter handling in run_batch.py #26225
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bugfix] reasoning_parser parameter handling in run_batch.py #26225
Conversation
Signed-off-by: inc-jeong <[email protected]>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run You ask your reviewers to trigger select CI tests on top of Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. 🚀 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request aims to fix an issue with reasoning_parser handling in run_batch.py. The intention is correct, adding validation and passing the parameter to OpenAIServingChat. However, the implementation incorrectly assumes a nested structure for the parsed command-line arguments, which will lead to an AttributeError. The args object is a flat namespace, and the reasoning_parser value should be accessed directly from it for validation. For initializing OpenAIServingChat, it's best to use the vllm_config object, which holds the definitive engine configuration. I've provided suggestions to correct these paths.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Signed-off-by: InChang Jeong <[email protected]>
|
Hello @aarnphm, @chaunceyjiang, could you please take a look when you have a moment? |
chaunceyjiang
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Could you write an e2e for this?
|
@chaunceyjiang , thank you for review.
Do you mean to write the test code in the https://github.com/vllm-project/vllm/tree/main/tests/entrypoints/openai directory? |
Yes. |
Signed-off-by: inc-jeong <[email protected]>
|
@chaunceyjiang , |
chaunceyjiang
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks~
|
@chaunceyjiang , It seems the failure is occurring during the CI stage — could you help me look into it? |
I retried it once — let’s see if it still fails. |
Signed-off-by: inc-jeong <[email protected]>
…oject#26225) Signed-off-by: inc-jeong <[email protected]> Signed-off-by: InChang Jeong <[email protected]> Co-authored-by: USER <[email protected]>
…oject#26225) Signed-off-by: inc-jeong <[email protected]> Signed-off-by: InChang Jeong <[email protected]> Co-authored-by: USER <[email protected]> Signed-off-by: Alberto Perdomo <[email protected]>
…oject#26225) Signed-off-by: inc-jeong <[email protected]> Signed-off-by: InChang Jeong <[email protected]> Co-authored-by: USER <[email protected]> Signed-off-by: Alberto Perdomo <[email protected]>
…oject#26225) Signed-off-by: inc-jeong <[email protected]> Signed-off-by: InChang Jeong <[email protected]> Co-authored-by: USER <[email protected]>
…oject#26225) Signed-off-by: inc-jeong <[email protected]> Signed-off-by: InChang Jeong <[email protected]> Co-authored-by: USER <[email protected]>
…oject#26225) Signed-off-by: inc-jeong <[email protected]> Signed-off-by: InChang Jeong <[email protected]> Co-authored-by: USER <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>
…oject#26225) Signed-off-by: inc-jeong <[email protected]> Signed-off-by: InChang Jeong <[email protected]> Co-authored-by: USER <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>
…oject#26225) Signed-off-by: inc-jeong <[email protected]> Signed-off-by: InChang Jeong <[email protected]> Co-authored-by: USER <[email protected]> Signed-off-by: 0xrushi <[email protected]>
…oject#26225) Signed-off-by: inc-jeong <[email protected]> Signed-off-by: InChang Jeong <[email protected]> Co-authored-by: USER <[email protected]> Signed-off-by: 0xrushi <[email protected]>
…oject#26225) Signed-off-by: inc-jeong <[email protected]> Signed-off-by: InChang Jeong <[email protected]> Co-authored-by: USER <[email protected]>
…oject#26225) Signed-off-by: inc-jeong <[email protected]> Signed-off-by: InChang Jeong <[email protected]> Co-authored-by: USER <[email protected]>
Purpose
This PR fixes the issue where the
reasoning_parserparameter was not being properly handled in run_batch.py, causing reasoning parser functionality to not work correctly during batch processing.related issue : #26224
Test Plan
Test Result
As-is
To-be
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.