[Misc] Clean Up EngineArgs.create_engine_config
#13734
Conversation
Signed-off-by: [email protected] <[email protected]>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
robertgshaw2-redhat
added
the
ready
| # NOTE: This is to allow model loading from S3 in CI | ||
| if (not isinstance(self, AsyncEngineArgs) and envs.VLLM_CI_USE_S3 | ||
| and self.model in MODELS_ON_S3 | ||
| and self.load_format == LoadFormat.AUTO): # noqa: E501 |
There was a problem hiding this comment.
nit: is noqa needed? Line length looks fine to me
| self.quantization = self.load_format = "gguf" | ||
|
|
||
| # NOTE: This is to allow model loading from S3 in CI | ||
| if (not isinstance(self, AsyncEngineArgs) and envs.VLLM_CI_USE_S3 |
There was a problem hiding this comment.
can we move this outside and before create_model_config gets called since this is changing both model name and load_format (which gets referenced later on in create_load_config)
There was a problem hiding this comment.
oops, merged before I saw this. will fix in follow up
Signed-off-by: [email protected] <[email protected]> Signed-off-by: Louis Ulmer <[email protected]>
SUMMARY:
EngineArgs.create_engine_configby moving misc methods into helper methods to improve code readibility