[Bug]: [V1] Molmo/Aria not supported on V1 due to xgrammar

[robertgshaw2-redhat]

Your current environment

Cannot use these models on V1 due to Xgrammar assert

🐛 Describe the bug

• run the following

VLLM_USE_V1=1 pytest -s -x models/decoder_only/vision_language/test_models.py -k molmo
VLLM_USE_V1=1 pytest -s -x models/decoder_only/vision_language/test_models.py -k aria

• get the following back

ERROR 03-10 03:06:35 [core.py:324] EngineCore hit an exception: Traceback (most recent call last):
ERROR 03-10 03:06:35 [core.py:324]   File "/home/rshaw/vllm/vllm/v1/engine/core.py", line 316, in run_engine_core
ERROR 03-10 03:06:35 [core.py:324]     engine_core = EngineCoreProc(*args, **kwargs)
ERROR 03-10 03:06:35 [core.py:324]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 03-10 03:06:35 [core.py:324]   File "/home/rshaw/vllm/vllm/v1/engine/core.py", line 271, in __init__
ERROR 03-10 03:06:35 [core.py:324]     super().__init__(vllm_config, executor_class, log_stats)
ERROR 03-10 03:06:35 [core.py:324]   File "/home/rshaw/vllm/vllm/v1/engine/core.py", line 65, in __init__
ERROR 03-10 03:06:35 [core.py:324]     self.structured_output_manager = StructuredOutputManager(vllm_config)
ERROR 03-10 03:06:35 [core.py:324]                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 03-10 03:06:35 [core.py:324]   File "/home/rshaw/vllm/vllm/v1/structured_output/__init__.py", line 44, in __init__
ERROR 03-10 03:06:35 [core.py:324]     tokenizer_info = xgr.TokenizerInfo.from_huggingface(
ERROR 03-10 03:06:35 [core.py:324]                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 03-10 03:06:35 [core.py:324]   File "/home/rshaw/vllm/venv/lib/python3.12/site-packages/xgrammar/tokenizer_info.py", line 188, in from_huggingface
ERROR 03-10 03:06:35 [core.py:324]     raise ValueError(msg)
ERROR 03-10 03:06:35 [core.py:324] ValueError: Input vocab_size less than minimum viable vocab size for tokenizer <class 'vllm.transformers_utils.tokenizer.get_cached_tokenizer.<locals>.CachedTokenizer'>.

Seems to be due to the relative sizes of the tokenizer vocab and model vocab

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[robertgshaw2-redhat]

note: the oracle PR will prevent users from launching with this

[russellb]

This PR will improve the situation: #14616

It won't be a problem unless you try to use structured output. Otherwise, vllm should start and operate normally.

We should still block this at a request level, but shouldn't need to at the engine startup level (after the PR goes in).

[russellb]

This PR will improve the situation: #14616

It won't be a problem unless you try to use structured output. Otherwise, vllm should start and operate normally.

We should still block this at a request level, but shouldn't need to at the engine startup level (after the PR goes in).

or I should actually make it more generic -- catch the exception when initialization fails, and reject the request.

[aarnphm]

will be addressed in #14630

[DarkLight1337]

Is this issue resolved? If so, let's close this and update #13726 accordingly

[russellb]

Is this issue resolved? If so, let's close this and update #13726 accordingly

@aarnphm says it should be. I'm testing now.

[russellb]

This is fixed. The tests pass, except one, but the failure is not structured output / xgrammar related:

...
>       raise ValueError(
            f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
            f"Model type should be one of {', '.join(c.__name__ for c in cls._model_mapping.keys())}."
        )
E       ValueError: Unrecognized configuration class <class 'transformers.models.aria.configuration_aria.AriaConfig'> for this kind of AutoModel: AutoModelForCausalLM.
E       Model type should be one of AriaTextConfig, BambaConfig, BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig, CamembertConfig, LlamaConfig, CodeGenConfig, CohereConfig, Cohere2Config, CpmAntConfig, CTRLConfig, Data2VecTextConfig, DbrxConfig, DiffLlamaConfig, ElectraConfig, Emu3Config, ErnieConfig, FalconConfig, FalconMambaConfig, FuyuConfig, GemmaConfig, Gemma2Config, GitConfig, GlmConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GPTJConfig, GraniteConfig, GraniteMoeConfig, JambaConfig, JetMoeConfig, LlamaConfig, MambaConfig, Mamba2Config, MarianConfig, MBartConfig, MegaConfig, MegatronBertConfig, MistralConfig, MixtralConfig, MllamaConfig, MoshiConfig, MptConfig, MusicgenConfig, MusicgenMelodyConfig, MvpConfig, NemotronConfig, OlmoConfig, Olmo2Config, OlmoeConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PersimmonConfig, PhiConfig, Phi3Config, PhimoeConfig, PLBartConfig, ProphetNetConfig, QDQBertConfig, Qwen2Config, Qwen2MoeConfig, RecurrentGemmaConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, RwkvConfig, Speech2Text2Config, StableLmConfig, Starcoder2Config, TransfoXLConfig, TrOCRConfig, WhisperConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, ZambaConfig.

../venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:567: ValueError
================================================================================================================================================== short test summary info ==================================================================================================================================================
FAILED models/decoder_only/vision_language/test_models.py::test_single_image_models[aria-test_case96] - ValueError: Unrecognized configuration class <class 'transformers.models.aria.configuration_aria.AriaConfig'> for this kind of AutoModel: AutoModelForCausalLM.