[Speculators] Move tests + fix integration

[dsikka]

This PR fixes the speculator model integration that enables the simplified vllm serve <speculator-model> command and ensures compatibility with S3 models in CI.

Background

The speculator integration allows users to run speculative decoding without explicitly providing a --speculative-config by automatically detecting speculator models and extracting the configuration. This integration kept breaking on main because:

1. Tests were located in tests/speculative_decoding/speculators/ which doesn't run in CI
2. The integration didn't properly handle S3 models used in CI testing

Changes

1. Moved Speculator Tests to CI-Monitored Directory

• Before: tests/speculative_decoding/speculators/test_eagle3.py
• After: tests/v1/spec_decode/test_speculators_eagle3.py

This ensures tests run in CI and prevent future breakage.

2. Fixed S3 Model Compatibility

Problem:

• Speculator detection (maybe_override_with_speculators()) was moved before create_model_config() to properly detect speculators before creating the model config
• However, HuggingFace's PretrainedConfig.get_config_dict() cannot load configs from S3 URLs (s3://...)
• This caused failures when running with S3 models in CI

Solution:
Skip speculator auto-detection for S3 models:

# Skip speculator detection for S3 models since HuggingFace cannot load
# configs directly from S3 URLs. S3 models can still use speculators with
# explicit --speculative-config.
if not is_s3(self.model):
    (self.model, self.tokenizer, self.speculative_config) = (
        maybe_override_with_speculators(...)
    )

Trade-off: S3 models cannot use automatic speculator detection but can still use speculators via explicit --speculative-config argument.

3. Added Comprehensive Integration Test

Added test_speculators_model_integration() in tests/v1/e2e/test_spec_decode.py to validate the simplified integration path:

What it tests:

• ✅ Speculator model auto-detection without explicit config
• ✅ Speculative config correctly extracted from model
• ✅ Verifier model properly identified
• ✅ Draft model set to speculator model
• ✅ Text generation works with speculative decoding
• ✅ Output correctness (66% match threshold vs reference)

Test models:

• nm-testing/SpeculatorLlama3-1-8B-Eagle3-converted-0717-quantized
• nm-testing/Speculator-Qwen3-8B-Eagle3-converted-071-quantized

Compatibility Matrix

Model Type	Speculator Auto-Detection	Manual --speculative-config
HuggingFace models	✅ Yes	✅ Yes
Local models	✅ Yes	✅ Yes
S3 models (CI)	❌ No	✅ Yes

Testing

Run the new test:

python -m pytest tests/v1/e2e/test_spec_decode.py::test_speculators_model_integration -v

Run specific variant:

python -m pytest tests/v1/e2e/test_spec_decode.py::test_speculators_model_integration[llama3_eagle3_speculator] -v

Files Changed

tests/v1/e2e/test_spec_decode.py | 79 ++++++++++++++++++++++++++
vllm/engine/arg_utils.py         | 22 ++++++--
2 files changed, 92 insertions(+), 9 deletions(-)

Related Issues

Fixes the issue where speculator integration tests weren't running in CI, preventing detection of breaking changes to the vllm serve <speculator-model> integration.

[dsikka]

One quick change - sorry!

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @dsikka.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[bkauf]

This PR has broken support for gs:// which is used by the Run AI model streamer