[Bugfix][CPU] fix missing input intermediate_tensors in the cpu_model_runner (vllm-project#8733)

bigPYJ1151 · sumitd2 · commit e48f208fd5a4 · 2024-11-14T10:34:27.000-05:00
Signed-off-by: Sumit Dubey &lt;sumit.dubey2@ibm.com&gt;
diff --git a/vllm/worker/cpu_model_runner.py b/vllm/worker/cpu_model_runner.py
@@ -456,6 +456,8 @@ def execute_model(
             model_input.attn_metadata,
             **MultiModalInputs.as_kwargs(model_input.multi_modal_kwargs or {},
                                          device=self.device),
+            "intermediate_tensors":
+            intermediate_tensors,
         }
 
         hidden_states = model_executable(**execute_model_kwargs)

Original file line number	Diff line number	Diff line change
`@@ -456,6 +456,8 @@ def execute_model(`
`456`	`456`	`model_input.attn_metadata,`
`457`	`457`	`**MultiModalInputs.as_kwargs(model_input.multi_modal_kwargs or {},`
`458`	`458`	`device=self.device),`
	`459`	`+ "intermediate_tensors":`
	`460`	`+ intermediate_tensors,`
`459`	`461`	`}`
`460`	`462`
`461`	`463`	`hidden_states = model_executable(**execute_model_kwargs)`