Add a default image prefiller implementation #13310

larryliu0820 · 2025-08-12T04:22:45Z

Summary: As titled. I need to create an interface IModule for Module class to override, to make it test-able.

Differential Revision: D80063769

cc @mergennachin @cccclai @helunwencser @jackzhxng

pytorch-bot · 2025-08-12T04:22:48Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13310

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 28 New Failures, 1 Unrelated Failure

As of commit 6b1ff95 with merge base cf669e3 ():

NEW FAILURES - The following jobs have failed:

Build documentation / build (buck2) / Build doc (gh)
At least one of the pre-conditions you specified did not hold
Build Presets / apple (ios-simulator) / build (gh)
<command line>:3:9: error: macro name must be an identifier
Build Presets / apple (ios) / build (gh)
<command line>:3:9: error: macro name must be an identifier
Build Presets / apple (llm) / build (gh)
<command line>:1:9: error: macro name must be an identifier
Build Presets / apple (macos) / build (gh)
<command line>:3:9: error: macro name must be an identifier
Build Presets / linux (linux, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
<command line>:1:9: error: macro name must be an identifier
Build Presets / linux (linux, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / build (gh)
RuntimeError: Command docker exec -t 887ccd245b49931b9b48b311ed66a9cf03fa930380d1b2cbc665d4b426077e0d /exec failed with exit code 2
Build Presets / linux (llm, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
<command line>:1:9: error: macro name must be an identifier
Build Presets / linux (llm, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / build (gh)
pull / android / build-llm-demo / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / test-llama-lora-linux / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
RuntimeError: Command docker exec -t bbc4d977c84303bb1b749044e3dcff700b1db9587c864f408cf41838cf248a8c /exec failed with exit code 2
pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
RuntimeError: Command docker exec -t 1491a225e58baa333ab7d70cd06274c11776ec92e0329cafe083130466cc4783 /exec failed with exit code 2
pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
RuntimeError: Command docker exec -t 63b0754d7ccfec8537beba041f834caea9208899aadf98ce6257a51c0c79333f /exec failed with exit code 2
pull / test-llama-runner-linux-android / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / test-llava-runner-linux / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / test-phi-3-mini-runner-linux / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / test-static-llama-qnn-linux / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / unittest / linux / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / unittest / macos / macos-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / unittest-editable / linux / linux-job (gh)
<command line>:1:9: error: macro name must be an identifier
pull / unittest-editable / macos / macos-job (gh)
<command line>:1:9: error: macro name must be an identifier

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / test-binary-size-linux-gcc / linux-job (gh) (trunk failure)
/pytorch/executorch/kernels/portable/cpu/op_stack.cpp:129:26: error: comparison of integer expressions of different signedness: ‘size_t’ {aka ‘long unsigned int’} and ‘ssize_t’ {aka ‘long int’} [-Werror=sign-compare]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-08-12T04:22:54Z

This pull request was exported from Phabricator. Differential Revision: D80063769

github-actions · 2025-08-12T04:23:26Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Differential Revision: D80063769

facebook-github-bot · 2025-08-12T16:30:39Z

This pull request was exported from Phabricator. Differential Revision: D80063769

jackzhxng · 2025-08-12T21:01:24Z

extension/llm/runner/image_prefiller.cpp

+ * LICENSE file in the root directory of this source tree.
+ */
+
+// Given a image tensor, prefill the KV cache of LLaVA.


Remove LLaVA

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Differential Revision: D80063769

facebook-github-bot · 2025-08-14T06:36:55Z

This pull request was exported from Phabricator. Differential Revision: D80063769

extension/llm/runner/multimodal_prefiller.cpp

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Differential Revision: D80063769

facebook-github-bot · 2025-08-14T23:38:57Z

This pull request was exported from Phabricator. Differential Revision: D80063769

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Differential Revision: D80063769

facebook-github-bot · 2025-08-15T00:01:19Z

This pull request was exported from Phabricator. Differential Revision: D80063769

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Differential Revision: D80063769

facebook-github-bot · 2025-08-15T00:03:34Z

This pull request was exported from Phabricator. Differential Revision: D80063769

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Reviewed By: jackzhxng Differential Revision: D80063769

facebook-github-bot · 2025-08-15T19:07:34Z

This pull request was exported from Phabricator. Differential Revision: D80063769

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Reviewed By: jackzhxng Differential Revision: D80063769

facebook-github-bot · 2025-08-15T19:54:05Z

This pull request was exported from Phabricator. Differential Revision: D80063769

facebook-github-bot · 2025-08-15T19:57:43Z

This pull request was exported from Phabricator. Differential Revision: D80063769

Summary: Pull Request resolved: #13310 As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Reviewed By: jackzhxng Differential Revision: D80063769

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Reviewed By: jackzhxng Differential Revision: D80063769

facebook-github-bot · 2025-08-16T03:22:28Z

This pull request was exported from Phabricator. Differential Revision: D80063769

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Reviewed By: jackzhxng Differential Revision: D80063769

facebook-github-bot · 2025-08-16T06:37:55Z

This pull request was exported from Phabricator. Differential Revision: D80063769

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Reviewed By: jackzhxng Differential Revision: D80063769

facebook-github-bot · 2025-08-16T07:11:36Z

This pull request was exported from Phabricator. Differential Revision: D80063769

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Reviewed By: jackzhxng Differential Revision: D80063769

facebook-github-bot · 2025-08-16T07:46:01Z

This pull request was exported from Phabricator. Differential Revision: D80063769

larryliu0820 requested review from kirklandsign, shoumikhin, jackzhxng, swolchok and mergennachin as code owners August 12, 2025 04:22

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 12, 2025

facebook-github-bot added the fb-exported label Aug 12, 2025

larryliu0820 added the module: llm Issues related to LLM examples and apps, and to the extensions/llm/ code label Aug 12, 2025

facebook-github-bot pushed a commit that referenced this pull request Aug 12, 2025

Add a default image prefiller implementation (#13310)

96ed832

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Differential Revision: D80063769

facebook-github-bot force-pushed the export-D80063769 branch from 1356bd1 to 96ed832 Compare August 12, 2025 16:30

jackzhxng approved these changes Aug 12, 2025

View reviewed changes

jackzhxng reviewed Aug 12, 2025

View reviewed changes

facebook-github-bot pushed a commit that referenced this pull request Aug 14, 2025

Add a default image prefiller implementation (#13310)

096c0a9

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Differential Revision: D80063769

facebook-github-bot force-pushed the export-D80063769 branch from 96ed832 to 096c0a9 Compare August 14, 2025 06:36

jackzhxng reviewed Aug 14, 2025

View reviewed changes

extension/llm/runner/multimodal_prefiller.cpp Show resolved Hide resolved

extension/llm/runner/multimodal_prefiller.cpp Outdated Show resolved Hide resolved

facebook-github-bot pushed a commit that referenced this pull request Aug 14, 2025

Add a default image prefiller implementation (#13310)

ab4c774

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Differential Revision: D80063769

facebook-github-bot force-pushed the export-D80063769 branch from 096c0a9 to ab4c774 Compare August 14, 2025 23:38

facebook-github-bot pushed a commit that referenced this pull request Aug 15, 2025

Add a default image prefiller implementation (#13310)

1afc7c3

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Differential Revision: D80063769

facebook-github-bot force-pushed the export-D80063769 branch from ab4c774 to 1afc7c3 Compare August 15, 2025 00:01

facebook-github-bot pushed a commit that referenced this pull request Aug 15, 2025

Add a default image prefiller implementation (#13310)

d2adca1

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Differential Revision: D80063769

facebook-github-bot force-pushed the export-D80063769 branch from 1afc7c3 to d2adca1 Compare August 15, 2025 00:03

jackzhxng approved these changes Aug 15, 2025

View reviewed changes

facebook-github-bot force-pushed the export-D80063769 branch from d2adca1 to 07fc7f8 Compare August 15, 2025 19:07

facebook-github-bot force-pushed the export-D80063769 branch from 07fc7f8 to eab0801 Compare August 15, 2025 19:53

larryliu0820 force-pushed the export-D80063769 branch from eab0801 to 1eddda4 Compare August 15, 2025 19:53

larryliu0820 force-pushed the export-D80063769 branch from 1eddda4 to 5100909 Compare August 15, 2025 19:57

larryliu0820 mentioned this pull request Aug 16, 2025

Add a generic multimodal runner #13166

Open

facebook-github-bot force-pushed the export-D80063769 branch from 5100909 to 2ba180b Compare August 16, 2025 03:22

facebook-github-bot force-pushed the export-D80063769 branch from 2ba180b to 1e20634 Compare August 16, 2025 06:37

facebook-github-bot force-pushed the export-D80063769 branch from 1e20634 to 6229f20 Compare August 16, 2025 07:11

Add a default image prefiller implementation (#13310)

6b1ff95

Summary: As titled. I need to create an interface `IModule` for `Module` class to override, to make it test-able. Reviewed By: jackzhxng Differential Revision: D80063769

facebook-github-bot force-pushed the export-D80063769 branch from 6229f20 to 6b1ff95 Compare August 16, 2025 07:45

Add a default image prefiller implementation #13310

Are you sure you want to change the base?

Add a default image prefiller implementation #13310

Conversation

larryliu0820 commented Aug 12, 2025 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13310

❌ 28 New Failures, 1 Unrelated Failure

Uh oh!

facebook-github-bot commented Aug 12, 2025

Uh oh!

github-actions bot commented Aug 12, 2025

This PR needs a release notes: label

Uh oh!

facebook-github-bot commented Aug 12, 2025

Uh oh!

jackzhxng Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Aug 14, 2025

Uh oh!

Uh oh!

Uh oh!

facebook-github-bot commented Aug 14, 2025

Uh oh!

facebook-github-bot commented Aug 15, 2025

Uh oh!

facebook-github-bot commented Aug 15, 2025

Uh oh!

facebook-github-bot commented Aug 15, 2025

Uh oh!

facebook-github-bot commented Aug 15, 2025

Uh oh!

facebook-github-bot commented Aug 15, 2025

Uh oh!

facebook-github-bot commented Aug 16, 2025

Uh oh!

facebook-github-bot commented Aug 16, 2025

Uh oh!

facebook-github-bot commented Aug 16, 2025

Uh oh!

facebook-github-bot commented Aug 16, 2025

Uh oh!

Uh oh!

larryliu0820 commented Aug 12, 2025 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Aug 12, 2025 •

edited

Loading

This PR needs a `release notes:` label

jackzhxng Aug 12, 2025 •

edited

Loading