Skip to content

[Quant][PT2E][X86] Enable annotation of aten.mul.tensor with X86InductorQuantizer #150831

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from

Conversation

Xia-Weiwen
Copy link
Collaborator

@Xia-Weiwen Xia-Weiwen commented Apr 8, 2025

Stack from ghstack (oldest at bottom):

Summary
This PR adds support of annotation of aten.mul.tensor in X86InductorQuantizer.
mul is not annotated by default. Users need to set the following to enable annotation of mul:

quantizer.set_function_type_qconfig(
    torch.mul, quantizer.get_global_quantization_config()
)

After convert_pt2e, users get patterns like

quantize_per_tensor_default = torch.ops.quantized_decomposed.quantize_per_tensor.default(x, ...)
dequantize_per_tensor_default = torch.ops.quantized_decomposed.dequantize_per_tensor.default(quantize_per_tensor_default, ...)
quantize_per_tensor_default_1 = torch.ops.quantized_decomposed.quantize_per_tensor.default(y, ...);
dequantize_per_tensor_default_1 = torch.ops.quantized_decomposed.dequantize_per_tensor.default(quantize_per_tensor_default_1, ...)
mul = torch.ops.aten.mul.Tensor(dequantize_per_tensor_default, dequantize_per_tensor_default_1);

Test plan

pytest test/quantization/pt2e/test_x86inductor_quantizer.py -k test_annotate_mul_tensor

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10

[ghstack-poisoned]
@Xia-Weiwen Xia-Weiwen requested a review from jerryzh168 as a code owner April 8, 2025 07:37
Copy link

pytorch-bot bot commented Apr 8, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/150831

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 71e079b with merge base 01f226b (image):

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the release notes: quantization release notes category label Apr 8, 2025
Xia-Weiwen added a commit that referenced this pull request Apr 8, 2025
…torQuantizer

ghstack-source-id: b336298
Pull Request resolved: #150831
@Xia-Weiwen Xia-Weiwen marked this pull request as draft April 8, 2025 07:38
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (1)

test/quantization/pt2e/test_x86inductor_quantizer.py:2877

  • [nitpick] Avoid using 'type' as a variable name because it shadows the built-in function. Consider renaming it to 'model_type' or a similar descriptive name.
for type in [0, 1, 2]:

@Xia-Weiwen Xia-Weiwen added the intel This tag is for PR from Intel label Apr 8, 2025
[ghstack-poisoned]
Xia-Weiwen added a commit that referenced this pull request Apr 9, 2025
…torQuantizer

ghstack-source-id: 23478a7
Pull Request resolved: #150831
Copy link
Collaborator

@leslie-fang-intel leslie-fang-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. A small comment.

@Xia-Weiwen Xia-Weiwen marked this pull request as ready for review April 10, 2025 02:29
@Xia-Weiwen
Copy link
Collaborator Author

Hi @jerryzh168 Could you please review this PR? Thanks.

else:
return x * y.sum().item()

for type in [0, 1, 2, 3]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think we can improve [0, 1, 2, 3] by defining different classes for each test case here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Updated.

[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
Xia-Weiwen added a commit that referenced this pull request Apr 14, 2025
…torQuantizer

ghstack-source-id: dbd0974
Pull Request resolved: #150831
@Xia-Weiwen
Copy link
Collaborator Author

Close this PR as we need to move to Torchao.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
intel This tag is for PR from Intel open source release notes: quantization release notes category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants