| CARVIEW |
Navigation Menu
-
Notifications
You must be signed in to change notification settings - Fork 24.7k
[Quantization][PrivateUse1] Adding more support QuantizedPrivateuse1 backends #139860
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
fmo-mt
requested review from
jerryzh168,
salilsdesai,
kimishpatel,
digantdesai and
jianyuh
as code owners
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/139860
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ No FailuresAs of commit bbde42a with merge base c19c384 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
bdhirsh
added
the
triaged
| @@ -8,37 +8,37 @@ | |||
| namespace at { | |||
| namespace native { | |||
|
|
|||
| Tensor& quantize_tensor_per_tensor_affine( | |||
| TORCH_API Tensor& quantize_tensor_per_tensor_affine( | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jerryzh168 you might be a better reviewer here - do we want these API's to get public symbols?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we do, and it would throw an error of undefined symbol if we don't.
PTAL @jerryzh168
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is not a public function I think, what does "getting public symbols" entail here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is not a public function I think, what does "getting public symbols" entail here?
it's not, but it should be if privateuse1 user like to implement quantization functions like torch.quantize_per_tensor, so we can reuse this native common functions.
| @@ -1,4 +1,5 @@ | |||
| #pragma once | |||
| #include <ATen/native/DispatchStub.h> | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, as I comment, it's because that during implementing some operators for privateuse1 backend, If DispatchStub.h is not included, we can't use the two stubs in this header, like masked_fill_kernel_quantized_stub and index_put_kernel_quantized_stub, which would throw an error during compilation.
|
@bdhirsh @jerryzh168 Hi, can we make this proceed? I believe that many other |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good to me!
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
…backends (pytorch#139860) Here's are some explanations of this PR. 1. Changes in `aten/src/ATen/core/Tensor.cpp` and `c10/core/DispatchKey.cpp`: Support toString method for `QuantizedPrivateUse1` backend, make pytorch print out correct backend string for it. 2. Add header `DispatchStub.h` in `aten/src/ATen/native/quantized/IndexKernel.h`: If this header is not included, we can't utilize `masked_fill_kernel_quantized_stub` even we include this `IndexKernel.h` header, it would throw an error during compilation. 3. Add multiple `TORCH_API`s in `aten/src/ATen/native/quantized/AffineQuantizer.h`: these functions is useful for other privateuse1 backends supporting quantization functions, if these `TORCH_API` are missed, it would throw an error during runtime (undefined symbol) Pull Request resolved: pytorch#139860 Approved by: https://github.com/bdhirsh
…backends (pytorch#139860) Here's are some explanations of this PR. 1. Changes in `aten/src/ATen/core/Tensor.cpp` and `c10/core/DispatchKey.cpp`: Support toString method for `QuantizedPrivateUse1` backend, make pytorch print out correct backend string for it. 2. Add header `DispatchStub.h` in `aten/src/ATen/native/quantized/IndexKernel.h`: If this header is not included, we can't utilize `masked_fill_kernel_quantized_stub` even we include this `IndexKernel.h` header, it would throw an error during compilation. 3. Add multiple `TORCH_API`s in `aten/src/ATen/native/quantized/AffineQuantizer.h`: these functions is useful for other privateuse1 backends supporting quantization functions, if these `TORCH_API` are missed, it would throw an error during runtime (undefined symbol) Pull Request resolved: pytorch#139860 Approved by: https://github.com/bdhirsh
…backends (pytorch#139860) Here's are some explanations of this PR. 1. Changes in `aten/src/ATen/core/Tensor.cpp` and `c10/core/DispatchKey.cpp`: Support toString method for `QuantizedPrivateUse1` backend, make pytorch print out correct backend string for it. 2. Add header `DispatchStub.h` in `aten/src/ATen/native/quantized/IndexKernel.h`: If this header is not included, we can't utilize `masked_fill_kernel_quantized_stub` even we include this `IndexKernel.h` header, it would throw an error during compilation. 3. Add multiple `TORCH_API`s in `aten/src/ATen/native/quantized/AffineQuantizer.h`: these functions is useful for other privateuse1 backends supporting quantization functions, if these `TORCH_API` are missed, it would throw an error during runtime (undefined symbol) Pull Request resolved: pytorch#139860 Approved by: https://github.com/bdhirsh
Here's are some explanations of this PR.
aten/src/ATen/core/Tensor.cppandc10/core/DispatchKey.cpp: Support toString method forQuantizedPrivateUse1backend, make pytorch print out correct backend string for it.DispatchStub.hinaten/src/ATen/native/quantized/IndexKernel.h: If this header is not included, we can't utilizemasked_fill_kernel_quantized_stubeven we include thisIndexKernel.hheader, it would throw an error during compilation.TORCH_APIs inaten/src/ATen/native/quantized/AffineQuantizer.h: these functions is useful for other privateuse1 backends supporting quantization functions, if theseTORCH_APIare missed, it would throw an error during runtime (undefined symbol)