CARVIEW |
Select Language
HTTP/2 200
date: Wed, 23 Jul 2025 00:01:45 GMT
content-type: text/html; charset=utf-8
vary: X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, X-Requested-With,Accept-Encoding, Accept, X-Requested-With
x-repository-download: git clone https://github.com/intel/neural-compressor.git
etag: W/"a1b0c289538b442941424591cede4c39"
cache-control: max-age=0, private, must-revalidate
strict-transport-security: max-age=31536000; includeSubdomains; preload
x-frame-options: deny
x-content-type-options: nosniff
x-xss-protection: 0
referrer-policy: no-referrer-when-downgrade
content-security-policy: default-src 'none'; base-uri 'self'; child-src github.githubassets.com github.com/assets-cdn/worker/ github.com/assets/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com *.rel.tunnels.api.visualstudio.com wss://*.rel.tunnels.api.visualstudio.com objects-origin.githubusercontent.com copilot-proxy.githubusercontent.com proxy.individual.githubcopilot.com proxy.business.githubcopilot.com proxy.enterprise.githubcopilot.com *.actions.githubusercontent.com wss://*.actions.githubusercontent.com productionresultssa0.blob.core.windows.net/ productionresultssa1.blob.core.windows.net/ productionresultssa2.blob.core.windows.net/ productionresultssa3.blob.core.windows.net/ productionresultssa4.blob.core.windows.net/ productionresultssa5.blob.core.windows.net/ productionresultssa6.blob.core.windows.net/ productionresultssa7.blob.core.windows.net/ productionresultssa8.blob.core.windows.net/ productionresultssa9.blob.core.windows.net/ productionresultssa10.blob.core.windows.net/ productionresultssa11.blob.core.windows.net/ productionresultssa12.blob.core.windows.net/ productionresultssa13.blob.core.windows.net/ productionresultssa14.blob.core.windows.net/ productionresultssa15.blob.core.windows.net/ productionresultssa16.blob.core.windows.net/ productionresultssa17.blob.core.windows.net/ productionresultssa18.blob.core.windows.net/ productionresultssa19.blob.core.windows.net/ github-production-repository-image-32fea6.s3.amazonaws.com github-production-release-asset-2e65be.s3.amazonaws.com insights.github.com wss://alive.github.com api.githubcopilot.com api.individual.githubcopilot.com api.business.githubcopilot.com api.enterprise.githubcopilot.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com copilot-workspace.githubnext.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: blob: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com private-avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com release-assets.githubusercontent.com secured-user-images.githubusercontent.com/ user-images.githubusercontent.com/ private-user-images.githubusercontent.com opengraph.githubassets.com copilotprodattachments.blob.core.windows.net/github-production-copilot-attachments/ github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com objects-origin.githubusercontent.com *.githubusercontent.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/ private-user-images.githubusercontent.com github-production-user-asset-6210df.s3.amazonaws.com gist.github.com; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; upgrade-insecure-requests; worker-src github.githubassets.com github.com/assets-cdn/worker/ github.com/assets/ gist.github.com/assets-cdn/worker/
server: github.com
content-encoding: gzip
accept-ranges: bytes
set-cookie: _gh_sess=PIZKWF8ix4wZMgmeWC7o4OiNO2KeqITWqJCTyEBDZDwOLHWJt34Fi9wtGWiib8iPgcOuhfTKCYOblHGCctcpmE%2Bg6XRhhFCAwoJeTiI4MjtAefHBDnFrkcP3EoGeK6u9vVQiEv8JlXHTE4dzTQddFMsspjK3ZbcXQfBg%2FXvTN6vY1OvYQqghuULvigbNHghVk%2FgsTCkbc2uJUpWJ73S9tPPrxH06RpA5ZNguF3f1rtHxNRmkVdVAUoEybjmPkwmgo8dWAkQnExPfVqk5ES8WDg%3D%3D--Ly4%2B0KfldGpTqoLH--xTVpa8Sk1XvluQFXPIWecA%3D%3D; Path=/; HttpOnly; Secure; SameSite=Lax
set-cookie: _octo=GH1.1.192000214.1753228904; Path=/; Domain=github.com; Expires=Thu, 23 Jul 2026 00:01:44 GMT; Secure; SameSite=Lax
set-cookie: logged_in=no; Path=/; Domain=github.com; Expires=Thu, 23 Jul 2026 00:01:44 GMT; HttpOnly; Secure; SameSite=Lax
x-github-request-id: EC02:1C6D98:1E0DCE:299ACF:68802668
support double quant for weight-only (#1420) · intel/neural-compressor@05c15a4 · GitHub
Copy file name to clipboard
Copy file name to clipboardExpand all lines: neural_compressor/torch/algorithms/weight_only/gptq.py
Skip to content
Navigation Menu
{{ message }}
-
Notifications
You must be signed in to change notification settings - Fork 277
Commit 05c15a4
authored
support double quant for weight-only (#1420)
Description
popular repos, like bitsandbytes, llama.cpp, provides double quant on scale to improve the compression ratio of weight-only model. Theoretically, double quant will quantize the scales of several blocks and use a hyper scale and a hyper zeropoint to recover it.
Using below args in RTNWeightQuantConfig to set double quant for scales
double_quant_dtype
double_quant_bits
double_quant_sym
double_quant_group_size
Demo code can be found in UT.1 parent de385a4 commit 05c15a4Copy full SHA for 05c15a4
File tree
Expand file treeCollapse file tree
9 files changed
+1008
-298
lines changedFilter options
- neural_compressor/torch
- algorithms
- weight_only
- quantization
- test/3x/torch
Expand file treeCollapse file tree
9 files changed
+1008
-298
lines changedneural_compressor/torch/algorithms/weight_only/__init__.py
Copy file name to clipboard+13Lines changed: 13 additions & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + |
neural_compressor/torch/algorithms/gptq.py renamed to neural_compressor/torch/algorithms/weight_only/gptq.py
Copy file name to clipboardExpand all lines: neural_compressor/torch/algorithms/weight_only/gptq.py+99-20Lines changed: 99 additions & 20 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
233 | 233 |
| |
234 | 234 |
| |
235 | 235 |
| |
| 236 | + | |
236 | 237 |
| |
237 | 238 |
| |
238 | 239 |
| |
| |||
241 | 242 |
| |
242 | 243 |
| |
243 | 244 |
| |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
244 | 249 |
| |
245 | 250 |
| |
246 | 251 |
| |
| |||
285 | 290 |
| |
286 | 291 |
| |
287 | 292 |
| |
| 293 | + | |
288 | 294 |
| |
289 | 295 |
| |
290 | 296 |
| |
| |||
293 | 299 |
| |
294 | 300 |
| |
295 | 301 |
| |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
296 | 314 |
| |
297 | 315 |
| |
298 | 316 |
| |
| 317 | + | |
299 | 318 |
| |
300 | 319 |
| |
301 | 320 |
| |
| |||
304 | 323 |
| |
305 | 324 |
| |
306 | 325 |
| |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
307 | 338 |
| |
308 | 339 |
| |
309 | 340 |
| |
| |||
467 | 498 |
| |
468 | 499 |
| |
469 | 500 |
| |
470 |
| - | |
471 |
| - | |
472 |
| - | |
473 |
| - | |
474 |
| - | |
475 |
| - | |
| 501 | + | |
476 | 502 |
| |
477 | 503 |
| |
478 | 504 |
| |
| |||
656 | 682 |
| |
657 | 683 |
| |
658 | 684 |
| |
659 |
| - | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
660 | 688 |
| |
661 | 689 |
| |
662 | 690 |
| |
| |||
712 | 740 |
| |
713 | 741 |
| |
714 | 742 |
| |
715 |
| - | |
716 |
| - | |
717 |
| - | |
718 |
| - | |
719 |
| - | |
| 743 | + | |
| 744 | + | |
| 745 | + | |
| 746 | + | |
| 747 | + | |
| 748 | + | |
| 749 | + | |
720 | 750 |
| |
721 | 751 |
| |
722 | 752 |
| |
| |||
726 | 756 |
| |
727 | 757 |
| |
728 | 758 |
| |
729 |
| - | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
| 763 | + | |
| 764 | + | |
| 765 | + | |
| 766 | + | |
| 767 | + | |
| 768 | + | |
| 769 | + | |
| 770 | + | |
| 771 | + | |
| 772 | + | |
| 773 | + | |
| 774 | + | |
| 775 | + | |
| 776 | + | |
| 777 | + | |
| 778 | + | |
| 779 | + | |
| 780 | + | |
| 781 | + | |
| 782 | + | |
730 | 783 |
| |
731 | 784 |
| |
732 | 785 |
| |
| |||
773 | 826 |
| |
774 | 827 |
| |
775 | 828 |
| |
776 |
| - | |
| 829 | + | |
777 | 830 |
| |
778 | 831 |
| |
779 | 832 |
| |
| |||
795 | 848 |
| |
796 | 849 |
| |
797 | 850 |
| |
| 851 | + | |
| 852 | + | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
| 857 | + | |
| 858 | + | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| 867 | + | |
798 | 868 |
| |
799 | 869 |
| |
800 | 870 |
| |
| |||
806 | 876 |
| |
807 | 877 |
| |
808 | 878 |
| |
809 |
| - | |
810 |
| - | |
811 |
| - | |
812 |
| - | |
| 879 | + | |
| 880 | + | |
| 881 | + | |
| 882 | + | |
813 | 883 |
| |
814 |
| - | |
815 |
| - | |
| 884 | + | |
| 885 | + | |
| 886 | + | |
| 887 | + | |
| 888 | + | |
| 889 | + | |
816 | 890 |
| |
817 | 891 |
| |
818 | 892 |
| |
| |||
848 | 922 |
| |
849 | 923 |
| |
850 | 924 |
| |
| 925 | + | |
851 | 926 |
| |
852 | 927 |
| |
853 | 928 |
| |
854 | 929 |
| |
855 | 930 |
| |
856 | 931 |
| |
857 | 932 |
| |
| 933 | + | |
| 934 | + | |
| 935 | + | |
| 936 | + | |
858 | 937 |
| |
859 | 938 |
| |
860 | 939 |
| |
|
You can’t perform that action at this time.
0 commit comments