CARVIEW |
Select Language
HTTP/2 200
date: Tue, 22 Jul 2025 21:51:12 GMT
content-type: text/html; charset=utf-8
vary: X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, X-Requested-With,Accept-Encoding, Accept, X-Requested-With
x-repository-download: git clone https://github.com/opencv/opencv.git
etag: W/"21a78d87bd17d6c6d63e4d71d4fd05df"
cache-control: max-age=0, private, must-revalidate
strict-transport-security: max-age=31536000; includeSubdomains; preload
x-frame-options: deny
x-content-type-options: nosniff
x-xss-protection: 0
referrer-policy: no-referrer-when-downgrade
content-security-policy: default-src 'none'; base-uri 'self'; child-src github.githubassets.com github.com/assets-cdn/worker/ github.com/assets/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com *.rel.tunnels.api.visualstudio.com wss://*.rel.tunnels.api.visualstudio.com objects-origin.githubusercontent.com copilot-proxy.githubusercontent.com proxy.individual.githubcopilot.com proxy.business.githubcopilot.com proxy.enterprise.githubcopilot.com *.actions.githubusercontent.com wss://*.actions.githubusercontent.com productionresultssa0.blob.core.windows.net/ productionresultssa1.blob.core.windows.net/ productionresultssa2.blob.core.windows.net/ productionresultssa3.blob.core.windows.net/ productionresultssa4.blob.core.windows.net/ productionresultssa5.blob.core.windows.net/ productionresultssa6.blob.core.windows.net/ productionresultssa7.blob.core.windows.net/ productionresultssa8.blob.core.windows.net/ productionresultssa9.blob.core.windows.net/ productionresultssa10.blob.core.windows.net/ productionresultssa11.blob.core.windows.net/ productionresultssa12.blob.core.windows.net/ productionresultssa13.blob.core.windows.net/ productionresultssa14.blob.core.windows.net/ productionresultssa15.blob.core.windows.net/ productionresultssa16.blob.core.windows.net/ productionresultssa17.blob.core.windows.net/ productionresultssa18.blob.core.windows.net/ productionresultssa19.blob.core.windows.net/ github-production-repository-image-32fea6.s3.amazonaws.com github-production-release-asset-2e65be.s3.amazonaws.com insights.github.com wss://alive.github.com api.githubcopilot.com api.individual.githubcopilot.com api.business.githubcopilot.com api.enterprise.githubcopilot.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com copilot-workspace.githubnext.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: blob: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com private-avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com release-assets.githubusercontent.com secured-user-images.githubusercontent.com/ user-images.githubusercontent.com/ private-user-images.githubusercontent.com opengraph.githubassets.com copilotprodattachments.blob.core.windows.net/github-production-copilot-attachments/ github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com objects-origin.githubusercontent.com *.githubusercontent.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/ private-user-images.githubusercontent.com github-production-user-asset-6210df.s3.amazonaws.com gist.github.com; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; upgrade-insecure-requests; worker-src github.githubassets.com github.com/assets-cdn/worker/ github.com/assets/ gist.github.com/assets-cdn/worker/
server: github.com
content-encoding: gzip
accept-ranges: bytes
set-cookie: _gh_sess=A9Ebybo9LTaClOFzhKvbU0wYYDpguDiiaS%2F4lKCC22K6oHnxya67jWZA7dL7viIKEE8RfQy5a196o68xaCXrCKHxrInSiKaFD0epj%2BL%2Brx2IEf8cGaykbaE0TYkF2YXOLkTYm%2B86Bj1%2BdaT%2FDMw7jBWsWbc2QfmlswY1NAjpkjykC9al0i1M4ZQ4C2FyttrNFFQvufmtLfV65TG7J5rKASV0ezaB%2Be0lluO6nIg25td%2FG4H2sszIIkIAjmgFbo08s7eB3DuHaAQpVg%2F3wrZ7xg%3D%3D--bhVdh5xjuDuCvjZQ--DQ31tvGXibCIIAyw9ecJVA%3D%3D; Path=/; HttpOnly; Secure; SameSite=Lax
set-cookie: _octo=GH1.1.1186846245.1753221072; Path=/; Domain=github.com; Expires=Wed, 22 Jul 2026 21:51:12 GMT; Secure; SameSite=Lax
set-cookie: logged_in=no; Path=/; Domain=github.com; Expires=Wed, 22 Jul 2026 21:51:12 GMT; HttpOnly; Secure; SameSite=Lax
x-github-request-id: DE48:351DD4:1680EE:1DDDCC:688007D0
Merge pull request #24476 from fengyuentau:attention_layer · opencv/opencv@0521a3a · GitHub
Copy file name to clipboardExpand all lines: modules/dnn/include/opencv2/dnn/all_layers.hpp
Copy file name to clipboardExpand all lines: modules/dnn/perf/perf_layer.cpp
Copy file name to clipboardExpand all lines: modules/dnn/perf/perf_net.cpp
Skip to content
Navigation Menu
{{ message }}
-
-
Notifications
You must be signed in to change notification settings - Fork 56.2k
Commit 0521a3a
authored
Merge pull request #24476 from fengyuentau:attention_layer
dnn: add attention layer #24476
Resolves #24609
Merge with: opencv/opencv_extra#1128.
Attention operator spec from onnxruntime: https://github.com/microsoft/onnxruntime/blob/v1.16.1/docs/ContribOperators.md#com.microsoft.Attention.
TODO:
- [x] benchmark (before this PR vs. with this PR vs. ORT).
- [x] Layer fusion: Take care Slice with end=INT64_MAX.
- [x] Layer fusion: match more potential attention (VIT) patterns.
- [x] Single-head attention is supported.
- [x] Test AttentionSubgraph fusion.
- [x] Add acc tests for VIT_B_32 and VitTrack
- [x] Add perf tests for VIT_B_32 and VitTrack
## Benchmarks
Platform: Macbook Air M1.
### Attention Subgraph
Input scale: [1, 197, 768].
| | mean (ms) | median (ms) | min (ms) |
| ---------------------- | --------- | ----------- | -------- |
| w/ Attention (this PR) | 3.75 | 3.68 | 3.22 |
| w/o Attention | 9.06 | 9.01 | 8.24 |
| ORT (python) | 4.32 | 2.63 | 2.50 |
### ViTs
All data in millisecond (ms).
| ViTs | With Attention | Without Attention | ORT |
| -------- | -------------- | ----------------- | ------ |
| vit_b_16 | 302.77 | 365.35 | 109.70 |
| vit_b_32 | 89.92 | 116.22 | 30.36 |
| vit_l_16 | 1593.32 | 1730.74 | 419.92 |
| vit_l_32 | 468.11 | 577.41 | 134.12 |
| VitTrack | 3.80 | 3.87 | 2.25 |
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake1 parent e64c5dc commit 0521a3aCopy full SHA for 0521a3a
File tree
Expand file treeCollapse file tree
13 files changed
+891
-66
lines changedFilter options
- modules/dnn
- include/opencv2/dnn
- perf
- src
- layers
- cpu_kernels
- onnx
- test
Expand file treeCollapse file tree
13 files changed
+891
-66
lines changedmodules/dnn/include/opencv2/dnn/all_layers.hpp
Copy file name to clipboardExpand all lines: modules/dnn/include/opencv2/dnn/all_layers.hpp+5Lines changed: 5 additions & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
1178 | 1178 |
| |
1179 | 1179 |
| |
1180 | 1180 |
| |
| 1181 | + | |
| 1182 | + | |
| 1183 | + | |
| 1184 | + | |
| 1185 | + | |
1181 | 1186 |
| |
1182 | 1187 |
| |
1183 | 1188 |
| |
|
modules/dnn/perf/perf_layer.cpp
Copy file name to clipboardExpand all lines: modules/dnn/perf/perf_layer.cpp+57Lines changed: 57 additions & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
739 | 739 |
| |
740 | 740 |
| |
741 | 741 |
| |
| 742 | + | |
| 743 | + | |
| 744 | + | |
| 745 | + | |
| 746 | + | |
| 747 | + | |
| 748 | + | |
| 749 | + | |
| 750 | + | |
| 751 | + | |
| 752 | + | |
| 753 | + | |
| 754 | + | |
| 755 | + | |
| 756 | + | |
| 757 | + | |
| 758 | + | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
| 763 | + | |
| 764 | + | |
| 765 | + | |
| 766 | + | |
| 767 | + | |
| 768 | + | |
| 769 | + | |
| 770 | + | |
| 771 | + | |
| 772 | + | |
| 773 | + | |
| 774 | + | |
| 775 | + | |
| 776 | + | |
| 777 | + | |
| 778 | + | |
| 779 | + | |
| 780 | + | |
| 781 | + | |
| 782 | + | |
| 783 | + | |
| 784 | + | |
| 785 | + | |
| 786 | + | |
| 787 | + | |
| 788 | + | |
| 789 | + | |
| 790 | + | |
| 791 | + | |
| 792 | + | |
| 793 | + | |
| 794 | + | |
| 795 | + | |
| 796 | + | |
| 797 | + | |
742 | 798 |
| |
743 | 799 |
| |
744 | 800 |
| |
| |||
750 | 806 |
| |
751 | 807 |
| |
752 | 808 |
| |
| 809 | + | |
753 | 810 |
| |
754 | 811 |
| |
755 | 812 |
| |
|
modules/dnn/perf/perf_net.cpp
Copy file name to clipboardExpand all lines: modules/dnn/perf/perf_net.cpp+6-4Lines changed: 6 additions & 4 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
93 | 93 |
| |
94 | 94 |
| |
95 | 95 |
| |
96 |
| - | |
97 | 96 |
| |
98 | 97 |
| |
99 | 98 |
| |
| |||
391 | 390 |
| |
392 | 391 |
| |
393 | 392 |
| |
394 |
| - | |
| 393 | + | |
395 | 394 |
| |
396 | 395 |
| |
397 | 396 |
| |
398 | 397 |
| |
399 | 398 |
| |
400 | 399 |
| |
401 |
| - | |
| 400 | + | |
402 | 401 |
| |
403 | 402 |
| |
404 |
| - | |
405 | 403 |
| |
406 | 404 |
| |
407 | 405 |
| |
| |||
413 | 411 |
| |
414 | 412 |
| |
415 | 413 |
| |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
416 | 418 |
| |
417 | 419 |
| |
418 | 420 |
|
+1Lines changed: 1 addition & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
162 | 162 |
| |
163 | 163 |
| |
164 | 164 |
| |
| 165 | + | |
165 | 166 |
| |
166 | 167 |
| |
167 | 168 |
| |
|
You can’t perform that action at this time.
0 commit comments