| CARVIEW |
Navigation Menu
-
-
Notifications
You must be signed in to change notification settings - Fork 56.2k
[GSoC] OpenCV.js: WASM SIMD optimization 2.0 #18068
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
fix the trailing whitespace.
lionkunonly
force-pushed
the
gsoc_2020_simd
branch
from
a7ad26e to
c3d8bd3
Compare
…n in perf test of warpAffine and warpPrespective
|
Hi, I have implemented the OpenCV.js loader and modified the |
modules/js/src/loader.js
Outdated
| let threadsPath = ""; | ||
| let mtSIMDPath = ""; | ||
|
|
||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove the redundant blank line (just need one).
modules/js/src/loader.js
Outdated
|
|
||
|
|
||
| this.judgeWASM = function() { | ||
| try{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please insert a space between try and {.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
actually, you can check WebAssembly by
return !(typeof WebAssembly === 'undefined')
modules/js/src/loader.js
Outdated
| let mtSIMDPath = ""; | ||
|
|
||
|
|
||
| this.judgeWASM = function() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
judge sounds not a good name. how about checkWasm?
modules/js/src/loader.js
Outdated
| } | ||
| } | ||
|
|
||
| this.judgeSIMD = function() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto.
modules/js/src/loader.js
Outdated
| } | ||
|
|
||
| this.judgeSIMD = function() { | ||
| return WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,4,1,96,0,0,3,2,1,0,10,9,1,7,0,65,0,253,15,26,11])); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could you please document what's the content of the Unit8Array?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
and you need to call checkWasm first.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By reading the docs of wasm-feature-detect, I think my implementation for detecting threads feature and simd feature is good enough. Because the browser will update in the future, which may block my implementation. So, I decide to utilize the wasm-feature-detect to solve my problem. Do you agree it ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It sounds good to me.
modules/js/src/loader.js
Outdated
| try { | ||
| let test1 = (new MessageChannel).port1.postMessage(new SharedArrayBuffer(1)); | ||
| let result = WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,4,1,96,0,0,3,2,1,0,5,4,1,3,1,1,10,11,1,9,0,65,0,254,16,2,0,26,11])); | ||
| return result; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return true?
modules/js/src/loader.js
Outdated
| let result = WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,4,1,96,0,0,3,2,1,0,5,4,1,3,1,1,10,11,1,9,0,65,0,254,16,2,0,26,11])); | ||
| return result; | ||
| } catch(e) { | ||
| return !1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return false?
modules/js/src/loader.js
Outdated
| } | ||
| } | ||
|
|
||
| this.setPaths = function(paths) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove this method and merge its implementation into the constructor.
modules/js/src/loader.js
Outdated
| } | ||
| } | ||
|
|
||
| this.loadOpenCV = function (onloadCallback) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you can encapsulate all functions into this method. Do you really need a loader object?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
function loadOpenCV(paths, onloadCallback) {...}It would have simpler usage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The loader object is not necessary. I think I can try to encapsulate all functions in loadOpenCV.
modules/js/src/loader.js
Outdated
| if (simdSupported && threadsSupported && OPENCV_URL == "" && self.mtSIMDPath != "") { | ||
| OPENCV_URL = self.mtSIMDPath; | ||
| } else if (simdSupported && threadsSupported && OPENCV_URL == "") { | ||
| throw new Error("The browser supports simd and threads, but the path of OpenCV.js with simd and threads optimization is empty"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think in this case, if developer only provides the threadsPath or simdPath, it should also work.
platforms/js/build_js.py
Outdated
| @@ -273,7 +277,7 @@ def build_doc(self): | |||
| log.info("===== Building OpenCV.js") | |||
| log.info("=====") | |||
| builder.build_opencvjs() | |||
|
|
|||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove trailing whitespaces
…ve trailing whitespaces
|
@huningxin @terfendail Hi, I have modified the implementation based on your comments in code review. However, the most important change in this implementation is that I use the WebAssembly Feature Detection in my implementation. The reason for using it is that I find the original implementation may fail when the browser update in the future. With the help of the WebAssembly Feature Detection would be available even if the browser update. Because this library would update at that time. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lionkunonly , could you please share the tests and the results you did for your loader?
modules/js/src/loader.js
Outdated
| OPENCV_URL = simdPath; | ||
| console.log("The OpenCV.js with simd optimization is loaded now."); | ||
| } else if (threadsSupported && threadsPath != "") { | ||
| if (simdSupported && threadsSupported) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if (simdSupported && threadsSimdPath === "")?
modules/js/src/loader.js
Outdated
| console.log("The browser supports wasm, but the path of OpenCV.js for wasm is empty"); | ||
| } | ||
|
|
||
| if(OPENCV_URL == "") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Put a space between if and (
modules/js/src/loader.js
Outdated
| throw new Error("The browser supports simd, but the path of OpenCV.js with simd optimization is empty"); | ||
| OPENCV_URL = threadsPath; | ||
| console.log("The OpenCV.js with threads optimization is loaded now"); | ||
| } else if (threadsPath) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
else if (threadsSupported)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, it is my fault. This choice should be deleted.
| Example Code: | ||
| @code{.javascipt} | ||
| // Create an instance | ||
| loader = new Loader(); | ||
|
|
||
| //Set paths configuration | ||
| pathsConfig = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let pathsConfig?
| Example Code: | ||
| @code{.javascipt} | ||
| // Create an instance | ||
| loader = new Loader(); | ||
|
|
||
| //Set paths configuration |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a space after //
|
|
||
| //Load OpenCV.js and use main function as the param | ||
| loader.loadOpenCV(main); | ||
| //Load OpenCV.js and use the pathsConfiguration and main function as the params. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto.
The Scene1: SIMD and threads are enabled.The path configuration of the example
The log for loading status and html elements
In this scene, the application load the opencv.js with simd and threads optimization successfully. The Scene2: SIMD and threads are enabled, but the path to threads+simd version is invalid.The path configuration in js file
Results
In this scene, the path for opencv.js with threads and simd optimization is empty. So the loader loads the opencv.js with simd optimization automatically. The Scene3: Threads is enabled, simd is diabled.The feature configuration in browser
Results
In this scene, the loader loads the opencv.js with threads optimization automatically without any failed log like the log in the scene2. Finally, I am trying to design a demo to show the effect of the loader and deploy the demo on the github.io. |
You can try the example with the Demo -- Perf tests and loader demo @huningxin @terfendail I think I have completed all the tasks in GSoC2020 now. Should I remove the |
lionkunonly
changed the title
|
@terfendail @terfendail The final report is Here. You can leave your comment in the Google drive version . |
The loader demo works great regarding to my test, even when disabling WASM in Chrome. Thanks for that! My another suggestion would be using
Please add the summary and analysis of your performance data into the final report. Both of them are valuable. Thanks! |
It seems all tasks are done. Please remove that so the OpenCV maintainers can review. |
|
@terfendail @huningxin Hi, Vitaly and Ningxin. I have updated the video report please check it here |
Overview
This pull request changes
perf_helpfunc.js. Reconstruct current perf tests.cvtColor,resize,threshold,Sobel,filter2D,Scharr,gaussianBlur,blur,medianBlur,erode,dilate,remap,warpAffine,warpPerspective,pyrDown.The Test
Test Environment:
Results
The performance of kernels after 64-bit implementation is similar to the performance before implementation.
shift(ms)widen(ms)The perform of the implementation with
wideninstructions does not bring improvement. One of the reasons is that thewasm_v8x16_shuffleandwideninstructions have to be used together. The original implementation is left finally.The collected performance data is stored in the Google Drive. Performance data
Performance Analysis
Because there are more than ten kernels are tested and some of them are tested with parameters with different data type and channels, the table that records the performance data is large. So I put the performance data in my personal Google drive.
Performance data
Analysis
Based on the collected performance data, the SIMD optimization works as we expected in most situations. It can achieve similar performance as the SSE2 optimization for the native OpenCV. For example, for the kernel
blurwith the parameter(1280x720, CV_8UC1, BORDER_REPLICATE) ksize=3, the SIMD version has1.357xspeed up and the SSE2 has1.415xspeed up. Sometimes the SIMD optimization is better. For example, for the kernelpyrDownwith the parameter(1920x1080, CV_32FC4), the SIMD version has3.094xspeed up and the SSE2 has1.83speed up. The data that achieves similar or better speed up is tagged with the color green.However, there still exist some bad cases, for example, for the kernel
blurwith(1280x720, CV_32FC1, BORDER_REPLICATE) ksize=3, the SIMD version has0.519speed up which is 2x slower than the SSE optimization for native OpenCV. Such data is tagged with the color yellow.The data tagged with color red means that it is unnormal. In my point of view, some yellow data is brought by the red unnormal data like the kernel
medianBlurwith the parameter(1280x720, CV_16SC1, 5).I hope the collected data can help people know the gap between the SIMD optimization in the OpenCV.js and the optimization in the native OpenCV better.