opencv
diff --git a/‎doc/tutorials/dnn/dnn_face/dnn_face.markdown
Copy file name to clipboard
+95Lines changed: 95 additions & 0 deletions b/‎doc/tutorials/dnn/dnn_face/dnn_face.markdown
Copy file name to clipboard
+95Lines changed: 95 additions & 0 deletions
diff --git a/‎doc/tutorials/dnn/dnn_text_spotting/dnn_text_spotting.markdown
Copy file name to clipboard
+1-1Lines changed: 1 addition & 1 deletion b/‎doc/tutorials/dnn/dnn_text_spotting/dnn_text_spotting.markdown
Copy file name to clipboard
+1-1Lines changed: 1 addition & 1 deletion
diff --git a/‎doc/tutorials/dnn/table_of_content_dnn.markdown
Copy file name to clipboard
+1Lines changed: 1 addition & 0 deletions b/‎doc/tutorials/dnn/table_of_content_dnn.markdown
Copy file name to clipboard
+1Lines changed: 1 addition & 0 deletions
diff --git a/‎modules/objdetect/CMakeLists.txt
Copy file name to clipboard
+1-1Lines changed: 1 addition & 1 deletion b/‎modules/objdetect/CMakeLists.txt
Copy file name to clipboard
+1-1Lines changed: 1 addition & 1 deletion
diff --git a/‎modules/objdetect/include/opencv2/objdetect.hpp
Copy file name to clipboard
+1Lines changed: 1 addition & 0 deletions b/‎modules/objdetect/include/opencv2/objdetect.hpp
Copy file name to clipboard
+1Lines changed: 1 addition & 0 deletions
diff --git a/‎modules/objdetect/include/opencv2/objdetect/face.hpp
Copy file name to clipboard
+125Lines changed: 125 additions & 0 deletions b/‎modules/objdetect/include/opencv2/objdetect/face.hpp
Copy file name to clipboard
+125Lines changed: 125 additions & 0 deletions
@@ -0,0 +1,95 @@
+# DNN-based Face Detection And Recognition {#tutorial_dnn_face}
+
+@tableofcontents
+
+@prev_tutorial{tutorial_dnn_text_spotting}
+@next_tutorial{pytorch_cls_tutorial_dnn_conversion}
+
+| | |
+| -: | :- |
+| Original Author | Chengrui Wang, Yuantao Feng |
+| Compatibility | OpenCV >= 4.5.1 |
+
+## Introduction
+
+In this section, we introduce the DNN-based module for face detection and face recognition. Models can be obtained in [Models](#Models). The usage of `FaceDetectorYN` and `FaceRecognizer` are presented in [Usage](#Usage).
+
+## Models
+
+There are two models (ONNX format) pre-trained and required for this module:
+- [Face Detection](https://github.com/ShiqiYu/libfacedetection.train/tree/master/tasks/task1/onnx):
+    - Size: 337KB
+    - Results on WIDER Face Val set: 0.830(easy), 0.824(medium), 0.708(hard)
+- [Face Recognition](https://drive.google.com/file/d/1ClK9WiB492c5OZFKveF3XiHCejoOxINW/view?usp=sharing)
+    - Size: 36.9MB
+    - Results:
+
+    | Database | Accuracy | Threshold (normL2) | Threshold (cosine) |
+    | -------- | -------- | ------------------ | ------------------ |
+    | LFW      | 99.60%   | 1.128              | 0.363              |
+    | CALFW    | 93.95%   | 1.149              | 0.340              |
+    | CPLFW    | 91.05%   | 1.204              | 0.275              |
+    | AgeDB-30 | 94.90%   | 1.202              | 0.277              |
+    | CFP-FP   | 94.80%   | 1.253              | 0.212              |
+
+## Usage
+
+### DNNFaceDetector
+
+```cpp
+// Initialize FaceDetectorYN
+Ptr<FaceDetectorYN> faceDetector = FaceDetectorYN::create(onnx_path, "", image.size(), score_thresh, nms_thresh, top_k);
+
+// Forward
+Mat faces;
+faceDetector->detect(image, faces);
+```
+
+The detection output `faces` is a two-dimension array of type CV_32F, whose rows are the detected face instances, columns are the location of a face and 5 facial landmarks. The format of each row is as follows:
+
+```
+x1, y1, w, h, x_re, y_re, x_le, y_le, x_nt, y_nt, x_rcm, y_rcm, x_lcm, y_lcm
+```
+, where `x1, y1, w, h` are the top-left coordinates, width and height of the face bounding box, `{x, y}_{re, le, nt, rcm, lcm}` stands for the coordinates of right eye, left eye, nose tip, the right corner and left corner of the mouth respectively.
+
+
+### Face Recognition
+
+Following Face Detection, run codes below to extract face feature from facial image.
+
+```cpp
+// Initialize FaceRecognizer with model path (cv::String)
+Ptr<FaceRecognizer> faceRecognizer = FaceRecognizer::create(model_path, "");
+
+// Aligning and cropping facial image through the first face of faces detected by dnn_face::DNNFaceDetector
+Mat aligned_face;
+faceRecognizer->alignCrop(image, faces.row(0), aligned_face);
+
+// Run feature extraction with given aligned_face (cv::Mat)
+Mat feature;
+faceRecognizer->feature(aligned_face, feature);
+feature = feature.clone();
+```
+
+After obtaining face features *feature1* and *feature2* of two facial images, run codes below to calculate the identity discrepancy between the two faces.
+
+```cpp
+// Calculating the discrepancy between two face features by using cosine distance.
+double cos_score = faceRecognizer->match(feature1, feature2, FaceRecognizer::DisType::COSINE);
+// Calculating the discrepancy between two face features by using normL2 distance.
+double L2_score = faceRecognizer->match(feature1, feature2, FaceRecognizer::DisType::NORM_L2);
+```
+
+For example, two faces have same identity if the cosine distance is greater than or equal to 0.363, or the normL2 distance is less than or equal to 1.128.
+
+## Reference:
+
+- https://github.com/ShiqiYu/libfacedetection
+- https://github.com/ShiqiYu/libfacedetection.train
+- https://github.com/zhongyy/SFace
+
+## Acknowledgement
+
+Thanks [Professor Shiqi Yu](https://github.com/ShiqiYu/) and [Yuantao Feng](https://github.com/fengyuentau) for training and providing the face detection model.
+
+Thanks [Professor Deng](https://www.whdeng.cn/), [PhD Candidate Zhong](https://github.com/zhongyy/) and [Master Candidate Wang](https://github.com/crywang/) for training and providing the face recognition model.
@@ -3,7 +3,7 @@
 @tableofcontents
 
 @prev_tutorial{tutorial_dnn_OCR}
-@next_tutorial{pytorch_cls_tutorial_dnn_conversion}
+@next_tutorial{tutorial_dnn_face}
 
 |    |    |
 | -: | :- |
 
@@ -10,6 +10,7 @@ Deep Neural Networks (dnn module) {#tutorial_table_of_content_dnn}
 -   @subpage tutorial_dnn_custom_layers
 -   @subpage tutorial_dnn_OCR
 -   @subpage tutorial_dnn_text_spotting
+-   @subpage tutorial_dnn_face
 
 #### PyTorch models with OpenCV
 In this section you will find the guides, which describe how to run classification, segmentation and detection PyTorch DNN models with OpenCV.
 
@@ -1,5 +1,5 @@
 set(the_description "Object Detection")
-ocv_define_module(objdetect opencv_core opencv_imgproc opencv_calib3d WRAP java objc python js)
+ocv_define_module(objdetect opencv_core opencv_imgproc opencv_calib3d opencv_dnn WRAP java objc python js)
 
 if(HAVE_QUIRC)
     get_property(QUIRC_INCLUDE GLOBAL PROPERTY QUIRC_INCLUDE_DIR)
 
@@ -768,5 +768,6 @@ class CV_EXPORTS_W QRCodeDetector
 }
 
 #include "opencv2/objdetect/detection_based_tracker.hpp"
+#include "opencv2/objdetect/face.hpp"
 
 #endif
@@ -0,0 +1,125 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at https://opencv.org/license.html.
+
+#ifndef OPENCV_OBJDETECT_FACE_HPP
+#define OPENCV_OBJDETECT_FACE_HPP
+
+#include <opencv2/core.hpp>
+
+/** @defgroup dnn_face DNN-based face detection and recognition
+ */
+
+namespace cv
+{
+
+/** @brief DNN-based face detector, model download link: https://github.com/ShiqiYu/libfacedetection.train/tree/master/tasks/task1/onnx.
+ */
+class CV_EXPORTS_W FaceDetectorYN
+{
+public:
+    virtual ~FaceDetectorYN() {};
+
+    /** @brief Set the size for the network input, which overwrites the input size of creating model. Call this method when the size of input image does not match the input size when creating model
+     *
+     * @param input_size the size of the input image
+     */
+    CV_WRAP virtual void setInputSize(const Size& input_size) = 0;
+
+    CV_WRAP virtual Size getInputSize() = 0;
+
+    /** @brief Set the score threshold to filter out bounding boxes of score less than the given value
+     *
+     * @param score_threshold threshold for filtering out bounding boxes
+     */
+    CV_WRAP virtual void setScoreThreshold(float score_threshold) = 0;
+
+    CV_WRAP virtual float getScoreThreshold() = 0;
+
+    /** @brief Set the Non-maximum-suppression threshold to suppress bounding boxes that have IoU greater than the given value
+     *
+     * @param nms_threshold threshold for NMS operation
+     */
+    CV_WRAP virtual void setNMSThreshold(float nms_threshold) = 0;
+
+    CV_WRAP virtual float getNMSThreshold() = 0;
+
+    /** @brief Set the number of bounding boxes preserved before NMS
+     *
+     * @param top_k the number of bounding boxes to preserve from top rank based on score
+     */
+    CV_WRAP virtual void setTopK(int top_k) = 0;
+
+    CV_WRAP virtual int getTopK() = 0;
+
+    /** @brief A simple interface to detect face from given image
+     *
+     *  @param image an image to detect
+     *  @param faces detection results stored in a cv::Mat
+     */
+    CV_WRAP virtual int detect(InputArray image, OutputArray faces) = 0;
+
+    /** @brief Creates an instance of this class with given parameters
+     *
+     *  @param model the path to the requested model
+     *  @param config the path to the config file for compability, which is not requested for ONNX models
+     *  @param input_size the size of the input image
+     *  @param score_threshold the threshold to filter out bounding boxes of score smaller than the given value
+     *  @param nms_threshold the threshold to suppress bounding boxes of IoU bigger than the given value
+     *  @param top_k keep top K bboxes before NMS
+     *  @param backend_id the id of backend
+     *  @param target_id the id of target device
+     */
+    CV_WRAP static Ptr<FaceDetectorYN> create(const String& model,
+                                              const String& config,
+                                              const Size& input_size,
+                                              float score_threshold = 0.9f,
+                                              float nms_threshold = 0.3f,
+                                              int top_k = 5000,
+                                              int backend_id = 0,
+                                              int target_id = 0);
+};
+
+/** @brief DNN-based face recognizer, model download link: https://drive.google.com/file/d/1ClK9WiB492c5OZFKveF3XiHCejoOxINW/view.
+ */
+class CV_EXPORTS_W FaceRecognizerSF
+{
+public:
+    virtual ~FaceRecognizerSF() {};
+
+    /** @brief Definition of distance used for calculating the distance between two face features
+     */
+    enum DisType { FR_COSINE=0, FR_NORM_L2=1 };
+
+    /** @brief Aligning image to put face on the standard position
+     *  @param src_img input image
+     *  @param face_box the detection result used for indicate face in input image
+     *  @param aligned_img output aligned image
+     */
+    CV_WRAP virtual void alignCrop(InputArray src_img, InputArray face_box, OutputArray aligned_img) const = 0;
+
+    /** @brief Extracting face feature from aligned image
+     *  @param aligned_img input aligned image
+     *  @param face_feature output face feature
+     */
+    CV_WRAP virtual void feature(InputArray aligned_img, OutputArray face_feature) = 0;
+
+    /** @brief Calculating the distance between two face features
+     *  @param _face_feature1 the first input feature
+     *  @param _face_feature2 the second input feature of the same size and the same type as _face_feature1
+     *  @param dis_type defining the similarity with optional values "FR_OSINE" or "FR_NORM_L2"
+     */
+    CV_WRAP virtual double match(InputArray _face_feature1, InputArray _face_feature2, int dis_type = FaceRecognizerSF::FR_COSINE) const = 0;
+
+    /** @brief Creates an instance of this class with given parameters
+     *  @param model the path of the onnx model used for face recognition
+     *  @param config the path to the config file for compability, which is not requested for ONNX models
+     *  @param backend_id the id of backend
+     *  @param target_id the id of target device
+     */
+    CV_WRAP static Ptr<FaceRecognizerSF> create(const String& model, const String& config, int backend_id = 0, int target_id = 0);
+};
+
+} // namespace cv
+
+#endif
Original file line number	Diff line number	Diff line change
`@@ -768,5 +768,6 @@ class CV_EXPORTS_W QRCodeDetector`
`768`	`768`	`}`
`769`	`769`
`770`	`770`	`#include "opencv2/objdetect/detection_based_tracker.hpp"`
	`771`	`+#include "opencv2/objdetect/face.hpp"`
`771`	`772`
`772`	`773`	`#endif`