FaceDetectionUsingFfmpegWithDlibInUnity

In a Unity environment, images streamed using FFmpeg are Mat-converted using OpenCV and face detection is performed by Dlib.

Environmental Construction

1. Create a Unity project with a 3D template.

2. Window→Package Manager → Set "Packages" to "My Assets" in the upper left corner and import "OpenCV For Unity".

   

• Set up the system using the setup tool.

  

3. Import and setup Dlib Face Landmark Detector in the same way.

   

4. Import and Setup FFmpeg for Unity.

5. Import Runtime Inspector & Hierarchy.

6. Import In-game Debug Console.

7. Import FfmpegWithOpenCVForUnityExample.unitypackage.

8. Rewrite the C# script named FfplayToMatHelperExample.cs in Assets/FfmpegWithOpenCVForUnityExample/FfplayToMatHelperExample/ (see diagram below) to the code named FfPlayStreamingFaceDetectionExample.cs

   

By setting the URL appropriately, face detection can be performed from streaming video. (However, we have confirmed that the delay is quite large.)

It is hard to see, but it detects a face and marks around the face. (All sky equirectangular format)