Mastering Face Recognition in Flutter: A Comprehensive Guide to Face Detection and Optimal Shot Extraction
If you're tasked with integrating face recognition into your mobile application and have chosen Flutter for development, you might encounter challenges in finding a suitable SDK or API. Fortunately, this article will guide you through the initial steps of face recognition (detection and optimal shot extraction) using Flutter and 3DiVi SDK 3.18.
In the latest 3DiVi Face SDK release, we've enhanced our Flutter API. With this API, you can:
The process of detection and optimal shot extraction looks like this: the client application captures frames from the smartphone camera, detects the user's face, selects the best shot, and forwards it to the server. The server application then evaluates the shot and provides feedback.
To accomplish face recognition and optimal shot selection, we utilize the VideoWorker class, which is tailored for tracking faces in video files and streams. In our application, the VideoWorker object runs on two distinct threads. On one thread, it consistently receives frames from the front camera.
format = Format.FORMAT_YUV_NV21;
convertRAW(img.planes, _data); // concatenates Android camera image planes
// into one array
_ri = RawImageF(img.width, img.height, format, _data.pointer!.cast()); // Create RawImage
//object
_videoWorker.addVideoFrame(_ri!, time); // Pass frame and current timestamp to VideoWorker
Concurrently, on the second thread, results from the VideoWorker are retrieved at regular intervals, for instance, every 50 milliseconds.
final callbackData = videoWorker.poolTrackResults(); final rawSamples = callbackData.trackingcallback_data.samples; final samplesQuality = callbackData.tracking_callback_data.samples_quality;
The poolTrackResults method returns tracking outcomes, encompassing objects like TrackingCallbackData, TemplateCreatedCallbackData, and TrackingLostCallbackData. For our needs, we focus on TrackingCallbackData, which offers data on face detections and their quality. Using methods from RawSample, we can get the track identifier for an individual and also record the quality rating of the detection in the frame.
领英推荐
if (rawSamples.length > 0) { for(var i = 0; i < rawSamples.length; i+=1) { personId = rawSamples[i].getID(); sampleQuality = samplesQuality[i]
After storing this information for a few real-time seconds, we can identify the highest quality face image based on the person's identifier. For the RawSample object corresponding to the best image quality, we can then obtain the best shot.
Uint8List bestShot = bestRawSample.cutFaceImage(ImageFormatCode.IMAGE_FORMAT_JPG, FaceCutType.FACE_CUT_FULL_FRONTAL);
The above example demonstrates the ease with which you can integrate the Face SDK Flutter API into your project and address your challenges. To learn more about the capabilities of the Flutter API, check out the following resources:
Note: To use our Flutter API in your applications, please purchase the ESL Mobile license.
With the power of Flutter and the capabilities of the 3DiVi Face SDK, you can create a robust face recognition system that not only detects faces but also ensures that the best shots are captured and verified. Whether you're a seasoned developer or just starting out, this combination of tools and technology can help you achieve your project goals efficiently and effectively.