Eureka! An Accidental Improvement
In early 2013, my manager at Digital Signal Corp (DSC) asked me to investigate using Gabor filters to improve our 3D face recognition. Gabor filters were successfully used for iris recognition and 2D face recognition. However, nobody had used 3D Face images. I did a lot of research, and I determined the only way to move forward was to use 2.5D images. A 2.5D image is a regular image but the pixel intensity represents depth.
I happened to use the function griddata in Matlab to do this conversion while I was doing the research. I was working on FRGC data collected by a Minolta scanner. This had 100k points per face with a high accuracy. The development went well, and I found some performance improvement. The next step was using data from DSC’s sensor which was 10k points per face, a lower accuracy, and for some faces, the point distribution on the face could be very uneven.
In the processing of using DSC’s data, I found the performance improvement was huge, but not of the Gabor filters. I found by just using the data after upsampling, the performance improved significantly. This helped fill holes that caused issues with the face registration process, and it allowed some surface smoothing using local neighbors. I got to work implementing it in C++, and it turned out to be a larger task. I wasted some time trying to match algorithms in Matlab, but it took me a bit to make sure I knew which griddata version I was using.
I had quite a few learnings from there, but the main result was 2 fold: I improved recognition and somehow, I shaved off 25% of the computing time. Part of this happened because of triangulation. When I upsampled, I made some initial triangulation, and then I efficiently upsampled using interpolation on each triangle face by scanning through the triangle list and determining what XY points on the grid were found in a given triangle. This caused the entire snippet to be super fast.
Each face had a reference model deformed into it, and the result was our standardized 3D face for PCA/LDA. This process was point to triangle. I improved it and made it point to surface using this upsampled data and some surface parameters generated by the 8 nearest neighbors. All of this added to more accuracy in the face registration process which improved the PCA/LDA results.
In the end, this change resulted in a ~70% error reduction of false rejects (while holding the false accepts constant), and it decreased the processing time by 25%. It was quite a win, but the time from initial development to full implementation was close to a year. Other projects were going on, but even still, this project took much longer than I originally planned. To top it off, the method also functioned in a way to replace a 3D correct 2D color image process we had been using to improve 2D recognition performance.
Machine Learning Engineer, Transformers and Einstein Summation Expert
6 年A very good read, On the uosamplinh phase you said that you are using nearest neighbors, did you thought standard image feutre extractor like SIFT or HOG? They could prove useful for giving youths size and orientation of individual parts of the face. Another thing that came to my mind is Facebook used SIFT+PCA+Fisher vectors for their face detection pipeline before the deep network era and I also experienced that fisher vectors can come handy in feature extraction by simulating the feature space like a GMM and giving you the first and second norms from each cluster. This is a very nice article and hope to see new articles like this.