【MLFP】《Face Presentation Attack with Latex Masks in Multispectral Videos》

CVPRW-2017

---

---

1 Background and Motivation

With the development of deep learning , Face recognition is more and more widely used ,face presentation attacks are becoming a serious point of concern

this paper , author In order to understand the impact of spectrums in detecting mask based presentation attacks, Multispectral （RGB / NIR / Thermal） Biopsy data set Multispectral Latex Mask based Video Face Prepresentation Attack (MLFP)

2 Related Work

nothing

3 Advantages / Contributions

Open multispectral datasets , And analyzed base method Biopsy on each spectrum / distinguish ying performance

4 Method

The following are the traditional feature extraction methods involved in the experiment

• HOG
• ULBP
• BSIF
• LPQ
• RDWT + Haralick

5 Experiments

5.1 Datasets and Metrics

1） Data sets ：MLFP

Comparison with Existing Databases

• Single Spectrum vs Multispectral
• Hard Masks vs Soft Masks：hard resin mask, Hard resin , Soft silica gel / Soft latex （ It fits the skin better ）
• Images vs Videos
• Constrained vs Unconstrained：unrestricted backgrounds and illumination（ Indoor and outdoor ）

Acquisition mode of each mode

• RGB：Android smartphones with 8 megapixels camera,1280x720
• Thermography ：FLIR ONE thermal imager for Android2,640x480
• NIR：Microsoft Kinect for Windows V2,424 × 512

2） The evaluation index ：ROC / EER

What is? ROC curve ？ Why use ROC? as well as AUC The calculation of

Receiver Operating Characteristic

Abscissa for FPR（False positive rate False positive rate ）, How many of all negative cases are predicted to be positive , FP/(FP+TN);

Vertical coordinate is true positive rate TPR（True postive rate）, How many real positive examples are predicted TP/(TP+FN);

---

come from [ The concept of depth ]· Evaluation indicators EER(Equal Error Rate) Introduce

EER（Equal Error Rate） It is a biometric security system algorithm , The threshold value used to predetermine its error acceptance rate and its error rejection rate . When the rates are equal , The common value is called equal error rate . This value indicates that the proportion of false acceptance is equal to the proportion of false rejection . The lower the error rate value , The higher the accuracy of biometric system .

TP,FP,TN,FN,FAR,FRR,HTER And other common experimental indicators

EER The smaller the better. ！

5.2 Experiments

5.2.1 Effect of Face Obfuscation on Face Verification Algorithms across Multiple Spectrums

Face recognition

10 personal
gallery set（ Equivalent to training set ）：10 real videos, Everyone 1 individual
probe set1（ Equivalent to test set ）:40 real videos, Everyone 4 individual ,to illustrate the performance of face verification under regular/normal conditions
probe set2（ Equivalent to test set ）:400 presentation attack videos, Everyone 40 individual ,to examine if an individual is able to conceal his/her identity using face masks

result

COTS：a commercial off-the-shelf system（ Face recognition system ）
ULBP：Uniform Local Binary Patterns （ Because there is no commercial thermal imaging face recognition system , The author uses this method to compare the features of face recognition ）

Under each mode , introduce mask after , All confuse the face information , It interferes with the face recognition system ！ Thermal imaging mode real and attack The effect is not very good, but also understandable , After all, there are no details on the face , It is difficult to extract the features of discrimination ！

5.2.2 Evaluating Face Presentation Attack Detection Algorithms across Multiple Spectrums

Face detection in vivo

1） Data set division

Partition based on Subjects,10 personal ,3 / 3 / 4 Three groups of , One training and two tests , Test set not encode any subject-specific information

Partition based on Masks,3 Kind of paper attack ,7 Kind of latex mask attack ,1 paper + 2 mask train, The rest of the test, The attack method of half human and half face is put in the test set

2） Image input mode

The biopsy used the whole face ,applying PAD algorithm on the input face images yield better results than detected face images only.

3）Video-based Face Presentation Attack Detection

For computing video-based results, the frame-based scores are combined by computing the average of all frames to produce the final video classification result.

have a look SVM Distribution of scores

The performance is best in thermal imaging mode

Real vs Attack classification analysis

RGB,real 69.7% acc < attack 72.9%,NIR And thermal imaging real > attack

Analysis of PAD in varying environments
indoor acc Larger than outdoor ,be attributed to the presence of uncontrolled background

4）Frame-based Face Presentation Attack Detection

The performance is best in thermal imaging mode

Under each mode frame-based than video-based Poor performance

This result highlights the importance of videos for face PAD as they are a larger source of information which turns out to be beneficial for detecting instances of face presentation attacks.

6 Conclusion（own） / Future work

• Dataset address ：http://iab-rubric.org/resources/face
  Data set description ：http://iab-rubric.org/index.php/MLFP

• gallery set and probe set

• ROC / EER

• (i) evasion, where the key aim is face obfuscation or identity concealment（3Dmask, Show your eyes and mouth ）
  Face obfuscation refers to concealment of one’s identity to avoid recognition by wearing a mask or any other mechanism.
  eg：immune evasion Immune evasion ( After the parasite invades the host with normal immune function , Some can escape the immune attack of the host and continue to live
  (ii) impersonation, where the objective is to impersonate the identity of another individual.（2D paper, I don't know if I understand it wrong , There is no other explanation for this word in the text ）

  Read the following explanation , Be suddenly enlightened （ Overview of in vivo detection —01）

  

• Gender / Age / Wrinkle / Hair / background / Modality / scene

• multi-spectral video-based face PAD algorithm

• Insufficient ：The dataset was collected independently for different channels and hence lacks temporal and spatial alignment between the channels. Experiments with fusion could not be performed since the channels were recorded independently

• 16bit turn 8bit

  def transfer_16bit_to_8bit(image_path):
      image_16bit = cv2.imread(image_path, cv2.IMREAD_UNCHANGED)
      min_16bit = np.min(image_16bit)
      max_16bit = np.max(image_16bit)
      image_8bit = np.array(np.rint(255 * ((image_16bit - min_16bit) / (max_16bit - min_16bit))), dtype=np.uint8)
      return image_8bit
  

• Read mat file , Can use scipy library ,mat The storage mode is dict, according to key To analyze the key information

  import scipy.io as scio
  data = scio.loadmat(‘example.mat')
  data[‘loss']
  

• The dataset was collected independently for different channels and hence lacks temporal and spatial alignment
  between the channels.