主页 > 产品

使用Python,Keras和OpenCV进行实时面部活体检测

时间:2019-10-15 来源:长空电竞
使用Python,Keras和OpenCV进行实时面部活体检测

你可以在互联网上找到的大多数面部识别算法和研究论文都受到照片攻击。这些方法在检测和识别来自网络摄像头的图像、视频和视频流中的人脸方面非常有效。然而,他们不能区分活人的脸和照片上的脸。这是因为这些算法适用于2D frames。

现在让我们想象一下,我们想要实现一个面部识别开门器。该系统能够很好地区分已知面孔和未知面孔;以一种只对授权人开放的方式。尽管如此,对于一个怀有恶意的人来说,只展示授权人的照片是很容易的。这就是3D探测器适用的地方,就像苹果公司的FaceID一样。但是如果我们没有3D探测器呢?

使用Python,Keras和OpenCV进行实时面部活体检测

奥巴马面部照片示例

本文的目的是实现一种基于眨眼检测的人脸活性检测算法,以阻止照片攻击。该算法通过网络摄像头实时工作,只有在对方眨眼时才显示其姓名。该程序运行如下:

  1. 检测摄像头生成的每个帧中的人脸。
  2. 对于每个检测到的脸,检测眼睛。
  3. 对于每个检测到的眼睛,检测眼睛是睁着的还是闭着的。
  4. 如果在某一时刻检测到眼睛是睁开的,然后又闭上,然后再睁开,我们就得出结论,这个人眨眼了,程序就会显示它的名字(在面部识别开门器的情况下,我们会授权这个人进入)。

对于人脸的检测和识别,您需要安装face_recognition库,它提供了非常有用的深度学习方法来查找和识别图像中的人脸。特别是face_locations、face_encodings和compare_faces函数是最有用的3个函数。face_locations方法可以使用两种方法检测人脸:梯度直方图(HoG)和卷积神经网络(CNN)。本文选择HoG方法。face_encodings函数是一个经过预处理的卷积神经网络,它能够将图像编码成包含128个特征的向量。这个嵌入向量应该表示足够的信息来区分两个不同的人。最后,compare_faces计算两个嵌入向量之间的距离。它将允许我们识别从网络摄像头帧中提取的人脸,并将其嵌入向量与我们数据集中所有编码的人脸进行比较。最接近的向量应该对应于同一个人。

1.已知的面部数据集编码

该算法能够识别自己和巴拉克奥巴马。我选了大约10张照片。下面是处理和编码已知面部数据集的Python代码。

def process_and_encode(images):
known_encodings = []
known_names = []
print("[LOG] Encoding dataset ...")
for image_path in tqdm(images):
# Load image
image = cv2.imread(image_path)
# Convert it from BGR to RGB
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

# detect face in the image and get its location (square boxes coordinates)
boxes = face_recognition.face_locations(image, model='hog')
# Encode the face into a 128-d embeddings vector
encoding = face_recognition.face_encodings(image, boxes)
# the person's name is the name of the folder where the image comes from
name = image_path.split(os.path.sep)[-2]
if len(encoding) > 0 :
known_encodings.append(encoding[0])
known_names.append(name)
return {"encodings": known_encodings, "names": known_names}
使用Python,Keras和OpenCV进行实时面部活体检测

现在我们知道了我们想要识别的每个人的编码,我们可以尝试通过网络摄像头识别和识别人脸。但是,在此之前,我们需要区分面部照片和活人脸部。

2.人脸活性检测

提醒一下,我们的目标是在某一时刻检测到眨眼动作。我训练了一个卷积神经网络来区分眼睛是睁着还是闭着。选择的机器学习模型是LeNet-5,该机器学习模型已经在Closed Eyes In The Wild(CEW)数据集(http://parnec.nuaa.edu.cn/xtan/data/ClosedEyeDatabases.html)上进行了训练。它由大约4800张24x24大小的眼睛图像组成。完整机器学习模型的Python代码如下:

import os
from PIL import Image
import numpy as np
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import AveragePooling2D
from keras.layers import Flatten
from keras.layers import Dense
from keras.models import model_from_json
from keras.preprocessing.image import ImageDataGenerator
from scipy.ndimage import imread
from scipy.misc import imresize, imsave
IMG_SIZE = 24
def collect():
train_datagen = ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
horizontal_flip=True,
)
val_datagen = ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
horizontal_flip=True, )
train_generator = train_datagen.flow_from_directory(
directory="dataset/train",
target_size=(IMG_SIZE, IMG_SIZE),
color_mode="grayscale",
batch_size=32,
class_mode="binary",
shuffle=True,
seed=42
)
val_generator = val_datagen.flow_from_directory(
directory="dataset/val",
target_size=(IMG_SIZE, IMG_SIZE),
color_mode="grayscale",
batch_size=32,
class_mode="binary",
shuffle=True,
seed=42
)
return train_generator, val_generator
def save_model(model):
model_json = model.to_json()
with open("model.json", "w") as json_file:
json_file.write(model_json)
# serialize weights to HDF5
model.save_weights("model.h5")
def load_model():
json_file = open('model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
loaded_model = model_from_json(loaded_model_json)
# load weights into new model
loaded_model.load_weights("model.h5")
loaded_model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
return loaded_model
def train(train_generator, val_generator):
STEP_SIZE_TRAIN=train_generator.n//train_generator.batch_size
STEP_SIZE_VALID=val_generator.n//val_generator.batch_size
print('[LOG] Intialize Neural Network')

model = Sequential()
model.add(Conv2D(filters=6, kernel_size=(3, 3), activation='relu', input_shape=(IMG_SIZE,IMG_SIZE,1)))
model.add(AveragePooling2D())
model.add(Conv2D(filters=16, kernel_size=(3, 3), activation='relu'))
model.add(AveragePooling2D())
model.add(Flatten())
model.add(Dense(units=120, activation='relu'))
model.add(Dense(units=84, activation='relu'))
model.add(Dense(units=1, activation = 'sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit_generator(generator=train_generator,
steps_per_epoch=STEP_SIZE_TRAIN,
validation_data=val_generator,
validation_steps=STEP_SIZE_VALID,
epochs=20
)
save_model(model)
def predict(img, model):
img = Image.fromarray(img, 'RGB').convert('L')
img = imresize(img, (IMG_SIZE,IMG_SIZE)).astype('float32')
img /= 255
img = img.reshape(1,IMG_SIZE,IMG_SIZE,1)
prediction = model.predict(img)
if prediction < 0.1:
prediction = 'closed'
elif prediction > 0.9:
prediction = 'open'
else:
prediction = 'idk'
return prediction
def evaluate(X_test, y_test):
model = load_model()
print('Evaluate model')
loss, acc = model.evaluate(X_test, y_test, verbose = 0)
print(acc * 100)
if __name__ == '__main__':
train_generator , val_generator = collect()
train(train_generator,val_generator)
使用Python,Keras和OpenCV进行实时面部活体检测

使用Python,Keras和OpenCV进行实时面部活体检测

使用Python,Keras和OpenCV进行实时面部活体检测

在评估此机器学习模型时,准确率达到了94%。

每次我们检测到一只眼睛,我们就用我们的机器学习模型预测它的状态,并跟踪眼睛状态。因此,使用下面的函数,检测眨眼变得非常容易,该函数试图在眼睛状态历史中找到closed-open-closed模式。

def isBlinking(history, maxFrames):
""" @history: A string containing the history of eyes status
where a '1' means that the eyes were closed and '0' open.
@maxFrames: The maximal number of successive frames where an eye is closed """
for i in range(maxFrames):
pattern = '1' + '0'*(i+1) + '1'
if pattern in history:
return True
return False
使用Python,Keras和OpenCV进行实时面部活体检测

3.活体人脸识别

我们几乎具备了建立人脸识别算法的所有要素。我们只需要一种实时检测人脸和眼睛的方法。我使用openCV预训练的Haar-cascade分类器来完成这些任务

def detect_and_display(model, video_capture, face_detector, open_eyes_detector, left_eye_detector, right_eye_detector, data, eyes_detected):
frame = video_capture.read()
# resize the frame
frame = cv2.resize(frame, (0, 0), fx=0.6, fy=0.6)
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

# Detect faces
faces = face_detector.detectMultiScale(
gray,
scaleFactor=1.2,
minNeighbors=5,
minSize=(50, 50),
flags=cv2.CASCADE_SCALE_IMAGE
)
# for each detected face
for (x,y,w,h) in faces:
# Encode the face into a 128-d embeddings vector
encoding = face_recognition.face_encodings(rgb, [(y, x+w, y+h, x)])[0]
# Compare the vector with all known faces encodings
matches = face_recognition.compare_faces(data["encodings"], encoding)
# For now we don't know the person name
name = "Unknown"
# If there is at least one match:
if True in matches:
matchedIdxs = [i for (i, b) in enumerate(matches) if b]
counts = {}
for i in matchedIdxs:
name = data["names"][i]
counts[name] = counts.get(name, 0) + 1
# The known encoding with the most number of matches corresponds to the detected face name
name = max(counts, key=counts.get)
face = frame[y:y+h,x:x+w]
gray_face = gray[y:y+h,x:x+w]
eyes = []

# Eyes detection
# check first if eyes are open (with glasses taking into account)
open_eyes_glasses = open_eyes_detector.detectMultiScale(
gray_face,
scaleFactor=1.1,
minNeighbors=5,
minSize=(30, 30),
flags = cv2.CASCADE_SCALE_IMAGE
)
# if open_eyes_glasses detect eyes then they are open
if len(open_eyes_glasses) == 2:
eyes_detected[name]+='1'
for (ex,ey,ew,eh) in open_eyes_glasses:
cv2.rectangle(face,(ex,ey),(ex+ew,ey+eh),(0,255,0),2)

# otherwise try detecting eyes using left and right_eye_detector
# which can detect open and closed eyes
else:
# separate the face into left and right sides
left_face = frame[y:y+h, x+int(w/2):x+w]
left_face_gray = gray[y:y+h, x+int(w/2):x+w]
right_face = frame[y:y+h, x:x+int(w/2)]
right_face_gray = gray[y:y+h, x:x+int(w/2)]
# Detect the left eye
left_eye = left_eye_detector.detectMultiScale(
left_face_gray,
scaleFactor=1.1,
minNeighbors=5,
minSize=(30, 30),
flags = cv2.CASCADE_SCALE_IMAGE
)
# Detect the right eye
right_eye = right_eye_detector.detectMultiScale(
right_face_gray,
scaleFactor=1.1,
minNeighbors=5,
minSize=(30, 30),
flags = cv2.CASCADE_SCALE_IMAGE
)
eye_status = '1' # we suppose the eyes are open
# For each eye check wether the eye is closed.
# If one is closed we conclude the eyes are closed
for (ex,ey,ew,eh) in right_eye:
color = (0,255,0)
pred = predict(right_face[ey:ey+eh,ex:ex+ew],model)
if pred == 'closed':
eye_status='0'
color = (0,0,255)
cv2.rectangle(right_face,(ex,ey),(ex+ew,ey+eh),color,2)
for (ex,ey,ew,eh) in left_eye:
color = (0,255,0)
pred = predict(left_face[ey:ey+eh,ex:ex+ew],model)
if pred == 'closed':
eye_status='0'
color = (0,0,255)
cv2.rectangle(left_face,(ex,ey),(ex+ew,ey+eh),color,2)
eyes_detected[name] += eye_status
# Each time, we check if the person has blinked
# If yes, we display its name
if isBlinking(eyes_detected[name],3):
cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
# Display name
y = y - 15 if y - 15 > 15 else y + 15
cv2.putText(frame, name, (x, y), cv2.FONT_HERSHEY_SIMPLEX,0.75, (0, 255, 0), 2)
return frame
使用Python,Keras和OpenCV进行实时面部活体检测

使用Python,Keras和OpenCV进行实时面部活体检测

上述函数是用于检测和识别真实面部的Python代码。它需要的参数:

  • model:我们的open/closed眼睛分类器
  • video_capture:流视频
  • face_detector:Haar-cascade face分类器。使用haarcascade_frontalface_alt.xml
  • open_eyes_detector:Haar-cascade 睁眼的 分类器。使用haarcascade_eye_tree_eyeglasses.xml
  • left_eye_detector:Haar-cascade左眼分类器。使用haarcascade_lefteye_2splits.xml,它可以检测睁开或闭上的眼睛。
  • right_eye_detector:Haar-cascade右眼分类器。使用haarcascade_righteye_2splits.xml,它可以检测睁开或闭上的眼睛。
  • data:已知编码和已知人名的字典
  • eyes_detected:一个字典,包含每个眼睛状态历史记录。

代码解析:

  • 在第2-4行,我们从webcam流中获取一个帧,然后调整它的大小以加快计算速度。
  • 在第10行,我们从帧中检测人脸,然后在第21行,我们将其编码为128-d向量。
  • 在第23-38行,我们将这个向量与已知的面部编码进行比较,并通过计算匹配的数量来确定此人的姓名。选择匹配数量最大的一个。
  • 从第45行开始,我们尝试将在面部框中进行眼睛检测。首先,我们尝试使用open_eye_detector检测睁眼。如果探测器成功,那么在第54行, 我们在眼睛状态历史中添加“1”,这意味着眼睛是睁着的,因为open_eye_detector无法检测闭着的眼睛。否则,如果第一个分类器失败了(可能是因为眼睛是闭着的,或者仅仅是因为它没有识别眼睛),那么我们尝试使用left_eye和right_eye检测器。为此我们需要将人脸分为左右两部分,分别提供给每个分类器。
  • 从第92行开始,我们提取眼部,我们使用之前训练的机器学习模型预测眼睛是否闭着。如果我们检测到一只眼闭着,我们假设双眼都闭合了,我们在眼睛状态历史中添加“0”。否则我们认为眼睛是睁着的。
  • 最后在第110行我们使用isBlinking()用于检测眨眼的函数,如果此人眨眼,我们会显示其姓名。

4.程序其他部分代码

import os
import cv2
import face_recognition
import numpy as np
from tqdm import tqdm
from collections import defaultdict
from imutils.video import VideoStream
from eye_status import *
def init():
face_cascPath = 'haarcascade_frontalface_alt.xml'
# face_cascPath = 'lbpcascade_frontalface.xml'
open_eye_cascPath = 'haarcascade_eye_tree_eyeglasses.xml'
left_eye_cascPath = 'haarcascade_lefteye_2splits.xml'
right_eye_cascPath ='haarcascade_righteye_2splits.xml'
dataset = 'faces'
face_detector = cv2.CascadeClassifier(face_cascPath)
open_eyes_detector = cv2.CascadeClassifier(open_eye_cascPath)
left_eye_detector = cv2.CascadeClassifier(left_eye_cascPath)
right_eye_detector = cv2.CascadeClassifier(right_eye_cascPath)
print("[LOG] Opening webcam ...")
video_capture = VideoStream(src=0).start()
model = load_model()
print("[LOG] Collecting images ...")
images = []
for direc, _, files in tqdm(os.walk(dataset)):
for file in files:
if file.endswith("jpg"):
images.append(os.path.join(direc,file))
return (model,face_detector, open_eyes_detector, left_eye_detector,right_eye_detector, video_capture, images)
if __name__ == "__main__":
(model, face_detector, open_eyes_detector,left_eye_detector,right_eye_detector, video_capture, images) = init()
data = process_and_encode(images)
eyes_detected = defaultdict(str)
while True:
frame = detect_and_display(model, video_capture, face_detector, open_eyes_detector,left_eye_detector,right_eye_detector, data, eyes_detected)
cv2.imshow("Face Liveness Detector", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cv2.destroyAllWindows()
video_capture.stop()
使用Python,Keras和OpenCV进行实时面部活体检测