On the basis of introducing Python foundation, OpenCV foundation, computer vision theoretical basis and the introduction of in-depth learning theory, 40 representative typical cases in the field of computer vision are introduced. These cases include traditional cases (number recognition, answer card recognition, object count, defect detection, etc.), in-depth learning cases (image classification, style migration, posture recognition, instance segmentation, etc.), and face recognition cases (expression recognition, driver fatigue monitoring, gender and age recognition).

This paper briefly intercepts part of the book of the 38th case (driver fatigue monitoring). For more information, you can refer to Chapter 28, Face Recognition Applications.

Fatigue driving is very easy to cause traffic accidents, one of its important manifestations is sleepiness, eyes will remain closed for more than the normal blinking time. Therefore, the aspect ratio (aspect ratio) of the eye can be used to judge whether the eye is closed (blinking) and whether the driver is driving under fatigue.

As shown in Figure 1, a demonstration of opening and closing eyes is performed, where:

1. The left side is normally open, with a height-to-width ratio of about 0.3.
2. The right side is closed (blinking) and the ratio of eye height to width is about 0.

1 eye model

Specifically, Dlib uses six key points to label the eyes, so its aspect ratio is calculated as:

Next, you need to determine the difference between normal blinking and prolonged eye closure (fatigue). Normally, blinking is a flash, while fatigued closing is relatively long. For example, in Figure 2:

1. In the figure above, the aspect ratio is only at a small value (about 0) in one instant and is normal for the rest of the time (about 0.3), corresponding to a normal blink.
2. In the figure below, the aspect ratio is at a low value (about 0) for a longer period of time, corresponding to a long period of eye closure. If its duration exceeds the threshold we have previously specified, the state is judged to be fatigue.

2. Eye aspect ratio

Therefore, on the basis of the eye aspect ratio, the duration of the eye aspect ratio should also be measured. Reflected in a video frame, to add a counter, the counter works as follows:

1. When the aspect ratio of the eye is less than 0.3, the eye of the current frame is considered closed, and the counter is added 1. If the counter is larger than the threshold value (e.g. 48), the eye closure time is considered too long, suggesting a risk; If the threshold is less than or equal to and the time the eye closes is within the normal range, the current closure is merely blinking without any additional processing.
2. When the eye aspect ratio is greater than or equal to 0.3, the current frame eye processing is open and the counter is cleared. Ensure that the counter can count again from zero the next time you close your eyes.

Based on the above analysis, the specific implementation flowchart is shown in Figure 3.

3 Flowchart

In this case, Dlib is a modern toolkit that contains machine learning algorithms and tools to construct software in programs to solve complex real-world problems. It is widely used in the fields of robots, embedded devices, mobile phones and large high performance computing environments by industry and academia. Dlib's open source license allows users to use it for free in any application.

[Core Code Interception (with full code in the book)] Fatigue detection using Dlib.

# =================Get the key point set corresponding to the left eye and right eye in the image=======

def getEYE(image,rect):

    landmarks=predictor(image, rect)

    # Key points are treated as (x,y)

    shape = np.matrix([[p.x, p.y] for p in landmarks.parts()])

    # Calculating left and right eyes

    leftEye = shape[42:48]   #Left eye, key point index from 42 to 47 (excluding 48)

    rightEye = shape[36:42]  #Right eye, key point index from 36 to 41 (excluding 42)

    return leftEye,rightEye

# =============Calculates the aspect ratio of the eye (closing or blinking is too small for less than 0.3 and opening is more than 0.3)======

def eye_aspect_ratio(eye):

    # The eyes are represented by six dots. Two from the top and two from the bottom, one from the left and one from the right, with the following structure:

    #---------------------------------------------

    #      1    2

    # 0 < --- These are the six key points of the eye

    #      5    4

    #---------------------------------------------

    # Euclidean distance (two distances 1 and 5, 2 and 4 in the vertical direction)

    A = dist.euclidean(eye[1], eye[5])

    B = dist.euclidean(eye[2], eye[4])

    # Euclidean distance (horizontal distance 0 and 3)

    C = dist.euclidean(eye[0], eye[3])

    #Aspect ratio

    ear = (A + B) / (2.0 * C)

    return ear

# ====================Calculates the aspect ratio mean of two eyes=================

def earMean(leftEye,rightEye):

    # Left eye aspect ratio leftEAR and right eye aspect ratio rightEAR were calculated

    leftEAR = eye_aspect_ratio(leftEye)

    rightEAR = eye_aspect_ratio(rightEye)

    # Mean Processing

    ear = (leftEAR + rightEAR) / 2.0 

    return ear

# =================Draw the eye frame (the orbital bounding box)==================

def drawEye(eye):

    # Circle your eyes 1:convexHull to get the convex hull

    eyeHull = cv2.convexHull(eye)

    # Circle your eyes 2:drawContours to draw the contour of the convex hull

    cv2.drawContours(frame, [eyeHull], -1, (0, 255, 0), 1)

As shown in Figure 2, when the aspect ratio of the eye is below the threshold for a long time, a danger alert "DANGEROUS!!!" is displayed.

Figure 2 shows the effect

Chapter 28, The Application of Face Recognition, also introduces the basic principles and implementation process of face expression recognition, facility, gender and age recognition cases. Welcome to Chapter 28, Face Application Cases, Computer Vision 40 Cases - From Beginning to Deep Learning (OpenCV-Python).

The book contains examples of face recognition:

1. Face Detection
2. Face recognition
3. Outlining Five Officials
4. face alignment
5. Emotional Recognition
6. Fatigue Driving Detection
7. Disguise Self
8. Sex and Age Identification