# 1, Preliminary work

What should children do if they are always distracted in class? The attention detection program is arranged! Driver fatigue driving how to do, attention detection program arrangement! This paper will achieve the purpose of detecting attention through the recognition of human eye state ⏳ Add confusion matrix module.

🚀 My environment:

- Locale: Python 3 six point five
- compiler: jupyter notebook
- Deep learning environment: tensorflow2 four point one
- Data link: https://pan.baidu.com/s/1kIk75W8vLC1TkjKaA0odLw Extraction code: nolr

🚀 In depth learning newcomers must see:

- Xiaobai introduction to in-depth learning Chapter 1: configuring in-depth learning environment
- Introduction to Xiaobai deep learning | Chapter 2: use of compiler - Jupiter notebook

🚀 Previous highlights:

- 100 cases of deep learning convolutional neural network (CNN) to realize mnist handwritten numeral recognition | day 1
- 100 cases of deep learning - convolutional neural network (CNN) color picture classification | day 2
- 100 cases of deep learning - convolutional neural network (CNN) garment image classification | day 3
- 100 cases of deep learning - convolutional neural network (CNN) flower recognition | day 4
- 100 cases of deep learning - convolutional neural network (CNN) weather recognition | day 5
- 100 cases of deep learning - convolutional neural network (VGG-16) to identify the pirate king straw hat group | day 6
- 100 cases of deep learning - convolutional neural network (VGG-19) to identify the characters in the spirit cage | day 7
- 100 cases of deep learning - convolutional neural network (ResNet-50) bird recognition | day 8
- 100 cases of deep learning - circular neural network (RNN) to achieve stock prediction | day 9
- 100 cases of deep learning - circular neural network (LSTM) to realize stock prediction | day 10
- 100 cases of deep learning - convolutional neural network (AlexNet) hand-in-hand teaching | day 11
- 100 cases of deep learning - convolutional neural network (CNN) identification verification code | day 12
- 100 cases of deep learning - convolutional neural network (perception V3) recognition of sign language | day 13
- 100 cases of deep learning - convolution neural network (Inception-ResNet-v2) recognition of traffic signs | day 14
- 100 cases of deep learning - convolutional neural network (CNN) for license plate recognition | day 15
- 100 cases of in-depth learning - convolutional neural network (CNN) to identify the Magic Baby Xiaozhi group | day 16

🚀 From column: 100 cases of deep learning

## 1. Set GPU

If you are using a CPU, you can comment out this part of the code.

import tensorflow as tf gpus = tf.config.list_physical_devices("GPU") if gpus: tf.config.experimental.set_memory_growth(gpus[0], True) #Set the amount of GPU video memory and use it on demand tf.config.set_visible_devices([gpus[0]],"GPU") # Print the graphics card information and confirm that the GPU is available print(gpus)

[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

## 2. Import data

import matplotlib.pyplot as plt # Support Chinese plt.rcParams['font.sans-serif'] = ['SimHei'] # Used to display Chinese labels normally plt.rcParams['axes.unicode_minus'] = False # Used to display negative signs normally import os,PIL # Set random seeds so that the results can be reproduced as much as possible import numpy as np np.random.seed(1) # Set random seeds so that the results can be reproduced as much as possible import tensorflow as tf tf.random.set_seed(1) import pathlib

data_dir = "D:/jupyter notebook/DL-100-days/datasets/017_Eye_dataset" data_dir = pathlib.Path(data_dir)

## 3. View data

image_count = len(list(data_dir.glob('*/*'))) print("The total number of pictures is:",image_count)

Total number of pictures: 4307

# 2, Data preprocessing

## 1. Load data

Using image_ dataset_ from_ The directory method loads the data from the disk into TF data. In dataset

batch_size = 64 img_height = 224 img_width = 224

Students with TensorFlow version 2.2.0 may encounter module 'TensorFlow keras. preprocessing' has no attribute 'image_ dataset_ from_ The error of 'directory' is reported. Just upgrade TensorFlow.

""" about image_dataset_from_directory()Please refer to the following article for details: https://mtyjkh.blog.csdn.net/article/details/117018789 """ train_ds = tf.keras.preprocessing.image_dataset_from_directory( data_dir, validation_split=0.2, subset="training", seed=12, image_size=(img_height, img_width), batch_size=batch_size)

Found 4307 files belonging to 4 classes. Using 3446 files for training.

""" about image_dataset_from_directory()Please refer to the following article for details: https://mtyjkh.blog.csdn.net/article/details/117018789 """ val_ds = tf.keras.preprocessing.image_dataset_from_directory( data_dir, validation_split=0.2, subset="validation", seed=12, image_size=(img_height, img_width), batch_size=batch_size)

Found 4307 files belonging to 4 classes. Using 861 files for validation.

We can use class_names the label of the output dataset. The labels will correspond to the directory name in alphabetical order.

class_names = train_ds.class_names print(class_names)

['close_look', 'forward_look', 'left_look', 'right_look']

## 2. Visual data

plt.figure(figsize=(10, 5)) # The width of the figure is 10 and the height is 5 plt.suptitle("Data display") for images, labels in train_ds.take(1): for i in range(8): ax = plt.subplot(2, 4, i + 1) ax.patch.set_facecolor('yellow') plt.imshow(images[i].numpy().astype("uint8")) plt.title(class_names[labels[i]]) plt.axis("off")

## 3. Recheck the data

for image_batch, labels_batch in train_ds: print(image_batch.shape) print(labels_batch.shape) break

(64, 224, 224, 3) (64,)

- Image_batch is the tensor of the shape (8, 224, 224, 3). This is a batch of 8 pictures with shape of 240x240x3 (the last dimension refers to color channel RGB).
- Label_batch is the tensor of shape (8,), and these labels correspond to 8 pictures

## 4. Configure dataset

- shuffle(): scramble data. For a detailed description of this function, please refer to: https://zhuanlan.zhihu.com/p/42417456
- prefetch(): prefetch data and speed up operation. For details, please refer to my previous two articles, which are explained in them.
- cache(): cache data sets into memory to speed up operation

AUTOTUNE = tf.data.AUTOTUNE train_ds = train_ds.cache().shuffle(1000).prefetch(buffer_size=AUTOTUNE) val_ds = val_ds.cache().prefetch(buffer_size=AUTOTUNE)

# 3, Call the official network model

model = tf.keras.applications.VGG16() # Print model information model.summary()

Model: "vgg16" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) [(None, 224, 224, 3)] 0 _________________________________________________________________ block1_conv1 (Conv2D) (None, 224, 224, 64) 1792 _________________________________________________________________ ...... _________________________________________________________________ flatten (Flatten) (None, 25088) 0 _________________________________________________________________ fc1 (Dense) (None, 4096) 102764544 _________________________________________________________________ fc2 (Dense) (None, 4096) 16781312 _________________________________________________________________ predictions (Dense) (None, 1000) 4097000 ================================================================= Total params: 138,357,544 Trainable params: 138,357,544 Non-trainable params: 0 _________________________________________________________________

# 4, Set dynamic learning rate

Here are the advantages and disadvantages of high learning rate and low learning rate.

- High learning rate
- Advantages: 1. Speed up the learning rate. 2. It helps to jump out of the local optimal value.
- Disadvantages: 1. Model training does not converge. 2. Only using the college attendance rate can easily lead to the inaccuracy of the model.

- Low learning rate
- Advantages: 1. It is conducive to model convergence and model refinement. 2. 2. Improve model accuracy.
- Disadvantages: 1. It is difficult to jump out of the local optimal value. 2. Slow convergence.

Note: the dynamic learning rate set here is exponential decay. Before each epoch starts, the learning_rate will be reset to the initial_learning_rate, and then the attenuation will start again. The calculation formula is as follows:

learning_rate = initial_learning_rate * decay_rate ^ (step / decay_steps)

# Set initial learning rate initial_learning_rate = 1e-4 lr_schedule = tf.keras.optimizers.schedules.ExponentialDecay( initial_learning_rate, decay_steps=20, # Knock on the blackboard!!! This refers to steps, not epochs decay_rate=0.96, # LR will become decay after one attenuation_ rate*lr staircase=True) # Feed exponential decay learning rate into optimizer optimizer = tf.keras.optimizers.Adam(learning_rate=lr_schedule)

# 5, Compile

Before preparing to train the model, you need to make some more settings. The following is added in the compilation step of the model:

- Loss function (loss): used to measure the accuracy of the model during training.
- optimizer: determines how the model updates based on the data it sees and its own loss function.
- metrics: used to monitor training and testing steps. The following example uses the accuracy rate, that is, the ratio of images correctly classified.

model.compile(optimizer=optimizer, loss ='sparse_categorical_crossentropy', metrics =['accuracy'])

# 6, Training model

epochs = 10 history = model.fit( train_ds, validation_data=val_ds, epochs=epochs )

Epoch 1/10 54/54 [==============================] - 29s 377ms/step - loss: 1.0835 - accuracy: 0.6709 - val_loss: 0.6919 - val_accuracy: 0.7933 Epoch 2/10 54/54 [==============================] - 14s 268ms/step - loss: 0.2332 - accuracy: 0.9248 - val_loss: 0.1173 - val_accuracy: 0.9628 Epoch 3/10 54/54 [==============================] - 14s 259ms/step - loss: 0.1072 - accuracy: 0.9634 - val_loss: 0.0771 - val_accuracy: 0.9779 Epoch 4/10 54/54 [==============================] - 14s 256ms/step - loss: 0.0663 - accuracy: 0.9794 - val_loss: 0.0566 - val_accuracy: 0.9826 Epoch 5/10 54/54 [==============================] - 14s 258ms/step - loss: 0.0480 - accuracy: 0.9855 - val_loss: 0.0609 - val_accuracy: 0.9768 Epoch 6/10 54/54 [==============================] - 14s 258ms/step - loss: 0.0431 - accuracy: 0.9852 - val_loss: 0.0597 - val_accuracy: 0.9768 Epoch 7/10 54/54 [==============================] - 14s 256ms/step - loss: 0.0289 - accuracy: 0.9910 - val_loss: 0.0539 - val_accuracy: 0.9837 Epoch 8/10 54/54 [==============================] - 14s 259ms/step - loss: 0.0221 - accuracy: 0.9927 - val_loss: 0.0626 - val_accuracy: 0.9744 Epoch 9/10 54/54 [==============================] - 14s 257ms/step - loss: 0.0281 - accuracy: 0.9910 - val_loss: 0.0605 - val_accuracy: 0.9814 Epoch 10/10 54/54 [==============================] - 14s 257ms/step - loss: 0.0203 - accuracy: 0.9936 - val_loss: 0.0663 - val_accuracy: 0.9791

# 7, Model evaluation

## 1. Accuracy and Loss diagram

acc = history.history['accuracy'] val_acc = history.history['val_accuracy'] loss = history.history['loss'] val_loss = history.history['val_loss'] epochs_range = range(epochs) plt.figure(figsize=(12, 4)) plt.subplot(1, 2, 1) plt.plot(epochs_range, acc, label='Training Accuracy') plt.plot(epochs_range, val_acc, label='Validation Accuracy') plt.legend(loc='lower right') plt.title('Training and Validation Accuracy') plt.subplot(1, 2, 2) plt.plot(epochs_range, loss, label='Training Loss') plt.plot(epochs_range, val_loss, label='Validation Loss') plt.legend(loc='upper right') plt.title('Training and Validation Loss') plt.show()

## 2. Confusion matrix

Seaborn is a drawing library. It carries out higher-level API encapsulation based on the Matplotlib core library, which allows you to easily draw more beautiful graphics. Seaborn's beauty is mainly reflected in more comfortable color matching and more delicate style of graphic elements.

from sklearn.metrics import confusion_matrix import seaborn as sns import pandas as pd # Define a function for drawing confusion matrix graph def plot_cm(labels, predictions): # Generating confusion matrix conf_numpy = confusion_matrix(labels, predictions) # Convert matrix to DataFrame conf_df = pd.DataFrame(conf_numpy, index=class_names ,columns=class_names) plt.figure(figsize=(8,7)) sns.heatmap(conf_df, annot=True, fmt="d", cmap="BuPu") plt.title('Confusion matrix',fontsize=15) plt.ylabel('True value',fontsize=14) plt.xlabel('Estimate',fontsize=14)

val_pre = [] val_label = [] for images, labels in val_ds:#Here, you can take part of the validation data (. take(1)) to generate a confusion matrix for image, label in zip(images, labels): # You need to add a dimension to the picture img_array = tf.expand_dims(image, 0) # Use the model to predict the characters in the picture prediction = model.predict(img_array) val_pre.append(class_names[np.argmax(prediction)]) val_label.append(class_names[label])

plot_cm(val_label, val_pre)

# 8, Save and load model

This is the simplest method of model saving and loading

# Save model model.save('model/17_model.h5')

# Loading model new_model = tf.keras.models.load_model('model/17_model.h5')

# 9, Forecast

# Use the loaded model (new_model) to see the prediction results plt.figure(figsize=(10, 5)) # The width of the figure is 10 and the height is 5 plt.suptitle("Display of prediction results") for images, labels in val_ds.take(1): for i in range(8): ax = plt.subplot(2, 4, i + 1) # display picture plt.imshow(images[i].numpy().astype("uint8")) # You need to add a dimension to the picture img_array = tf.expand_dims(images[i], 0) # Use the model to predict the characters in the picture predictions = new_model.predict(img_array) plt.title(class_names[np.argmax(predictions)]) plt.axis("off")

Previous highlights

- 100 cases of deep learning convolutional neural network (CNN) to realize mnist handwritten numeral recognition | day 1
- 100 cases of deep learning - convolutional neural network (CNN) color picture classification | day 2
- 100 cases of deep learning - convolutional neural network (CNN) garment image classification | day 3
- 100 cases of deep learning - convolutional neural network (CNN) flower recognition | day 4
- 100 cases of deep learning - convolutional neural network (CNN) weather recognition | day 5
- 100 cases of deep learning - convolutional neural network (VGG-16) to identify the pirate king straw hat group | day 6
- 100 cases of deep learning - convolutional neural network (VGG-19) to identify the characters in the spirit cage | day 7
- 100 cases of deep learning - convolutional neural network (ResNet-50) bird recognition | day 8
- 100 cases of deep learning - circular neural network (RNN) to achieve stock prediction | day 9
- 100 cases of deep learning - circular neural network (LSTM) to realize stock prediction | day 10
- 100 cases of deep learning - convolutional neural network (AlexNet) hand-in-hand teaching | day 11
- 100 cases of deep learning - convolutional neural network (CNN) identification verification code | day 12
- 100 cases of deep learning - convolutional neural network (perception V3) recognition of sign language | day 13
- 100 cases of deep learning - convolution neural network (Inception-ResNet-v2) recognition of traffic signs | day 14
- 100 cases of deep learning - convolutional neural network (CNN) for license plate recognition | day 15
- 100 cases of in-depth learning - convolutional neural network (CNN) to identify the Magic Baby Xiaozhi group | day 16

🚀 From column: 100 cases of deep learning

Unfinished ～

Continuous updates welcome likes 👍, Collection ⭐, follow 👀

give the thumbs-up 👍: Praise gives me the motivation to continuously update

Collection ⭐ Note: you can find articles at any time after collection

follow 👀: Follow me to receive the latest articles for the first time