Chapter 6
6.1 Using image features to complete image recognition
6.1.1 Load training dataset
cifar_train = data.get('cifar10-small', subset='train')
fig() + plot(cifar_train)
6.1.2 Get image features
hog = HOGExtractor()
hog_train = cifar_train.map(hog, on_field=0)
6.1.3 Observing HOG features
index = 10
hog_feature, label = hog_train[index]
fig() + plot(hog_feature, type='featvec')
index = 10
image, label = cifar_train[index]
fig() + plot(image, type='hog')
6.1.4 Create a classifier and train it with a support vector machine
mlc = multi_class_classifier()
mlc.train(hog_train, alg=SVM())
6.1.5 Testing the model on a training subset
acc1 = mlc.accuracy(hog_train)
print('Cifar_train Accuracy: ', acc1)
6.1.6 Load the test data set and obtain image features
cifar_test = data.get('cifar10-small', subset='test')
fig() + plot(cifar_test)
hog_test = cifar_test.map(hog, on_field=0)
6.1.7 Testing the model on a test subset
acc2 = mlc.accuracy(hog_test)
print('Cifar_test Accuracy: ', acc2)
6.1.8 Making predictions with classifiers
index = 10
image, label = cifar_test[index]
fig()+plot(image)
print('label:', label)
hog_feature = hog(image)
prediction = mlc.predict(hog_feature)
print('prediction: ', prediction)
6.2 Image Classification Using Deep Neural Networks
6.2.1 Load training dataset
cifar_train = data.get('cifar10-small', subset='train')
fig() + plot(cifar_train)
6.2.2 Types of acquired data
label_names = cifar_train.meta['label_names']
print(label_names)
6.2.3 Building a neural network
backbone = ssnet(h=32, w=32, c=3)
net = NNClassifier(backbone, ncls=10)
6.2.4 Training of simulated neural network
net.demo_train(cifar_train, epoch=5)
6.2.5 Load the test data set and use the neural network to make predictions
cifar_test = data.get('cifar10-small', subset='test')
img, label = cifar_test[100]
fig() + plot(img)
label_name = label_names[label]
pred = net.predict(img)
pred_name = label_names[pred]
print("Label is {}, Prediction is {}.".format(label_name, pred_name))
6.3 Extracting facial features from photos
6.3.1 Get Dataset
face_photos = data.get('faces')
photo = face_photos[109]
fig() + plot(photo)
6.3.2 Face detection and facial features alignment
photo_d = detect_faces(photo)
fig() + plot(photo_d)
6.3.3 Face normalization and cropping
faces = crop_aligned_faces(photo_d)
fig(2, 1) + [plot(faces[0]), plot(faces[1])]
6.3.4 Identity Feature Extraction
feature1 = extract_face_feature(faces[0])
feature2 = extract_face_feature(faces[1])
6.4.5 Identity feature comparison
fig(2,1) + [plot(feature1, type='featvec'), plot(feature2, type='featvec')]
6.4 Face Clustering in Photo Album
6.4.1 Get Dataset
ds = data.get('facefeat')
faces = ds.field(0)
features = ds.field(1)
6.4.2 Model creation and training
model = KMeans(K=7)
model.train(features)
6.4.3 Model prediction and effect display
pred = model.predict_all(features)
fig() + plot(faces, pred, type='face_cluster')
6.4.4 Selection of K value (elbow method)
Ks = [4, 5, 6, 7, 8]
losses = []
for k in Ks:
model = KMeans(K=k)
model.train(features, visual=False)
losses.append(model.loss)
fig() + plot(losses, Ks, type='elbow')
6.5 Understanding Optical Flow in Video
6.5.1 Get Video Dataset
ucf = data.get('ucf-small', subset='train')
video, label = ucf[1]
fig() + plot(video)
6.5.2 Get frames in video
frame0 = video[5]
frame1 = video[6]
fig(2,1)+[plot(frame0), plot(frame1)]
6.5.3 Calculate the optical flow of adjacent frames
flow = optical_flow(frame0, frame1)
6.5.4 Visualizing Optical Flow Using Vectors
frame_flow = image_add_flow(frame0, flow)
fig() + plot(frame_flow)
6.5.5 Visualizing Optical Flow Using a Grayscale Image
flow_x, flow_y = split_flow(flow)
fig(2, 1) + [plot(flow_x), plot(flow_y)]
6.5.6 Extracting Optical Flow Video
flowvideo = optical_flow_video(video)
flow0 = flowvideo[0]
video_arrow = video_add_flow(video, flowvideo)
fig() + plot(video_arrow)
6.6 Using the optical flow histogram to complete the task of behavior recognition
6.6.1 Loading the Video Dataset
trainset = data.get('ucf-small', subset='train')
video, label = trainset[10]
fig() + plot(video)
print(trainset.meta['label_names'][label])
6.6.2 Calculating Optical Flow Histogram Features
flowvideo = optical_flow_video(video)
feat = hof(flowvideo)
fig() + plot(feat, type='featvec')
6.6.3 Computing features for all videos
trainflow = trainset.map(optical_flow_video, on_field=0)
trainhof = trainflow.map(hof, on_field=0)
print(trainhof[10][0].shape)
6.6.4 Training a classifier
mlc = multi_class_classifier()
mlc.train(trainhof, alg=SVM())
6.6.5 Testing the classifier on the training set
acctrain = mlc.accuracy(trainhof)
print('Train Accuracy:', acctrain)
6.6.6 Testing the classifier on the test set
testset = data.get('ucf-small', subset='test')
testflow = testset.map(optical_flow_video, on_field=0)
testhof = testflow.map(hof, on_field=0)
acctest = mlc.accuracy(testhof)
print('Test Accuracy:', acctest)
6.7 Generating Star Images with Generative Adversarial Networks
6.7.1 Get Dataset
celeba = data.get('celeba-tiny')
fig() + plot(celeba)
6.7.2 Building a Generative Adversarial Network
gan_model = DCGAN()
6.7.3 Training of GAN network
gan_model.demo_train(celeba, step=200)
6.7.4 Generate and display pictures
image = gan_model.generate_tile(5, 5)
fig() + plot(image)
6.7.5 Loading pre-trained models with different iterations
gan_model.loadparam(1.5)
image = gan_model.generate_tile(5, 5)
fig() + plot(image)