卷积神经网络训练情感分析
全连接神经网络几乎对网络模型没有任何限制，但缺点是过度拟合，即拟合了过多噪声。全连接神经网络模型的特点是灵活、参数多。在实际应用中，我们可能会对模型加上一些限制，使其适合数据的特点。并且由于模型的限制，其参数会大幅减少。这降低了模型的复杂度，模型的普适性进而会提高。
接下来我们介绍卷积神经网络（CNN）在自然语言的典型应用
在自然语言领域，卷积的作用在于利用文字的局部特征。一个词的前后几个词必然和这个词本身相关，这组成该词所代表的词群。词群进而会对段落文字的意思进行影响，决定这个段落到底是正向的还是负向的。对比传统方法，利用词包（Bag of Words），和TF-IDF等，其思想有相通之处。但最大的不同点在于，传统方法是人为构造用于分类的特征，而深度学习中的卷积让神经网络去构造特征。
接下来介绍如何利用Keras搭建卷积神经网络来处理情感分析的分类问题。下面的代码构造了卷积神经网络的结构。
1、导入模块和数据
部分导入内容请看上一篇学习笔记（三）
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Conv1D, MaxPooling1D2、构建模型
model = Sequential()
model.add(Embedding(vocab_size, 64, input_length = maxword))
model.add(Conv1D(filters=64, kernel_size=3, padding="same", activation="relu"))
model.add(MaxPooling1D(pool_size=2))
model.add(Dropout(0.25))
model.add(Conv1D(filters=128, kernel_size=3, padding="same", activation="relu"))
model.add(MaxPooling1D(pool_size=2))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(64, activation="relu"))
model.add(Dense(32, activation="relu"))
model.add(Dense(1, activation="sigmoid"))
model.compile(loss="binary_crossentropy", optimizer="rmsprop", metrics=["accuracy"])
print(model.summary())打印结果
Layer (type) Output Shape Param #
=================================================================
embedding_2 (Embedding) (None, 400, 64) 5669568
_________________________________________________________________
conv1d_1 (Conv1D) (None, 400, 64) 12352
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 200, 64) 0
_________________________________________________________________
dropout_1 (Dropout) (None, 200, 64) 0
_________________________________________________________________
conv1d_2 (Conv1D) (None, 200, 128) 24704
_________________________________________________________________
max_pooling1d_2 (MaxPooling1 (None, 100, 128) 0
_________________________________________________________________
dropout_2 (Dropout) (None, 100, 128) 0
_________________________________________________________________
flatten_2 (Flatten) (None, 12800) 0
_________________________________________________________________
dense_6 (Dense) (None, 64) 819264
_________________________________________________________________
dense_7 (Dense) (None, 32) 2080
_________________________________________________________________
dense_8 (Dense) (None, 1) 33
=================================================================
Total params: 6,528,001
Trainable params: 6,528,001
Non-trainable params: 0
_________________________________________________________________
None参数的数量比起之前的57,982,369明显减少
3、拟合神经网络
model.fit(X_train, Y_train, validation_data = (X_test, Y_test), epochs=20, batch_size=100)
scores = model.evaluate(X_test, y_test, verbose = 1)
print(scores)运行速度比起之前也是大幅度上升，输出结果如下：
Train on 10000 samples, validate on 1000 samples
Epoch 1/20
10000/10000 [==============================] - 35s 3ms/step - loss: 0.6546 - acc: 0.5753 - val_loss: 0.4126 - val_acc: 0.8350
Epoch 2/20
10000/10000 [==============================] - 34s 3ms/step - loss: 0.3148 - acc: 0.8664 - val_loss: 0.3148 - val_acc: 0.8610
Epoch 3/20
10000/10000 [==============================] - 33s 3ms/step - loss: 0.1795 - acc: 0.9329 - val_loss: 0.3273 - val_acc: 0.8640
Epoch 4/20
10000/10000 [==============================] - 32s 3ms/step - loss: 0.1086 - acc: 0.9624 - val_loss: 0.3339 - val_acc: 0.8680
Epoch 5/20
10000/10000 [==============================] - 32s 3ms/step - loss: 0.0571 - acc: 0.9825 - val_loss: 0.5621 - val_acc: 0.8320
Epoch 6/20
10000/10000 [==============================] - 32s 3ms/step - loss: 0.0433 - acc: 0.9861 - val_loss: 0.7488 - val_acc: 0.8320
Epoch 7/20
10000/10000 [==============================] - 32s 3ms/step - loss: 0.0308 - acc: 0.9919 - val_loss: 0.5513 - val_acc: 0.8710
Epoch 8/20
10000/10000 [==============================] - 35s 3ms/step - loss: 0.0200 - acc: 0.9955 - val_loss: 0.7324 - val_acc: 0.8660
Epoch 9/20
10000/10000 [==============================] - 33s 3ms/step - loss: 0.0137 - acc: 0.9970 - val_loss: 0.8412 - val_acc: 0.8540
Epoch 10/20
10000/10000 [==============================] - 34s 3ms/step - loss: 0.0071 - acc: 0.9984 - val_loss: 0.8964 - val_acc: 0.8550
Epoch 11/20
10000/10000 [==============================] - 36s 4ms/step - loss: 0.0101 - acc: 0.9978 - val_loss: 0.9371 - val_acc: 0.8590
Epoch 12/20
10000/10000 [==============================] - 34s 3ms/step - loss: 1.2869e-05 - acc: 1.0000 - val_loss: 1.0918 - val_acc: 0.8570
Epoch 13/20
10000/10000 [==============================] - 36s 4ms/step - loss: 0.0111 - acc: 0.9978 - val_loss: 1.2172 - val_acc: 0.8570
Epoch 14/20
10000/10000 [==============================] - 32s 3ms/step - loss: 6.3851e-07 - acc: 1.0000 - val_loss: 1.2375 - val_acc: 0.8560
Epoch 15/20
10000/10000 [==============================] - 32s 3ms/step - loss: 0.0017 - acc: 0.9995 - val_loss: 1.2625 - val_acc: 0.8600
Epoch 16/20
10000/10000 [==============================] - 32s 3ms/step - loss: 3.3651e-07 - acc: 1.0000 - val_loss: 1.2808 - val_acc: 0.8570
Epoch 17/20
10000/10000 [==============================] - 34s 3ms/step - loss: 0.0113 - acc: 0.9977 - val_loss: 1.5189 - val_acc: 0.8450
Epoch 18/20
10000/10000 [==============================] - 34s 3ms/step - loss: 0.0107 - acc: 0.9981 - val_loss: 1.4063 - val_acc: 0.8500
Epoch 19/20
10000/10000 [==============================] - 34s 3ms/step - loss: 4.7961e-07 - acc: 1.0000 - val_loss: 1.4254 - val_acc: 0.8520
Epoch 20/20
10000/10000 [==============================] - 34s 3ms/step - loss: 1.3925e-07 - acc: 1.0000 - val_loss: 1.4367 - val_acc: 0.8480运行的速度比之前快了将近五倍，准确率上升了2%！