Chinese emotion recognition 4

candle NLP CNN LSTM

Mix LSTM and CNN

Convolutional neural network is effective for sparse data structure. Critics IMDB data in a single evaluation does
have a one-dimensional sequence space sparse structure of the word, CNN invariant feature can pick out the bad feelings. Characterized by the CNN spatial learning, learning may be LSTM layer sequence. After the word embedded layer, by adding a maximum pool dimensional CNN layer and the combined features provided to LSTM. 32 using a filter having characteristics convolution layer, and the step size is set to 3, cell layers using a standard step size of the step 2 in FIG feature size is halved.

The key issue

Parameters of the problem of the individual layers
slowly bookset it
https://keras.io/

Code

'''
序列分类：IMDB 影评分类 LSTM
混合使用 LSTM 和 CNN
'''
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
from keras.datasets import imdb
import numpy as np
from keras.preprocessing import sequence
from keras.models import Sequential
from keras.layers.embeddings import Embedding
from keras.layers import LSTM
from keras.layers import Dense
from keras.layers.convolutional import Conv1D, MaxPooling1D

seed = 7
top_words = 5000
max_words = 500
out_dimension = 32
batch_size = 128
epochs = 2
dropout_rate = 0.2

def build_model():
    model = Sequential()
    model.add(Embedding(top_words, out_dimension, input_length=max_words))
    model.add(Conv1D(filters=32, kernel_size=3, padding='same',activation='relu'))
    model.add(MaxPooling1D(pool_size=2))
    model.add(LSTM(units=100))
    model.add(Dense(units=1, activation='sigmoid'))
    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
    # 输出模型的概要信息
    model.summary() 
    return model

np.random.seed(seed=seed)
# 导入数据
(x_train, y_train), (x_validation, y_validation) = imdb.load_data(num_words=top_words)
x_train = sequence.pad_sequences(x_train, maxlen=max_words)
x_validation = sequence.pad_sequences(x_validation, maxlen=max_words)

# 生成模型并训练模型
model = build_model()
model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose=2)
scores = model.evaluate(x_validation, y_validation, verbose=2)
print('Accuracy: %.2f%%' % (scores[1] * 100))

result

Model: "sequential_7"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_4 (Embedding)      (None, 500, 32)           160000    
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 500, 32)           3104      
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 250, 32)           0         
_________________________________________________________________
lstm_7 (LSTM)                (None, 100)               53200     
_________________________________________________________________
dense_4 (Dense)              (None, 1)                 101       
=================================================================
Total params: 216,405
Trainable params: 216,405
Non-trainable params: 0
_________________________________________________________________
M:\Anaconda3\lib\site-packages\tensorflow_core\python\framework\indexed_slices.py:433: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
Epoch 1/2
 - 285s - loss: 0.5007 - accuracy: 0.7304
Epoch 2/2
 - 285s - loss: 0.2494 - accuracy: 0.9010
Accuracy: 87.25%

LSTM relative to the pure model, CNN + higher accuracy of the model LSTM

layer	Accuracy
LSTM	85.58%
CNN+LSTM	87.25%