TextCNN-- sentiment analysis
As a text-dimensional image, which can be captured between adjacent word associated with a dimensional convolution neural network .
It works one-dimensional convolution layer
⼆ dimensional convolution with the sample layer ⼀, ⼀ dimensional convolution layer so that a mutual correlation operation uses one dimension. In ⼀ dimensional cross-correlation calculation, convolution starts from the leftmost window-connector ⽅ START input array, the order from left to right, sequentially slides on the START input array. When the convolution window-connector is slid to a position ⼀, START Submenu array input window and the core-connector array element by multiplying the summed to give the number of output elements corresponding to the position of the group.
1 def corr1d(X, K): 2 w = K.shape[0] 3 Y = torch.zeros((X.shape[0] - w + 1)) 4 for i in range(Y.shape[0]): 5 Y[i] = (X[i: i + w] * K).sum() 6 return Y 7 8 #测试 9 X, K = torch.tensor([0, 1, 2, 3, 4, 5, 6]), torch.tensor([1, 2]) 10 corr1d(X, K) #output: tensor([ 2., 5., 8., 11., 14., 17.])
Multi-channel input START ⼀ dimensional mutual correlation operation is also input to the multi-channel ⼆ START similar dimensional cross correlation operation: on each channel, the core and the corresponding output START do ⼀ dimensional cross correlation calculation between the channel and results obtained by adding the output.
. 1 DEF corr1d_multi_in (X, K): 2 # first loops and calculates a one-dimensional cross correlation result along the X dimension, and K 0 (channel dimension), then all the results are stacked along the second dimension accumulation 0 . 3 return torch.stack ([corr1d (X, K) for X, K in ZIP (X-, K)]). SUM (Dim = 0) . 4 . 5 # test . 6 X-torch.tensor = ([[0,1,2,3,4 , 5,6 ], . 7 [1,2,3,4,5,6,7 ], . 8 [2,3,4,5,6,7,8 ]]) . 9 K = torch.tensor ([[ 1,2], [. 3,. 4], [-1, -3 ]]) 10 corr1d_multi_in (X-, K) # Output: Tensor ([2., 8. The, 14. A, 20. A, 26. A, 32. ])
Timing largest pool layer
Using the timing manipulation textCNN pooled from the largest (max-over-time pooling) actually corresponds ⼀ layer is largest dimension of the global pool layer: Suppose START input comprises a plurality of channels, each channel by a value on the composition at different time steps, the output of each channel, i.e. the channel is largest of all time step values. Thus, the timing of the cell layer is largest number of time steps START input on each channel may be different. Since timing is largest pool of main destination time is the most ᯿ to grab a timing feature, it usually can not model the effects of Face to add character.
1 class GlobalMaxPool1d(nn.Module): 2 def __init__(self): 3 super(GlobalMaxPool1d, self).__init__() 4 def forward(self, x): 5 # x shape: (batch_size, channel, seq_len) 6 return F.max_pool1d(x, kernel_size=x.shape[2]) # shape:(batch_size, channel, 1)
TextCNN model
Using the model textCNN main ⼀ dimensional convolution layer and timing layer is largest pool. Text is assumed that the START input sequence of words composed of word vectors for each word Using dimensional representation. Then the width of the sample input START, ADVANCED 1, the channel number is input START. The calculated textCNN divided into the following steps to get accustomed.
1. ⼀ dimensional convolution core defining a plurality of, and use these to check convolutional do convolution calculation are input START. Different widths convolution kernel may capture the relevance of different number of adjacent words.
2. The outputs of all channels are pooled to make the timing is largest, then pooling the outputs of these channels are connected to a vector.
3. After the link layer connection by the full vector transform each category for the relevant output. This ⼀ step can use the discarded layer to deal with over-fitting.
. 1 class TextCNN (nn.Module): 2 DEF the __init__ (Self, Vocab, embed_size, kernel_sizes, NUM_CHANNELS): . 3 Super (TextCNN, Self). The __init__ () . 4 self.embedding = nn.Embedding (len (Vocab), embed_size ) 5 # is not involved in the training of an embedded layer . 6 self.constant_embedding = nn.Embedding (len (Vocab), embed_size) . 7 self.dropout nn.Dropout = (0.5 ) . 8 self.decoder nn.Linear = (SUM (NUM_CHANNELS), 2 ) 9 # timing is not the maximum cell layer weights, it is possible to share one example 10 self.pool =GlobalMaxPool1d () . 11 self.convs nn.ModuleList = () # create a plurality of one-dimensional convolution layer 12 is for C, K in ZIP (NUM_CHANNELS, kernel_sizes): 13 is self.convs.append (nn.Conv1d (in_channels = 2 * embed_size, 14 out_channels = C, 15 kernel_size = K)) 16 . 17 DEF Forward (Self, Inputs): 18 is # will be two shapes (batch size, number of words, word vector dimension) output by the embedded layer is connected to word vector . 19 embeddings = torch.cat (( 20 is self.embedding (Inputs), 21 is self.constant_embedding (Inputs)), Dim = 2) # (the batch_size, seq_len, embed_size * 2) 22 is # The input format Conv1d claims, the word vector dimension, i.e. channel-dimensional transform to one-dimensional convolution of the previous layer, dimension 23 is embeddings embeddings.permute = (0, 2, 1 ) 24 # for each one-dimensional convolution layer, after the timing of the maximum pool will get a shape (batch size, channel size, 1) 25 # of Tensor. use flatten function to remove the last dimension, then the connecting channel dimension 26 is encoding torch.cat = ([self.pool (F.relu (CONV (embedding that))). Squeeze (-1) for CONV in self.convs], =. 1 Dim ) 27 # using discarded after applying method to obtain an output fully connected layer 28 outputs = self.decoder (self.dropout (encoding)) 29 return outputs 30 31 embed_size, kernel_sizes, nums_channels = 100, [3, 4, 5], [100, 100, 100] 32 net = TextCNN(vocab, embed_size, kernel_sizes, nums_channels)
OK, the record about the model, above all from the "hands-on learning deep learning" this book.