1. tf.nn.embedding_lookup (W, X) W dimension is [len (vocabulary_list), 128], X dimension is [?, 8], the combined dimension of [?, 8, 128]
Code explain: i.e., 128 data extracted from line W corresponding to a number according to each row in the X, such as X [1, 3] is data 3062, i.e., data extracted from line 128 of 3062 W
import numpy as np import tensorflow as tf data = np.array([[2, 1], [3, 4], [5, 6]]) data = tf.convert_to_tensor(data) lk = [[0,1],[1,0],[0,0], [1, 1]] lookup_data = tf.nn.embedding_lookup(data,lk) init = tf.global_variables_initializer() sess = tf.Session() print(sess.run(tf.shape(data))) print(sess.run(tf.shape(lk))) print(sess.run(tf.shape(lookup_data))) print(sess.run(lookup_data))
2. Saver.restore (sess, './model/-10') loading the path sess
Here only you need to enter the last argument pathname No input back
Code Description: The main use of the CNN name gender prediction, which uses tf.embedding_lookup () a dimension of conversion, the original dimensions are 8, after the converted dimension is 8, 128??
Data Description: Use csv.reader () to read csv file, add the name to the train_x, gender as male, label train_y.append ([0, 1]), sex woman, label train_y.append ([1 , 0])
Construction of vocabulary_dict dictionary, word frequency statistics of each word, circulating in the dictionary on the +1 count on good dictionary, sorted using the sort key value is obtained after a good sort
Each word is used to represent a number, use enumerate
Cycle train_x, obtaining a digital word corresponding to each, and added to the list, if the length of the list is less than the maximum length, filled with 0
Code Description: Code description herein: mainly be described in two parts:
Part I: reading data, i.e. train_vec_x, production and train_y
Part II: Construction of the model
Part III: model training operation
Part I: reading data, i.e., the production train_vec_x and train_y
Step: Construction and train_x train_y, the read data are added to the train_x and train_y
The first step: Use with open ( 'name.csv') to open the file
Step 2: Use csv.reader (csvfile) reads the file data
Third Step: circulation data, determines the size of data is equal to 2, if is equal to 2, using tran_x.append (row [0])
Step Four: Analyzing row [1] == 'M': If train_y.append ([0, 1]), whether the person train_y.append ([1, 0]) as a name tag for each
Step two: construction vocabulary_dict dictionary for each word corresponding word frequency statistics
The first step: circulating train_x, each cycle word for name
Step 2: Use token = [word for word in name] is the name of a string combination into a list
Third Step: circulation token, if the word in vocabulary_dict, vocabulary_dict [word] + = 1, i.e., the number of word frequency plus one, if not, vocabulary_dict [word] = 1, the equivalent of doing an initial
The third step: vocabulary_list = [ ''] + sorted (vocabulary_dict, key = vocabulary_dict.get, reverse = True), sorted according to the value of the dictionary, the dictionary returns the key, i.e., each word
Step Four: Use dict ((x, y) for (y, x) in emurate (vocabulary_list)) to each word with 0, 1 .... corresponding
Step five: circulation train_x, generate a list of maps
The first step: empty list structure train_vec_x, recycling trian_x in the name
Step Two: name_vec empty list structure using token = [word for word in name] will be a list of names into word
Third Step: circulation of token word, obtain the mapping dictionary vocab [word], the corresponding index numbers, add it to name_vec
Step Four: If the length is less than the maximum length name_vec will continue name_vec zero padding, i.e., append (0)
Step 5: train_vec_x.append (name_vec) each name corresponding digital map add
Part II: Construction of the network model, the predicted results of the model output, i.e. [None, 2]
The first step: setting the basic parameters of the network, batch_size, input_size, num_classes, num_batch
Step two: function configured network model, input vocabulary_size, embedding_size, and filter_num
The third step: using tf.placeholder (tf.int32, [None, input_size]) X initialization configuration using tf.placeholder (tf.float32, [None, 2]) y initialization configuration
Step Four: with tf.name_scope ( 'embedding') for embedding change
Step: Construction of W = tf.Vairbale (tf.random_normal ([vocabulary_size, embedding_size]))
Step 2: Use tf.embedding_lookup (W, X) carried forward the embedding mapping, then the output of 8, 128?
The third step: using tf.expand_dim (embedding_chars, axis = -1) to convert to a dimension, 8, 128, 1?
Step 5: three kinds of convolution convolution kernel size
Step: defines three dimension size of the convolution kernel, is divided into 3, 4, 5, configured output_pool = [] List
Step Two: cycling three sizes convolution kernel
The third step: configuration with tf.name_scope ( 'conv_pool% d'% (conv_size,))
The fourth step: configuration parameters W, a size of [conv_size, embedding_size, 1, num_filter], configuration parameters b, a size of [num_filter]
Step 5: Use tf.nn.conv2d perform convolution operation, using tf.nn.relu activate operation
Step 6: tf.nn.max_pool pooling operation, ksize = [1, input_size - conv_filter + 1, 1, 1] is the size of the convolved
Seventh step: adding the result of the convolution to output_pool list
Eighth step: The output_pool using a dimension merge operations tf.concat
Step Six: the results of the combined changes in dimension, i.e. the dimension becomes [- 1, 3 * 128] For the subsequent operation of a fully connected
Step 7: Use with tf.name_scope ( 'dropout') for dropout operation, using tf.nn.dropout
Step eight: Use with tf.name_scope ( 'output') connecting operation of the whole structure
Step: Construction of dimension W [3 * 128, num_class], the configuration of dimension b [num_class]
Step 2: Use tf.nn.xw_plus_b for tf.matmul (x, w) + b operations
The third step: Return output results
Part III: Definitions train_network () training model
Step: Use neture_network (vocabulary_size = len (vocabulary_size)) to obtain an output output
Step 2: Use tf.reduce_mean (tf.nn.softmax ... logits ()) the definition of loss value loss
The third step: using tf.train.Adaoptimer (1e-3) .minimize (loss) value of loss reduction operation train_op
Step four: Use tf.train.Saver (tf.global_variable ()) function definition is stored Saver
Step 5: Use with tf.Session () constructor to initialize sess execute the function, use sess.run () were variable
Step Six: circulation epoch performs num_batch cycle
Step 7: Use tran_vec_x [i * batch_size: (i + 1) * batch] obtained x_batch, using train_y [i * batch_size: (i + 1) * batch] obtained y_batch
Step eight: Use sess.run ([train_op, loss], feed_dict = {X: x_batch, y: y_batch})
Step 9: If the iteration 1000, print epoch, iteration and loss
Step 10: iterative two epoch, that is, save sess with Saver.save (),
Code: main.py
TF tensorflow AS Import
Import numpy AS NP
Import CSV
# Part I: read data
# Step 1: Create a list train_x, train_y, read from the file name stored in train_x, gender read using [0, 1 ] or [1, 0] stored in the train_y
train_x = [] # creating list
train_y = []
# open file name.csv
with open ( 'name.csv', 'R & lt', encoding = 'UTF-. 8') AS csvFile:
# csv.reader used for reading, data read is two, as a first name, a second sex as
data_csv = csv.reader (csvFile)
# loop to read data
for Row in data_csv:
# If the size of the current row is 2
IF len (row) == 2:
# current row of data and a first name added to the train_x
train_x.append (row [0])
# if the current row is the second data 'M', the tag train_y add [0,. 1]
IF Row [. 1] == 'M':
train_y.append ([0,. 1])
# is not, then put value = 1, i.e. vocabulary_dict [word] = 1
the else:
# No tag or tags train_y added [. 1, 0]
train_y.append ([. 1, 0])
# size printing longest name
max_len_name = max ([len (name) for name in train_x])
Print (max_len_name)
# the maximum length is set name. 8
max_len_name. 8 =
# Step: word frequency dictionary structure to build a vocabulary dictionary
vocabulary_dict} = {
# name circulation
for in train_x name:
# the name of each of the word apart, becomes into a list of
token = [word for word in name]
# cycle word list
for word in token:
# If this word in the dictionary vocabulary in, value + = 1
IF word in vocabulary_dict:
vocabulary_dict [word] + = 1
the else:
vocabulary_dict [Word] =. 1
# step: list after the word frequency dictionary sorting operation according to the value, obtained ordering
= vocabulary_list [ ''] + the sorted (vocabulary_dict, Key = vocabulary_dict.get, Reverse = True)
# Step IV: Use dict (x, y) in the configuration of a digital map bag of words
vocab = dict ((x, y ) for ( Y, X) in the enumerate (vocabulary_list))
# fifth step: circulation train_x, word names corresponding to the combination number, the name is converted into a digital train_x list, if the length of the list of numbers is less than 8, i.e. .append (0)
= train_vec_x []
for name in train_x:
# configured name VEC
name_vec = []
# the name list split into a word
token = [word for word in name]
# cycle the word list
for word in token:
# the Dictionary the key, be added to the value obtained name_vec
name_vec.append (Vocab [Word])
name_vec.append (0)
# name vector added to the composition good in train_vec_x
# name_vec If the length is less than 8, using .append (0), a length of 8 filled
len the while (name_vec) <max_len_name:
train_vec_x.append (name_vec)
##################
# Part II: Construction of the network model, the output is [None, num_class] i.e. score value for each class
# Step: Construction of parameters input_size, batch_size, num_classes, num_batch
dimension input_size = max_len_name # entered, the name is the longest distance, ie. 8
the batch_size each batch size = 64 #
num_classes = category # 2 results
num_batch = len (train_vec_x) // batch_size # samples once after several BATCH
# Step 2: use tf.placeholder configuration X and Y
X = tf.placeholder (tf.int32, [None, input_size])
Y tf.placeholder = (tf.float32, [None, num_classes])
# size keep_prob input configured for performing Dropout
dropout_keep_prob = tf.placeholder (tf.float32)
# step: input function configured network
def neture_network (vocabulary_size , embedding_size = 128, filter_num = 128 ):
# fourth step: embedding the pre mapping
with tf .name_scope ( 'embedding'):
# Configuration W, a size of [vocabulary_size, embedding_size]
W is = tf.Variable (tf.random_uniform ([vocabulary_size, embedding_size], -1.0, 1.0))
# tf.nn.embedding_lookup used for projection map, the map size is [ ?,. 8, 128]
embedding_chares tf.nn.embedding_lookup = (W is, X-)
# use tf.expand_dims increased dimensions, the dimensions it becomes [?,. 8, 128,. 1]
embedding_chares = tf.expand_dims (embedding_chares, Axis = -1)
# step 5: three different sizes of convolution convolution convolution, the results were combined
# three kinds of the convolution kernel size are. 3,. 4,. 5
conv_sizes = [. 3,. 4,. 5 ]
# configured pooled results list
pool1_output = []
# cyclic convolution kernel of different sizes
for conv_size in conv_sizes:
scope with.name_scope configuration parameter #
tf.name_scope with ( 'D CONV-padding%'% (conv_size,)):
# size dimension of the pooling [?, 1, 1, 128]
# Convolution configuration parameters W, dimension [conv_size, embedding_size,. 1, filter_num]
W is = tf.Variable (tf.random_uniform ([conv_size, embedding_size,. 1, filter_num]))
B # convolution parameter configuration, dimension [filter_num]
B = tf.Variable (tf.constant (0.1, Shape = [filter_num]))
# convolving operation and the activation operation
conv = tf.nn.relu (tf.nn.conv2d (embedding_chares, W, strides = [. 1,. 1,. 1,. 1], padding = 'the VALID') + B)
# maximum value pool operation, ksize = [1, input_size = conv_size + 1, 1, 1] to the middle of the dimension after convolution two intermediate layers characterized in dimensions
pool = tf.nn.max_pool (conv, ksize = [1, input_size - conv_size + 1, 1, 1], strides = [1, 1, 1, 1], padding = 'VALID' )
pool1_output.append (the pool)
# combined dimensions, overlapping operation, i.e., the dimension [?, 1, 13 * 128]
= tf.concat FCIN (pool1_output, Axis =. 3)
# Step Six: changes dimension, for subsequent operation of a fully connected
FCIN = tf.reshape (FCIN, Shape = [-. 1,. 3 * 128])
# of seven steps: dropout operation performed
with tf.name_scope ( 'dropout'):
fc_dropout = tf.nn.dropout (FCIN, keep_prob = dropout_keep_prob)
# eighth step: the whole final join operation, the category score for calculation
with tf.name_scope ( 'Output'):
W is, the dimension of a fully connected configuration # [* 128. 3, 2]
W is = tf.Variable (tf.random_uniform ([* 128. 3, num_classes], -1.0, 1.0))
# b, connected to construct the full dimension [2]
B = tf.Variable (tf.constant (0.1, Shape = [num_classes]))
# obtain the final output score
output = tf.nn.xw_plus_b (fc_dropout, W,
# part III: training operation model
def train_network ():
# return a score value
Output return
# Step: call neture_network obtain an output result Output
Output = neture_network (vocabulary_size = len (vocabulary_list))
# Step 2: Use tf.reduce_mean (tf.nn.softmax _... logits) to obtain the value of loss Loss
Loss tf.reduce_mean = (tf.nn.softmax_cross_entropy_with_logits (logits = Output, Labels = Y))
# step: loss value is reduced using an adaptive loss operation train_op
train_op = tf.train.AdamOptimizer (-1E. 3) .minimize ( Loss)
# fourth step: a storage configured using tf.train.Saver sess
Saver = tf.train.Saver (tf.global_variables ())
# step 5: tf.Session () function is performed to obtain sess, and parameter initialization operation
with tf.Session () AS Sess:
# parameter initialization operation
sess.run (tf.global_variables_initializer ())
# sixth step: circulation epoch, cycle num_batch
for E in Range (. 11):
for I in Range (num_batch):
# Seventh Step: The index value and the configuration batch_x batch_y
batch_x train_vec_x = [I * the batch_size: (I +. 1) * the batch_size]
batch_y train_y = [I * the batch_size: (I +. 1) * the batch_size]
# eighth step: train_op performed using sess.run and Loss
_, _loss sess.run = ([train_op, Loss], = {X-feed_dict: batch_x, Y: batch_y, dropout_keep_prob: 0.7})
# step 9: If the iteration print result 1000
IF I == 0% 1000:
Print ( 'Epoch', E, 'ITER', I, 'Loss:', _loss)
# save the results of two if it iterates
if e% 2 == 0 :
saver.save (Sess, './model/epoch', global_step = E)
train_network ()
Next, the trained parametric test constructor test_sex, since the result is the same network, there is no need to modify the above-described two portions, the third portion only needs to be changed, test_sex (name_list)
The first step: the model prediction structure test_sex, enter the name for the list
Step two: the digital map, configured train_x
The first step: the empty list structure train_x
Step two: circulation name_list, get name
The third step: using token = [word for word in name] to split a word into the name list, configured name_vec
Fourth Step: circulation token, the name_vec.append (vocan [word]), will be converted into a digital word WORD
Step five: If the length is less than the maximum length name_vec 8, i.e. using .append (0) zero padding operation
Step Six: Add name_vec to train_x list constructed train_x
Step Three: Call neture_network () to obtain output
Step Four: Use tf.argmax (output, axis = 1) obtained score value is larger the index value, i.e. the value of the category
Step 5: Use tf.train.Saver () function to get saved Saver
Step 6: Using with tf.Session () as sess sess constructor function
Step 7: Use Saver.restore (sess, './model/-10') loading sess
Step eight: Use sess.run (y_pred, feed_dict = {X: train_x, keep_prob = 1.0}) to obtain the prediction results
Step 9: cycle predictions, and if the result is 1, the print category for men, the result if the index value is 0, the print category for women
Step #: test_sex configured for model predictions, input NAME_LIST
DEF test_sex (NAME_LIST):
# Step: configured for train_x
# configured to store a X
X = []
# circulation list name, a name is obtained when
in NAME_LIST name for:
# name_vec configuration list, a vector for storing the name
name_vec = []
# name split into a list of words
token = [word for word in name]
# cycles per word
for word in token:
# the words bag digital word corresponding to a vector of numbers added to the name of
name_vec.append (Vocab [word])
# if the name is a vector of numbers less than 8, adding 0
the while len (name_vec) <max_len_name:
name_vec.append (0)
# Add the name of digital word vector to vector
x.append (name_vec)
# step Three: call models get output
= neture_network Output (vocabulary_size = len (vocabulary_list))
# Step IV: Use tf.argmax maximum index value, or 0. 1
y_pred tf.argmax = (Output, Axis =. 1)
# Step 5: tf.train .Saver () constructor function Saver
Saver = tf.train.Saver (tf.global_variables ())
# step 6: tf.Session () constructor function is executed
with tf.Session () AS sess:
# step Seven: load sess function
saver.restore (sess, './model/epoch')
# eighth step: sess.run performed using y_pred, obtain the actual predictions
y_pred_ = sess.run (y_pred, feed_dict = {X: x, dropout_keep_prob: 1.0})
# step 9: predictor loop, if the index value is 1, the print of the male, NO is the woman who printed
for I in Range (len (y_pred_)):
IF y_pred_ [I] == 1:
print (name_list [i], 'M')
the else:
print(name_list[i], '女')
