Table of contents
foreword
Based on the recurrent neural network, this project builds a multi-layer RNN model. By training the data set and adjusting the model and data set, the function of generating Tibetan acrostics and lyrics is realized.
In this project, we used a Recurrent Neural Network (RNN) as the basis of the model. RNN is a neural network capable of processing sequence data, and it excels at processing sequence data such as text and speech.
We build a multi-layer RNN model, which means we stack multiple RNN layers to enhance the expressive power of the model. The multi-layer RNN model can better capture the complex relationship and contextual information in the data.
By training the dataset, we let the model learn different types of acrostics and lyrics. After tweaking the model and dataset, we further improved the quality and diversity of the generated acrostics and lyrics.
Ultimately, our model can accept user-input acrostics and generate acrostics with corresponding acrostics, or complete lyrics based on user-provided beginnings of lyrics. Through this project, we can create creative and emotional text works, bringing more artistic experience and fun to users.
overall design
This part includes the overall structure diagram of the system and the system flow chart.
System overall structure diagram
The overall structure of the system is shown in the following two figures.
System flow chart
The system flow is shown in the following two figures.
operating environment
This section includes Python environment, Tensorflow environment and Pycharm environment.
Python environment
Based on Python 3.7.3, it is developed in the PyCharm environment.
Tensorflow environment
Open Anaconda Prompt and enter the Tsinghua warehouse image:
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
conda config -set show_channel_urls yes
Create a Python 3.7 environment named TensorFlow. At this time, there is a matching problem between the Python version and the later version of TensorFlow. In this step, choose Python3.x.
conda create -n tensorflow python==3.7
If there is a need to confirm, press the Y key.
Activate the TensorFlow environment in Anaconda Prompt:
conda activate tensorflow
Install the CPU version of TensorFlow:
pip install -upgrade --ignore -installed tensorflow
Installed.
PyCharm environment
The IDE is PyCharm, any version is fine.
module implementation
This project consists of two parts: ancient poem generation and lyrics generation. The ancient poem generation consists of data preprocessing, model building, model training and storage, using the model to generate ancient poems, generating Tibetan acrostic poems, and displaying modules with word clouds. The functions and related codes of each module are introduced below.
Generation of ancient poems
1. Data preprocessing
This part includes loading required libraries and data, data processing and organizing training data. The download address is https://github.com/chinese-poetry/chinese-poetry .
1) Load library and data
The relevant code is as follows:
import tensorflow as tf
import numpy as np
import glob
import json
from collections import Counter
from tqdm import tqdm
from snownlp import SnowNLP #主要做繁体字转简体字
poets = []
paths = glob.glob('chinese-poetry/json/poet.*.json') #加载数据
The specific style of training ancient poetry data is shown in the figure below.
2) Data processing
The relevant code is as follows:
for path in paths: #对每一个json文件
data = open(path, 'r').read() #将它读取
data = json.loads(data) #将字符串加载成字典
for item in data: #对每一项
content = ''.join(item['paragraphs']) #将正文取出拼接到一起
if len(content) >= 24 and len(content) <= 32: #取长度合适的诗
content = SnowNLP(content)
poets.append('[' + content.han + ']') #如果是繁体转为简体
poets.sort(key=lambda x: len(x)) #按照诗的长度排序
3) Organize data
The relevant code is as follows:
batch_size = 64
X_data = []
Y_data = []
for b in range(len(poets) // batch_size): #分批次
start = b * batch_size #开始位置
end = b * batch_size + batch_size #结束位置
batch = [[char2id[c] for c in poets[i]] for i in range(start, end)]
#两层循环每首诗的每个字转换成序列再迭代
maxlen = max(map(len, batch))#当前最长的诗为多少字
X_batch = np.full((batch_size, maxlen - 1), 0, np.int32) #用零进行填充
Y_batch = np.full((batch_size, maxlen - 1), 0, np.int32) #用零进行填充
for i in range(batch_size):
X_batch[i, :len(batch[i]) - 1] = batch[i][:-1] #每首诗最后一个字不要
Y_batch[i, :len(batch[i]) - 1] = batch[i][1:] #每首诗第一个字不要
X_data.append(X_batch)
Y_data.append(Y_batch)
#整理字符与ID之间的映射
chars = []
for item in poets:
chars += [c for c in item]
chars = sorted(Counter(chars).items(), key=lambda x:x[1], reverse=True)
print('共%d个不同的字' % len(chars))
print(chars[:10])
#空位为了特殊字符
chars = [c[0] for c in chars]#对于每一个字
char2id = {
c: i + 1 for i, c in enumerate(chars)} #构造字符与ID的映射
id2char = {
i + 1: c for i, c in enumerate(chars)} #构造ID与字符的映射
2. Model construction
After the data is loaded into the model, it is necessary to define the model structure, use multi-layer RNN, define the loss function and optimizer.
1) Define the model structure and optimizer
The relevant code is as follows:
hidden_size = 256 #隐藏层大小
num_layer = 2
embedding_size = 256
#进行占位
X = tf.placeholder(tf.int32, [batch_size, None])
Y = tf.placeholder(tf.int32, [batch_size, None])
learning_rate = tf.Variable(0.0, trainable=False) #定义学习率，不可训练
2) Using multi-layer RNN
The relevant code is as follows:
cell = tf.nn.rnn_cell.MultiRNNCell(
[tf.nn.rnn_cell.BasicLSTMCell(hidden_size, state_is_tuple=True) for i in range(num_layer)],
state_is_tuple=True)
initial_state = cell.zero_state(batch_size, tf.float32) #全零初始状态
embeddings = tf.Variable(tf.random_uniform([len(char2id) + 1, embedding_size], -1.0, 1.0))
embedded = tf.nn.embedding_lookup(embeddings, X) #得到嵌入后的结果
outputs, last_states = tf.nn.dynamic_rnn(cell, embedded, initial_state=initial_state)
outputs = tf.reshape(outputs, [-1, hidden_size]) #改变形状
logits = tf.layers.dense(outputs, units=len(char2id) + 1)
logits = tf.reshape(logits, [batch_size, -1, len(char2id) + 1])
probs = tf.nn.softmax(logits) #得到概率
3) Define loss function and optimizer
The relevant code is as follows:
loss = tf.reduce_mean(tf.contrib.seq2seq.sequence_loss(logits, Y, tf.ones_like(Y, dtype=tf.float32))) #求出损失
params = tf.trainable_variables()
grads, _ = tf.clip_by_global_norm(tf.gradients(loss, params), 5)
#进行梯度截断操作
optimizer = tf.train.AdamOptimizer(learning_rate).apply_gradients(zip(grads, params))
#得到优化器
3. Model training and storage
After defining the model architecture and compiling, train the model through the training set to make the model generate ancient poems. Here the model is fitted and saved using the training and test sets.
1) Model training
The relevant code is as follows:
sess = tf.Session()
sess.run(tf.global_variables_initializer())
for epoch in range(50):
sess.run(tf.assign(learning_rate, 0.002 * (0.97 ** epoch))) #指数衰减
data_index = np.arange(len(X_data))
np.random.shuffle(data_index) #每一轮迭代数据打乱
X_data = [X_data[i] for i in data_index]
Y_data = [Y_data[i] for i in data_index]
losses = []
for i in tqdm(range(len(X_data))):
ls_, _ = sess.run([loss, optimizer],feed_dict={
X: X_data[i],Y: Y_data[i]})
losses.append(ls_)
2) Save the model
The relevant code is as follows:
saver = tf.train.Saver()
saver.save(sess, './poet_generation_tensorflow')
import pickle
with open('dictionary.pkl', 'wb') as fw:
pickle.dump([char2id, id2char], fw) #保存成一个pickle文件
After the model is saved, it can be reused or transplanted to other environments.
4. Use the model to generate ancient poems
This part includes loading resources, redefining the network, using multi-layer RNN, defining loss functions and optimizers, and generating ancient poems.
1) Load resources
The relevant code is as follows:
import tensorflow as tf
import numpy as np
import pickle
#加载模型
with open('dictionary.pkl', 'rb') as fr:
[char2id, id2char] = pickle.load(fr)
2) Redefine the network
The relevant code is as follows:
batch_size = 1
hidden_size = 256 #隐藏层大小
num_layer = 2
embedding_size = 256
#占位操作
X = tf.placeholder(tf.int32, [batch_size, None])
Y = tf.placeholder(tf.int32, [batch_size, None])
learning_rate = tf.Variable(0.0, trainable=False)
3) Using multi-layer RNN
The relevant code is as follows:
cell = tf.nn.rnn_cell.MultiRNNCell(
[tf.nn.rnn_cell.BasicLSTMCell(hidden_size, state_is_tuple=True) for i in range(num_layer)],
state_is_tuple=True)
initial_state = cell.zero_state(batch_size, tf.float32) #初始化
embeddings = tf.Variable(tf.random_uniform([len(char2id) + 1, embedding_size], -1.0, 1.0))
embedded = tf.nn.embedding_lookup(embeddings, X)
outputs, last_states = tf.nn.dynamic_rnn(cell, embedded, initial_state=initial_state)
outputs = tf.reshape(outputs, [-1, hidden_size])
logits = tf.layers.dense(outputs, units=len(char2id) + 1)
probs = tf.nn.softmax(logits) #得到概率
targets = tf.reshape(Y, [-1])
4) Define loss function and optimizer
The relevant code is as follows:
loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits, labels=targets))
params = tf.trainable_variables()
grads, _ = tf.clip_by_global_norm(tf.gradients(loss, params), 5)
optimizer = tf.train.AdamOptimizer(learning_rate).apply_gradients(zip(grads, params))
sess = tf.Session()
sess.run(tf.global_variables_initializer())#得到初始状态
saver = tf.train.Saver()
saver.restore(sess, tf.train.latest_checkpoint('./'))
5) Generate ancient poems
The relevant code is as follows:
def generate():
states_ = sess.run(initial_state)
gen = ''
c = '['
while c != ']'
gen += c
x = np.zeros((batch_size, 1))
x[:, 0] = char2id[c]
probs_, states_ = sess.run([probs, last_states], feed_dict={
X: x, initial_state: states_}) #得到状态与概率
probs_ = np.squeeze(probs_) #去掉维度
pos = int(np.searchsorted(np.cumsum(probs_), np.random.rand() * np.sum(probs_))) #根据概率分布产生一个整数
c = id2char[pos]
return gen[1:]
5. Generate acrostics
The relevant code is as follows:
def generate_with_head(head):
states_ = sess.run(initial_state)
gen = ''
c = '['
i = 0
while c != ']':
gen += c
x = np.zeros((batch_size, 1))
x[:, 0] = char2id[c]
probs_, states_ = sess.run([probs, last_states], feed_dict={
X: x, initial_state: states_})
probs_ = np.squeeze(probs_)
pos = int(np.searchsorted(np.cumsum(probs_), np.random.rand() * np.sum(probs_)))
if (c=='[' or c == '。' or c == '，') and i<len(head):
#判断为第一个字的条件
c = head[i]
i += 1
else:
c = id2char[pos]
return gen[1:]
#将结果写入文件中
f=open('guhshiwordcloud.txt','w',encoding='utf-8')
f.write(generate())
f.write(generate_with_head('天地玄黄'))
f.write(generate_with_head('宇宙洪荒'))
f.write(generate_with_head('寒来暑往'))
f.close()
6. Display the generated ancient poems with word cloud
This part includes loading the required library, opening the generated ancient poem file, extracting keywords and weights, generating objects, generating required colors from pictures, displaying word clouds and saving pictures.
1) Load the required libraries
Related operations are as follows:
from wordcloud import WordCloud, ImageColorGenerator
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
import jieba.analyse
2) Open the generated ancient poem file
Related operations are as follows:
text = open('guhshiwordcloud.txt','r', encoding='UTF-8').read()
3) Extract keywords and weights
Related operations are as follows:
freq = jieba.analyse.extract_tags(text, topK=200, withWeight=True)
#权重控制关键词在词云里面的大小
print(freq[:20])#打印前20个
freq = {
i[0]: i[1] for i in freq} #转成字典
4) Generate objects
Related operations are as follows:
mask = np.array(Image.open("color_mask.png")) #以图片为参考
wc = WordCloud(mask=mask, font_path='Hiragino.ttf', mode='RGBA', background_color=None).generate_from_frequencies(freq)
5) Generate the required color from the picture
Related operations are as follows:
image_colors = ImageColorGenerator(mask)
wc.recolor(color_func=image_colors)
6) Display word cloud
Related operations are as follows:
plt.imshow(wc, interpolation='bilinear') #设定插值形式
plt.axis("off") #无坐标轴
plt.show()
7) Save the picture
Related operations are as follows:
wc.to_file('gushiwordcloud.png')
Lyric Generation
This part includes data preprocessing, model building, model training and saving, and lyrics production. The functions and related codes of each module are introduced below.
1. Data preprocessing
Related operations are as follows:
#首先加载相应的库
from keras.models import Sequential
from keras.layers import Dense, LSTM, Embedding
from keras.callbacks import LambdaCallback
import numpy as np
import random
import sys
import pickle
sentences = []
#读取训练所需数据并进行预处理
with open('../lyrics.txt', 'r', encoding='utf8') as fr: #读取歌词文件
lines = fr.readlines() #一行一行读取
for line in lines:
line = line.strip() #去掉空格字符
count = 0
for c in line:
if (c >= 'a' and c <= 'z') or (c >= 'A' and c <= 'Z'):
count += 1 #统计英文字符个数
if count / len(line) < 0.1: #进行筛选
sentences.append(line)
#整理字符和ID之间的映射
chars = {
}
for sentence in sentences:
for c in sentence:
chars[c] = chars.get(c, 0) + 1
chars = sorted(chars.items(), key=lambda x:x[1], reverse=True)
chars = [char[0] for char in chars]
vocab_size = len(chars)
char2id = {
c: i for i, c in enumerate(chars)}
id2char = {
i: c for i, c in enumerate(chars)}
with open('dictionary.pkl', 'wb') as fw:
pickle.dump([char2id, id2char], fw)
2. Model construction
Related operations are as follows:
maxlen = 10
step = 3
embed_size = 128
hidden_size = 128
vocab_size = len(chars)
batch_size = 64
epochs = 20
X_data = []
Y_data = []
for sentence in sentences:
for i in range(0, len(sentence) - maxlen, step): #每次平移3
#根据前面的字来预测后面一个字
X_data.append([char2id[c] for c in sentence[i: i + maxlen]])
y = np.zeros(vocab_size, dtype=np.bool)
y[char2id[sentence[i + maxlen]]] = 1
Y_data.append(y)
X_data = np.array(X_data)
Y_data = np.array(Y_data)
print(X_data.shape, Y_data.shape)
#定义序列模型
model = Sequential()
model.add(Embedding(input_dim=vocab_size, output_dim=embed_size, input_length=maxlen))
model.add(LSTM(hidden_size, input_shape=(maxlen, embed_size)))
model.add(Dense(vocab_size, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
#定义序列样本生成函数
def sample(preds, diversity=1.0):
preds = np.asarray(preds).astype('float64')
preds = np.log(preds + 1e-10) / diversity #取对数
exp_preds = np.exp(preds)
preds = exp_preds / np.sum(exp_preds)
probas = np.random.multinomial(1, preds, 1)
return np.argmax(probas) #取最大的值拿到生成的字
#定义每轮训练后的回调函数
def on_epoch_end(epoch, logs):
print('-' * 30)
print('Epoch', epoch)
index = random.randint(0, len(sentences)) #随机选取一句
for diversity in [0.2, 0.5, 1.0]:
print('----- diversity:', diversity)
sentence = sentences[index][:maxlen] #随机选一首歌把前十个字取出
print('----- Generating with seed: ' + sentence)
sys.stdout.write(sentence)
for i in range(400):
x_pred = np.zeros((1, maxlen))
for t, char in enumerate(sentence):
x_pred[0, t] = char2id[char] #把x相应位置的值改为ID
preds = model.predict(x_pred, verbose=0)[0]
next_index = sample(preds, diversity)
next_char = id2char[next_index] #把下一个字拿出来
sentence = sentence[1:] + next_char
sys.stdout.write(next_char)
sys.stdout.flush()
3. Model training and saving
Related operations are as follows:
model.fit(X_data, Y_data, batch_size=batch_size, epochs=epochs, callbacks=[LambdaCallback(on_epoch_end=on_epoch_end)])
model.save('song.h5')
4. Generate lyrics
The following code uses the previously generated model to generate lyrics and needs to provide a starting lyric:
#导入所需要的库
from keras.models import load_model
import numpy as np
import pickle
import sys
maxlen = 10
model = load_model('song.h5') #加载使用上一段代码生成的模型
with open('dictionary.pkl', 'rb') as fr: #打开.pkl文件
[char2id, id2char] = pickle.load(fr)
#定义序列样本生成函数
def sample(preds, diversity=1.0):
preds = np.asarray(preds).astype('float64')
preds = np.log(preds + 1e-10) / diversity #使用对数
exp_preds = np.exp(preds)
preds = exp_preds / np.sum(exp_preds)
probas = np.random.multinomial(1, preds, 1)
return np.argmax(probas) #最大的概率
#提供一句起始歌词
sentence = '天地玄黄宇宙洪荒'
sentence = sentence[:maxlen] #不能超过最大长度
diversity = 1.0
print('----- Generating with seed: ' + sentence)
print('----- diversity:', diversity)
sys.stdout.write(sentence)
for i in range(400): #迭代400次
x_pred = np.zeros((1, maxlen)) #初始化为0
for t, char in enumerate(sentence):
x_pred[0, t] = char2id[char]
preds = model.predict(x_pred, verbose=0)[0]
next_index = sample(preds, diversity)
next_char = id2char[next_index] #把下一个字拿出来
sentence = sentence[1:] + next_char
sys.stdout.write(next_char)
sys.stdout.flush()
System test
This part includes two parts: generating ancient poems and acrostic poems, and generating lyrics.
1. Generate ancient poems and acrostic poems
This section includes training models for ancient poem generation, ancient poems, acrostic poems and word clouds.
1) Training the model generated by ancient poems
The model is saved in the corresponding file for subsequent code to generate ancient poems and acrostic poems, as shown in the figure.
2) Generate ancient poems
Use the previous code to train and save the model to generate four sentences of seven-character verses, as shown in the figure.
3) Generate acrostics
Use the previous code to train and save the model to generate Tibetan acrostic poems, and input "Heaven and Earth Xuanhuang", the result is shown in the figure.
Enter "universe prehistoric", the result is shown in the figure.
Enter "cold to summer", the result is shown in the figure.
4) Generate word cloud
Generate an ordinary ancient poem and three acrostic poems and save them in a file to generate a word cloud, as shown in Figure 1-4.
2. Generate lyrics
The model is saved in the corresponding file for subsequent use. The training generation model is shown in Figure 5, and the result of generating lyrics is shown in Figure 6.
Project source code download
See my blog resource download page for details
Other information download
If you want to continue to learn about artificial intelligence-related learning routes and knowledge systems, welcome to read my other blog " Heavy | Complete artificial intelligence AI learning-basic knowledge learning route, all materials can be downloaded directly from the network disk without paying attention to routines "
This blog refers to Github's well-known open source platform, AI technology platform and experts in related fields: Datawhale, ApacheCN, AI Youdao and Dr. Huang Haiguang, etc. There are about 100G related materials, and I hope to help all friends.