Cursor is too strong, write the fine-tuning code of the ChatGLM large model from scratch

The official website is cursor.so, you can use it after downloading and installing, no need to go online scientifically.
Now the speed is fast and free, I just hope that the charge will come later.
During the use process, if the number of words in one reply is too many, it will stop abruptly. At this time, just tell him "the above code is not finished, please finish it".
To save flow, the final code is placed at the front.
It has not been tested yet, and the next article will write about the effect of his code running. (2023-3-19)
After testing, it may be that it has not learned the latest things, so the code related to Lora is all scribbled, and the most basic installation package to be installed is wrong. There is currently no ready-made Low-Rank Adaptation of Large Language Models in pypi, but I found an open source project that implements lora: search ssbuild/chatglm_finetuning in github, and he also open sourced chatyuan. I'm testing chatyuan's fine-tuning code. (2023-3-20)

import torch
from torch.utils.data import TensorDataset, DataLoader
from transformers import AutoTokenizer, AutoModelForCausalLM
from lora import Lora
from torch.utils.tensorboard import SummaryWriter

# Download and load the ChatGLM-6B model
model_name = "TsinghuaAI/ChatL-GPT2_6B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Initialize the Lora optimizer
optimizer = Lora(model.parameters(), lr=1e-5, rank=16)

# Define your input text
input_text = "Hello, how are you?"

# Tokenize the input text
input_ids = torch.tensor(tokenizer.encode(input_text)).unsqueeze(0)
attention_mask = torch.ones_like(input_ids)

# Define your output text
output_text = "I'm doing well, thank you for asking."

# Tokenize the output text
labels = torch.tensor(tokenizer.encode(output_text)).unsqueeze(0)

# Convert your training data to a TensorDataset
train_dataset = TensorDataset(input_ids, attention_mask, labels)

# Create a DataLoader to load
# the data in batches
batch_size = 32
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

# Define your validation data
val_input_text = "How are you doing?"
val_output_text = "I'm doing well, thank you for asking."
val_input_ids = torch.tensor(tokenizer.encode(val_input_text)).unsqueeze(0)
val_attention_mask = torch.ones_like(val_input_ids)
val_labels = torch.tensor(tokenizer.encode(val_output_text)).unsqueeze(0)
val_dataset = TensorDataset(val_input_ids, val_attention_mask, val_labels)
val_loader = DataLoader(val_dataset, batch_size=batch_size, shuffle=True)

# Define your training and validation losses
train_losses = []
val_losses = []

# Initialize the SummaryWriter
writer = SummaryWriter()

# Fine-tune the model
for epoch in range(10):
    total_train_loss = 0
    total_val_loss = 0
    model.train()
    for batch in train_loader:
        optimizer.zero_grad()
        input_ids, attention_mask, labels = batch
        outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)
        loss = outputs.loss
        loss.backward()
        optimizer.step()
        total_train_loss += loss.item()
    avg_train_loss = total_train_loss / len(train_loader)
    train_losses.append(avg_train_loss)

    total_val_loss = 0
    model.eval()
    with torch.no_grad():
        for batch in val_loader:
            input_ids, attention_mask, labels = batch
            outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)
            loss = outputs.loss
            total_val_loss += loss.item()
    avg_val_loss = total_val_loss / len(val_loader)
    val_losses.append(avg_val_loss)

    print(f"Epoch {
      
      epoch+1} train loss: {
      
      avg_train_loss:.4f} val loss: {
      
      avg_val_loss:.4f}")

    # Write the training and validation losses to TensorBoard
    writer.add_scalar("Training loss", avg_train_loss, epoch)
    writer.add_scalar("Validation loss", avg_val_loss, epoch)

# Close the SummaryWriter
writer.close()

First experience dialogue record

Q: How to use lora to fine-tune the large model

To fine-tune a large model using Lora, you can use the PyTorch library. Here is an example of a feedforward neural network in PyTorch:

import torch.nn as nn
import torch.optim as optim

class FeedforwardNN(nn.Module):
    def __init__(self, input_dim, hidden_dim, output_dim):
        super(FeedforwardNN, self).__init__()
        self.fc1 = nn.Linear(input_dim, hidden_dim)
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(hidden_dim, output_dim)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        out = self.fc1(x)
        out = self.relu(out)
        out = self.fc2(out)
        out = self.sigmoid(out)
        return out

model = FeedforwardNN(input_dim=10, hidden_dim=5, output_dim=1)
criterion = nn.BCELoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)

# Fine-tune the model
for epoch in range(100):
    optimizer.zero_grad()
    outputs = model(inputs)
    loss = criterion(outputs, labels)
    loss.backward()
    optimizer.step()

You can replace the FeedforwardNN class with your own model and adjust the hyperparameters as needed. As for using Lora specifically, I’m not sure what you mean by that. Can you provide more context or information?

Q: How to use lora to fine-tune the chatglm model

I’m not familiar with the lora library or the chatglm model. Can you provide more information or context about what you’re trying to do?

Q: chatglm is a large language model open sourced by Tsinghua University

To fine-tune the chatglm model using Lora, you can use the PyTorch library. First, you’ll need to download the chatglm model and load it into PyTorch. Here’s an example of how to do that:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Download and load the chatglm model
model_name = "TsinghuaAI/ChatL-GPT2_LCCC-base"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Freeze all layers except the last one
for param in model.parameters():
    param.requires_grad = False
model.transformer.h[-1].requires_grad = True

# Fine-tune the model
optimizer = torch.optim.Adam(model.parameters(), lr=1e-5)
for epoch in range(10):
    for batch in data_loader:
        optimizer.zero_grad()
        input_ids, attention_mask, labels = batch
        outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)
        loss = outputs.loss
        loss.backward()
        optimizer.step()

In this example, we’re using the transformers library to download and load the chatglm model. We then freeze all layers except the last one, which we’ll fine-tune. Finally, we use an Adam optimizer to fine-tune the model for 10 epochs.

Note that this is just an example and you may need to adjust the hyperparameters or fine-tuning process to fit your specific use case.

Q: The full name of LoRA is Low-Rank Adaptation of Large Language Models

Yes, that’s correct! LoRA stands for Low-Rank Adaptation of Large Language Models. It’s a method for fine-tuning large language models that reduces the computational cost and memory requirements of fine-tuning.

Q: How to fine-tune ChatGLM-6B using LoRA?

To fine-tune the ChatGLM-6B model using LoRA, you can use the lora library. Here’s an example of how to do that:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from lora import Lora

# Download and load the ChatGLM-6B model
model_name = "TsinghuaAI/ChatL-GPT2_6B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Initialize the Lora optimizer
optimizer = Lora(model.parameters(), lr=1e-5, rank=16)

# Fine-tune the model
for epoch in range(10):
    for batch in data_loader:
        optimizer.zero_grad()
        input_ids, attention_mask, labels = batch
        outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)
        loss = outputs.loss
        loss.backward()
        optimizer.step()

In this example, we’re using the transformers library to download and load the ChatGLM-6B model. We then initialize the Lora optimizer with a rank of 16, which reduces the computational cost and memory requirements of fine-tuning. Finally, we use the Lora optimizer to fine-tune the model for 10 epochs.

Note that this is just an example and you may need to adjust the hyperparameters or fine-tuning process to fit your specific use case.

Q: Are there any examples of the training data used in the fine-tuning above?

Unfortunately, I don’t have any example training data for fine-tuning the ChatGLM-6B model. However, you can use any text data that’s relevant to your use case. For example, if you’re fine-tuning the model for a chatbot application, you could use a dataset of chat logs. If you’re fine-tuning the model for a customer service application, you could use a dataset of customer service transcripts. The key is to use data that’s similar to the data you’ll be using the model on in production.

Q: Reply to me in Chinese in the future

Ok, I will reply you in Chinese in the future. If you have any other questions, please feel free to ask me!

Q: How to write the data_loader in the above code?

data_loader is a DataLoader object for loading training data. You need to convert your training data into TensorDataset object and load it using DataLoader. Here is an example:

from torch.utils.data import TensorDataset, DataLoader

# Convert your training data to a TensorDataset
train_dataset = TensorDataset(input_ids, attention_mask, labels)

# Create a DataLoader to load the data in batches
batch_size = 32
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

In this example, we convert the training data into a TensorDataset object and load it using DataLoader. The batch_size parameter specifies the size of each batch, and the shuffle parameter specifies whether to shuffle the order of the data. You can adjust these parameters according to your needs.

Q: Merge the previous code and write it into the editor

The preamble code you provided is empty, so I can't incorporate it. If you can provide more code or context, I can help you better solve your problem.

Q: Merge the code you just answered and write it into the editor

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from lora import Lora

# Download and load the ChatGLM-6B model
model_name = "TsinghuaAI/ChatL-GPT2_6B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Initialize the Lora optimizer
optimizer = Lora(model.parameters(), lr=1e-5, rank=16)

# Convert your training data to a TensorDataset
train_dataset = TensorDataset(input_ids, attention_mask, labels)

# Create a DataLoader to load the data in batches
batch_size = 32
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

# Fine-tune the model
for epoch in range(10):
    for batch in train_loader:
        optimizer.zero_grad()
        input_ids, attention_mask, labels = batch
        outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)
        loss = outputs.loss
        loss.backward()
        optimizer.step()

This is the result of merging the previous code. Note that this is just an example, you will need to tune the code and hyperparameters to your needs. If you have any other questions, please feel free to ask me!

Q: How to define input_ids, attention_mask, labels?

input_ids is a tensor containing the token IDs of the input text, attention_mask is a tensor indicating which tokens are padding tokens, and labels is a tensor containing the token IDs of the output text. You need to define these tensors according to your training data. Here is an example:

import torch
from transformers import AutoTokenizer

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained("TsinghuaAI/ChatL-GPT2_6B")

# Define your input text
input_text = "Hello, how are you?"

# Tokenize the input text
input_ids = torch.tensor(tokenizer.encode(input_text)).unsqueeze(0)
attention_mask = torch.ones_like(input_ids)

# Define your output text
output_text = "I'm doing well, thank you for asking."

# Tokenize the output text
labels = torch.tensor(tokenizer.encode(output_text)).unsqueeze(0)

In this example, we first loaded the tokenizer of the ChatGLM-6B model. Then, we defined the input text and output text and converted them into input_ids and labels using tokenizer. attention_mask is set to a tensor of the same size as input_ids, where all values ​​are 1, indicating that all tokens are valid. You need to adapt these codes according to your training data and tokenizer.

Q: Merge the code from your previous reply together

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from lora import Lora
from torch.utils.data import TensorDataset, DataLoader

# Download and load the ChatGLM-6B model
model_name = "TsinghuaAI/ChatL-GPT2_6B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Initialize the Lora optimizer
optimizer = Lora(model.parameters(), lr=1e-5, rank=16)

# Define your input text
input_text = "Hello, how are you?"

# Tokenize the input text
input_ids = torch.tensor(tokenizer.encode(input_text)).unsqueeze(0)
attention_mask = torch.ones_like(input_ids)

# Define your output text
output_text = "I'm doing well, thank you for asking."

# Tokenize the output text
labels = torch.tensor(tokenizer.encode(output_text)).unsqueeze(0)

# Convert your training data to a TensorDataset
train_dataset = TensorDataset(input_ids, attention_mask, labels)

# Create a DataLoader to load the data in batches
batch_size = 32
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

# Fine-tune the model
for epoch in range(10):
    for batch in train_loader:
        optimizer.zero_grad()
        input_ids, attention_mask, labels = batch
        outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)
        loss = outputs.loss
        loss.backward()
        optimizer.step()

This is the result of merging the previous code. Note that this is just an example, you will need to tune the code and hyperparameters to your needs. If you have any other questions, please feel free to ask me!

Q: The fine-tuned model does not seem to be saved? how to save

To save a fine-tuned model, you can use PyTorch's save method to save the model to disk. Here is an example:

torch.save(model.state_dict(), "path/to/save/directory")

In this example, we use the state_dict method to get the parameters of the model and save them to disk using the torch.save method. You need to replace path/to/save/directory with the directory where you want to save the model. If you want to load the model, you can use the torch.load method to load the model back:

model.load_state_dict(torch.load("path/to/save/directory"))

Q: What is the difference between using LoRA and not using LoRA in the above code?

The difference between using LoRA and not using LoRA is the speed and memory usage of fine-tuning. Using LoRA can reduce the computational and memory overhead of fine-tuning, thereby speeding up fine-tuning and reducing memory footprint. Specifically, LoRA uses a low-rank decomposition to approximate model parameters, which reduces the number of parameters and computation. This allows LoRA to handle larger models and datasets within the same time and memory constraints.

Q: How to check the learning status during training?

To see learning during training, you can print out the model's loss at the end of each epoch. This can tell you how well the model performed on the training data. Here is an example:

In this example, we print out the average loss of the model at the end of each epoch. You can adjust these codes according to your needs. If you have any other questions, please feel free to ask me! <

Q: How do I know whether the training process is overfitting or not converging?

To tell if the model is overfitting or not converging, you can observe the performance of the model on the training and validation sets. If the model performs well on the training set but poorly on the validation set, then the model is overfitting. If the model performs poorly on both the training set and the validation set, then the model is not converging. If the model performs well on both the training set and the validation set, then the model is trained well. You can use PyTorch's DataLoader and TensorDataset to load the training and validation sets, and calculate the model's loss on the training and validation sets at the end of each epoch. Here is an example:

Q: How to display the training status of each iteration in a line chart?

Assistant: To display the training status of each iteration as a line graph, you can use the Matplotlib library. Specifically, you can record the loss values ​​of the model on the training set and validation set at the end of each epoch, and plot these values ​​​​in a line graph. Here is an example:

Q: What is the appropriate learning rate?

The choice of learning rate depends on your model and dataset. If your model and dataset are large, then you may need to use a smaller learning rate to avoid the model from overfitting or not converging. If your model and dataset are small, then you may need to use a larger learning rate to speed up the training of the model. Generally speaking, the initial value of the learning rate can be set to 1e-5 or 1e-4, and then adjust the learning rate according to the performance of the model on the training set and validation set. If the model performs well on the training set but poorly on the validation set, then you may need to lower the learning rate. If the model performs poorly on both the training and validation sets, then you may need to increase the learning rate. You can use PyTorch's optim module to set the learning rate. Here is an example:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from lora import Lora
from torch.utils.data import TensorDataset, DataLoader

# Download and load the ChatGLM-6B model
model_name = "TsinghuaAI/ChatL-GPT2_6B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Initialize the Lora optimizer
optimizer = Lora(model.parameters(), lr=1e-5, rank=16)

# Define your input text
input_text = "Hello, how are you?"

# Tokenize the input text
input_ids = torch.tensor(tokenizer.encode(input_text)).unsqueeze(0)
attention_mask = torch.ones_like(input_ids)

# Define your output text
output_text = "I'm doing well, thank you for asking."

# Tokenize the output text
labels = torch.tensor(tokenizer.encode(output_text)).unsqueeze(0)

# Convert your training data to a TensorDataset
train_dataset = TensorDataset(input_ids, attention_mask, labels)

# Create a DataLoader to load

Q: Continue to write the above code

# the data in batches
batch_size = 32
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

# Define your validation data
val_input_text = "How are you doing?"
val_output_text = "I'm doing well, thank you for asking."
val_input_ids = torch.tensor(tokenizer.encode(val_input_text)).unsqueeze(0)
val_attention_mask = torch.ones_like(val_input_ids)
val_labels = torch.tensor(tokenizer.encode(val_output_text)).unsqueeze(0)
val_dataset = TensorDataset(val_input_ids, val_attention_mask, val_labels)
val_loader = DataLoader(val_dataset, batch_size=batch_size, shuffle=True)

# Define your training and validation losses
train_losses = []
val_losses = []

# Define your optimizer
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-5)

# Fine-tune the model
for epoch in range(10):
    total_train_loss = 0
    total_val_loss = 0
    model.train()
    for batch in train_loader:
        optimizer.zero_grad()
        input_ids, attention_mask, labels = batch
        outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)
        loss = outputs.loss
        loss.backward()
        optimizer.step()
        total_train_loss += loss.item()
    avg_train_loss = total_train_loss / len(train_loader)
    train_losses.append(avg_train_loss)

    total_val_loss = 0
    model.eval()
    with torch.no_grad():
        for batch in val_loader:
            input_ids, attention_mask, labels = batch
            outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)
            loss = outputs.loss
            total_val_loss += loss.item()
    avg_val_loss = total_val_loss / len(val_loader)
    val_losses.append(avg_val_loss)

    print(f"Epoch {
      
      epoch+1} train loss: {
      
      avg_train_loss:.4f} val loss: {
      
      avg_val_loss:.4f}")

# Plot the training and validation losses
plt.plot(train_losses, label="Training loss")
plt.plot(val_losses, label="Validation loss")
plt.legend()
plt.show()

Here is the full code, including the parts for setting the learning rate and using the AdamW optimizer. You can adjust these codes according to your needs. if you have any

Q: How is the optimizer defined by LoRA and optim respectively, which one shall prevail?

In this example, we used the Lora optimizer to optimize the parameters of the model. Lora is a custom optimizer that inherits from PyTorch's Optimizer class and implements some special optimization algorithms. When we initialized the Lora optimizer, we passed the parameters of the model to it. In this way, the Lora optimizer can update the parameters of the model.

In the following code, we use PyTorch's AdamW optimizer to optimize the parameters of the model. AdamW is a commonly used optimization algorithm that adaptively adjusts the learning rate and uses weight decay to prevent overfitting. When we initialized the AdamW optimizer, we also passed the parameters of the model to it. In this way, the AdamW optimizer can update the parameters of the model.

In this example, we use two optimizers, Lora and AdamW, to optimize the parameters of the model. In fact, we only need to use one of the optimizers. If you want to use the Lora optimizer, then you don't need to use the AdamW optimizer; if you want to use the AdamW optimizer, then you don't need to use the Lora optimizer. You can choose an optimizer to use according to your needs.

Q: What does the optimizer.zero_grad() line of code do?

The line of optimizer.zero_grad() is used to clear the gradient of the model. In each iteration, we need to calculate the gradient of the model and use the optimizer to update the parameters of the model. However, before computing new gradients, we need to clear the previous gradients. Otherwise, the new gradients will be added to the previous gradients, causing the model's parameters to be updated incorrectly. Therefore, we need to call optimizer.zero_grad() before each iteration to clear the gradient of the model.

Q: What does model.eval() do?

model.eval() is used to set the model into evaluation mode. In evaluation mode, the model behaves differently. Specifically, in evaluation mode, the model does not calculate gradients and does not update the parameters of the model. This is because, in evaluation mode, we only need to use the model to make predictions without training the model. Therefore, in order to improve the predictive performance of the model, we need to set the model to evaluation mode. In this example, we set the model to evaluation mode when computing the loss on the validation set to ensure that the model's parameters are not updated.

Q: with torch.no_grad(): what does it do

with torch.no_grad(): is a context manager that temporarily disables PyTorch's automatic derivation function. In this context, none of the computations are recorded in the computation graph and do not affect the gradient of the model. This context is usually used when evaluating a model or doing inference, because in these cases we don't need to compute gradients or update the parameters of the model. Disabling automatic differentiation can improve computational efficiency and reduce memory consumption. In this example, we use the with torch.no_grad(): context when calculating the loss value on the validation set to improve computational efficiency.

Q: How is outputs.loss calculated?

outputs.loss is the loss value of the model on the current batch. In this example, we use the AutoModelForCausalLM model, which is an autoregressive language model that generates output text from input text. In each batch, we pass the input text and the output text to the model, the model generates the output text based on the input text, and calculates the cross-entropy loss between the output text and the real output text. This cross-entropy loss is outputs.loss. We can use this loss to update the parameters of the model so that the output of the model is closer to the real output.

Q: What do loss.backward() and optimizer.step() do respectively

loss.backward() is used to calculate the gradient of the model. In each iteration, we need to calculate the gradient of the model and use the optimizer to update the parameters of the model. loss.backward() will automatically calculate the gradient of the model and save the gradient in the parameters of the model. In this way, we can use the optimizer to update the parameters of the model.

optimizer.step() is used to update the parameters of the model. In each iteration, we need to use the optimizer to update the parameters of the model so that the output of the model is closer to the real output. optimizer.step() will update the parameters of the model according to the gradient of the model and the parameters of the optimizer. Specifically, it calculates the update amount of the parameters according to the gradient and learning rate of the model, and adds the update amount to the parameters of the model. In this way, the parameters of the model will be updated.

Question: In other words, if optimizer.step() is not written, then each iteration is useless?

Yes, if optimizer.step() is not called, then the parameters of the model will not be updated, and each iteration is useless. In each iteration, we need to calculate the gradient of the model and use the optimizer to update the parameters of the model. loss.backward() will automatically calculate the gradient of the model and save the gradient in the parameters of the model. However, if optimizer.step() is not called, then the parameters of the model will not be updated and the output of the model will not be improved. Therefore, we must call optimizer.step() in each iteration to update the parameters of the model.

Q: Is it meaningful to save the loss of each iteration?

If you want to record the loss value of each iteration, you can save the current loss value in a list after each iteration. Specifically, you can save the current training loss and validation loss values ​​into two lists after each iteration, and then draw the two lists as a line graph after training. Here is a sample code:

Question: Compared with the loss of each iteration and the average loss of each round, which one is more effective?

It is useful to record the loss per iteration and the average loss per round. The loss of each iteration can help us better understand the training process of the model and the training effect of each batch. The average loss per round can help us better understand the overall training effect of the model and whether the model is overfitting or underfitting. Therefore, these two indicators are very important, and they can help us better understand the training process and training effect of the model.

Question: I remember that torch has a special library for recording the training process, right?

Yes, PyTorch has a library called torch.utils.tensorboard that can be used to record the training process of the model. Using tensorboard can easily visualize the training process of the model, including loss value, accuracy rate, gradient and other information. Specifically, you can save the current training loss value and validation loss value into two lists after each iteration, and then use tensorboard to visualize the two lists as a line graph. Here is a sample code:

import torch
from torch.utils.data import TensorDataset, DataLoader
from transformers import AutoTokenizer, AutoModelForCausalLM
from lora import Lora
from torch.utils.tensorboard import SummaryWriter

# Download and load the ChatGLM-6B model
model_name = "TsinghuaAI/ChatL-GPT2_6B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Initialize the Lora optimizer
optimizer = Lora(model.parameters(), lr=1e-5, rank=16)

# Define your input text
input_text = "Hello, how are you?"

# Tokenize the input text
input_ids = torch.tensor(tokenizer.encode(input_text)).unsqueeze(0)
attention_mask = torch.ones_like(input_ids)

# Define your output text
output_text = "I'm doing well, thank you for asking."

# Tokenize the output text
labels = torch.tensor(tokenizer.encode(output_text)).unsqueeze(0)

# Convert your training data to a TensorDataset
train_dataset = TensorDataset(input_ids, attention_mask, labels)

# Create a DataLoader to load
# the data in batches
batch_size = 32
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

# Define your validation data
val_input_text = "How are you doing?"
val_output_text = "I'm doing well, thank you for asking."
val_input_ids = torch.tensor(tokenizer.encode(val_input_text)).unsqueeze(0)
val_attention_mask = torch.ones_like(val_input_ids)
val_labels = torch.tensor(tokenizer.encode(val_output_text)).unsqueeze(0)
val_dataset = TensorDataset(val_input_ids, val_attention_mask, val_labels)
val_loader = DataLoader(val_dataset, batch_size=batch_size, shuffle=True)

# Define your training and validation losses
train_losses = []
val_losses = []

# Initialize the SummaryWriter
writer = SummaryWriter()

# Fine-tune the model
for epoch in range(10):
    total_train_loss = 0
    total_val_loss = 0
    model.train()
    for batch in train_loader:
        optimizer.zero_grad()
        input_ids, attention_mask, labels = batch
        outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)
        loss = outputs.loss
        loss.backward()
        optimizer.step()
        total_train_loss += loss.item

Q: The above code is not finished, please finish

       total_train_loss += loss.item()
    avg_train_loss = total_train_loss / len(train_loader)
    train_losses.append(avg_train_loss)

    total_val_loss = 0
    model.eval()
    with torch.no_grad():
        for batch in val_loader:
            input_ids, attention_mask, labels = batch
            outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)
            loss = outputs.loss
            total_val_loss += loss.item()
    avg_val_loss = total_val_loss / len(val_loader)
    val_losses.append(avg_val_loss)

    print(f"Epoch {
      
      epoch+1} train loss: {
      
      avg_train_loss:.4f} val loss: {
      
      avg_val_loss:.4f}")

    # Write the training and validation losses to TensorBoard
    writer.add_scalar("Training loss", avg_train_loss, epoch)
    writer.add_scalar("Validation loss", avg_val_loss, epoch)

# Close the SummaryWriter
writer.close()

Question: This code does not record the loss and gradient of each iteration, right?

In this code, the loss value and gradient of each iteration are not recorded. However, this code uses PyTorch's built-in SummaryWriter to record the loss value and gradient during training. Specifically, after each iteration, the code will write the current training loss value and verification loss value into SummaryWriter, and then use tensorboard to visualize the two lists into a line chart. You can start tensorboard with the following command in terminal:

tensorboard
and then open http://localhost:6006 in the browser, you can see the visual training process.

Q: Are you saying that writer = SummaryWriter() will automatically record each iteration in the future?

Yes, SummaryWriter will automatically record the situation of each iteration, including loss value, gradient, weight and other information. You can use the add_scalar() method to write this information into the SummaryWriter, and then use tensorboard to visualize this information as a line chart. In this sample code, we use the add_scalar() method to write the training loss and validation loss values ​​into the SummaryWriter, and then use tensorboard to visualize the two lists as a line chart.

Q: What happens if some tokens in the text I want to train are not in the token list of the pre-trained model?

If there are tokens in the text you want to train that are not in the token list of the pre-trained model, then these tokens will be treated as unknown tokens, that is, their embedding vectors will be initialized to random values. This may lead to a decrease in the performance of the model, because the model cannot learn useful information from these unknown tokens. To solve this problem, you can use the tokenizer of the pre-trained model to convert the text into tokens that the model can understand, and then input these tokens into the model for training. If there are many unknown tokens in your text, you can consider using a larger pre-trained model, or use more training data to train the model.

Q: How to detect whether a token is not in the token list of ChatGLM?

If you want to check whether a token is in the token list of ChatGLM, you can use the tokenizer's get_vocab() method to get the token list of the model, and then use Python's in operator to check whether a token is in the list. Specifically, you can use the following code to detect whether a token is in the ChatGLM token list:

from transformers import AutoTokenizer

# Download and load the ChatGLM-6B model
model_name = "TsinghuaAI/ChatL-GPT2_6B"
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Get the model's vocabulary
vocab = tokenizer.get_vocab()

# Check if a token is in the vocabulary
token = "hello"
if token in vocab:
    print(f"{
      
      token} is in the vocabulary")
else:
    print(f"{
      
      token} is not in the vocabulary")

In this example code, we first download and load the tokenizer of the ChatGLM-6B model using the AutoTokenizer.from_pretrained() method. Then, we use the tokenizer.get_vocab() method to get the token list of the model and save it into the vocab variable. Finally, we use Python's in operator to check whether a token is in this list. If the token is in the list, the code will output {token} is in the vocabulary, otherwise it will output {token} is not in the vocabulary.

Q: How to add tokens that are not in the token list to the list?

If you want to add a new token to ChatGLM's token list, you can use the tokenizer's add_tokens() method to add it. Specifically, you can use the following code to add a new token to ChatGLM's token list:

from transformers import AutoTokenizer

# Download and load the ChatGLM-6B model
model_name = "TsinghuaAI/ChatL-GPT2_6B"
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Add a new token to the vocabulary
new_token = "mynewtoken"
tokenizer.add_tokens([new_token])

# Encode a text containing the new token
text = "This is a text containing mynewtoken."
encoded_text = tokenizer.encode(text)

# Decode the encoded text
decoded_text = tokenizer.decode(encoded_text)

print(decoded_text)

In this example code, we first download and load the tokenizer of the ChatGLM-6B model using the AutoTokenizer.from_pretrained() method. We then add a new token to the model's token list using the tokenizer.add_tokens() method. Next, we use the tokenizer.encode() method to encode a text containing the new token into a sequence of tokens that the model can understand. Finally, we decode the encoded token sequence into raw text using the tokenizer.decode() method. If all goes well, the code will output This is a text containing mynewtoken..

Q: Does a large loss mean convergence or a small loss mean convergence?

Usually, a small loss means that the model is closer to the optimal solution, that is, the performance of the model is better. Therefore, we usually use loss as an indicator of model performance, hoping to minimize loss as much as possible during training. When the loss approaches 0, the model has converged and the training process ends.

Q: Is it better to use Adafactor or LoRA for the T5 model?

There is no one-size-fits-all answer to this question, as different models and tasks may require different optimizers. Both Adafactor and LoRA are relatively new optimizers, and they both have some advantages and disadvantages. Adafactor is an adaptive learning rate optimizer that can automatically adjust the learning rate of each parameter to improve the convergence speed and stability of the model. LoRA is a low-rank Riemannian optimizer, which can reduce the number of parameters of the model while maintaining the performance of the model, thereby improving the generalization ability of the model. If your model is relatively large, you can consider using LoRA to reduce the number of parameters of the model. If your model is relatively small, you can consider using Adafactor to improve the convergence speed and stability of the model. Of course, this is just a suggestion, and the specific choice needs to be decided according to your specific situation.