Hello everyone, I am Wei Xue AI. Today I will introduce to you the Natural Language Processing Practical Project 18-Logits and Loss Function Calculation Application Project in NLP Model Training. In NLP model training, Logits are often used to calculate the loss function and perform optimization. The calculation of the loss function is used to measure the difference between the model prediction results and the true label, thereby guiding the update of model parameters.
Logits are the output of the model in the classification task, and the probability distribution of the category can be obtained after passing the Softmax function. By inputting logits into the cross-entropy loss function, the gap between the model's predictions and the true labels can be calculated, thereby measuring the model's performance.
Table of contents
- introduction
- Understanding Logits
- Understand the loss function
- Chinese text sentence training process
- Implementation code: Chinese text sentence training
- Summarize
1 Introduction
In the field of natural language processing (NLP), model training is an important link. During the model training process, we need to understand and master some key concepts, such as Logits and loss functions. These two concepts are critical to understanding and optimizing model performance.
2 Understand Logits
Logits usually refers to the original predicted value obtained through the last layer of linear transformation of the neural network (but without normalization operations such as softmax or sigmoid). For example, we use models such as BERT to do the following