https://arxiv.org/pdf/2109.01652.pdf
Propose a paper on instruction learning
Instructional learning is an idea proposed by the Quoc V.Le team of Google Deepmind in an article titled "Finetuned Language Models Are Zero-Shot Learners" in 2021. The purpose of instruction learning and hint learning is to mine the knowledge possessed by the language model itself. The difference is that Prompt stimulates the completion ability of the language model , such as generating the second half of the sentence based on the first half of the sentence, or filling in the blanks. Instruct is to stimulate the comprehension ability of the language model, which allows the model to take correct actions by giving more obvious instructions. We can understand these two different learning styles with the following example:
- Tips for learning: Bought this necklace for my girlfriend, she likes it very much, this necklace is too ____.
- Instructions to learn: Judge the sentiment of the sentence: I bought this necklace for my girlfriend and she loves it. Options: A=good; B=fair; C=bad.
The advantage of instruction learning is that it can also do zero-shot on other tasks after multi-task fine-tuning, while hint learning is all for one task. Generalization is not as good as instruction learning. We can understand fine-tuning, hint learning and instruction learning through Figure 2.
