【提示学习】Automatic Multi-Label Prompting: Simple and Interpretable Few-Shot Classification

Paper information

name	content
paper title	Automatic Multi-Label Prompting: Simple and Interpretable Few-Shot Classification
Paper address	https://arxiv.org/abs/2204.06305
field of study	NLP, text classification, hint learning
proposed model	AMuLaP
source	NAACL 2022 (main conference)

read summary

  The article proposes a simple and efficient way to build verbalization:

  1.For each label $y_i\in Y$ , iterate over all training samples$x_j\in D_{train}$，$x_j$The ground truth label of is also $y_i$. Use the model to predict [MASK]the labeling probability of the marker, and take the average of the predicted probabilities of these n samples as $z_i$，in which $z_i$is a vector over the entire vocabulary representing the average probability for each word in the vocabulary.

  2.For each $y_i\in Y$ , initialize an empty candidate token set$S\left(yi\right)$ .

  3.For every $v\in V$ , where$V$ is the vocabulary of the model, we start from$z_i$Retrieve vv inThe probability value of$v$$z_i^v$。

  4.Iterate through all tags, each tag has a $z$ , traversing each position, will$v$ assigns the most probable token set S ( ym ) S(y_m)of the mth class$S({and}_m)$，其中$m=argma{x}_i{z}_i^v$。

Suppose there are 3 labels and the vocabulary has 2000, then the shape of Z is 3*2000, and then traverse the vocabulary [0-1999], we compare the probability of each word on the 3 labels, take the largest and then put Enter the S ( y ) S(y) corresponding to this label$S\left(y\right)$ .

  5.对于$y_i\in Y$ , we start from$Select\; the\; top-k\; tokens\; with\; the\; largest\; probability\; z^v\_i\; in\; S\left(yi\right)$ , and get the truncated word set$S\left(yi\right)$ .