The return value token_type_ids of AutoTokenizer in transformers (2)

In many cases, AutoTokenizer will return the result of token_type_ids:

 Explanation of token_type_ids:

For two sentence pairs, the previous sentence is marked as 0, and the next sentence is marked as 1.

Guess you like

Origin blog.csdn.net/wtl1992/article/details/132046021