Intensive lectures on practical application cases of MATLAB algorithms-[Deep Learning] Pre-training Model Transformer-XL&Longformer

Table of contents

Transformer-XL： Attentive Language Models Beyonds a Fixed-Length Context

1. The origin of Transformer-XL

2. Transformer-XL modeling longer sequences

2.1 Segment-Level Cycle Mechanism