© Author｜Wang Xiaolei
Institution｜Renmin University of China
Directions | Conversational Information Access
By | RUC AI Box
Enter the NLP group —> join the NLP exchange group (remark nips/emnlp/nlpcc enters the corresponding contribution group)
This article sorts out the papers related to large language models published in top conferences since 2022.
guide
At the end of last year, ChatGPT launched by OpenAI has become popular all over the world in just a few months. This large-scale language model based on GPT-3.5 has amazing natural language generation and understanding capabilities, and can perform tasks such as dialogue, translation, and summarization like humans. Due to its excellent performance, ChatGPT and the large language model behind it quickly became a hot topic in the field of artificial intelligence, attracting the attention and participation of a large number of researchers and developers.
This article sorts out 100 papers related to large language models published in major conferences (ACL, EMNLP, ICLR, ICML, NeurIPS, etc.) in 2022 . The list of papers has been synchronously updated to the Github warehouse (https://github.com/RUCAIBox/Top-conference-paper-list) , welcome to pay attention and Star.
Catalog (catalog)
Training【Training】
Pre-Training [pre-training]
Instruction Tuning [instruction fine-tuning]
Utilization
In-Context Learning【Context Learning】
Chain-of-Thought Prompting [Thinking Chain Tips]
Compression [compression]
Others【Other】
Application [application]
Multi-Modal【Multi-modal】
Code [code]
Retrieval [retrieval]
Text Generation [text generation]
Others【Other】
Analysis & Evaluation【Analysis and Evaluation】
Training【Training】
Pre-Training [pre-training]
UL2: Unifying Language Learning Paradigms
Learning to Grow Pretrained Models for Efficient Transformer Training
Efficient Large Scale Language Modeling with Mixtures of Experts
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
InCoder: A Generative Model for Code Infilling and Synthesis
CodeBPE: Investigating Subtokenization Options for Large Language Model Pretraining on Source Code
CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search
UniMax: Fairer and More Effective Language Sampling for Large-Scale Multilingual Pretraining
GLM-130B: An Open Bilingual Pre-trained Model
When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial Domain
Instruction Tuning [instruction fine-tuning]
What Makes Instruction Learning Hard? An Investigation and a New Challenge in a Synthetic Environment
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning
Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
Boosting Natural Language Generation from Instructions with Meta-Learning
Help me write a Poem - Instruction Tuning as a Vehicle for Collaborative Poetry Writing
Multitask Instruction-based Prompting for Fallacy Recognition
Not All Tasks Are Born Equal: Understanding Zero-Shot Generalization
HypeR: Multitask Hyper-Prompted Training Enables Large-Scale Retrieval Generalization
Utilization
In-Context Learning【Context Learning】
What learning algorithm is in-context learning? Investigations with linear models
Ask Me Anything: A simple strategy for prompting language models
Large Language Models are Human-Level Prompt Engineers
Using Both Demonstrations and Language Instructions to Efficiently Learn Robotic Tasks
kNN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference
Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners
Selective Annotation Makes Language Models Better Few-Shot Learners
Active Example Selection for In-Context Learning
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
In-Context Learning for Few-Shot Dialogue State Tracking
Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback
Controllable Dialogue Simulation with In-context Learning
Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again
XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsing
On the Compositional Generalization Gap of In-Context Learning
Towards In-Context Non-Expert Evaluation of Reflection Generation for Counselling Conversations
Towards Few-Shot Identification of Morality Frames using In-Context Learning
Chain-of-Thought Prompting [Thinking Chain Tips]
ReAct: Synergizing Reasoning and Acting in Language Models
Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning
Neuro-Symbolic Procedural Planning with Commonsense Prompting
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought
PINTO: Faithful Language Reasoning Using Prompt-Generated Rationales
Decomposed Prompting: A Modular Approach for Solving Complex Tasks
Complexity-Based Prompting for Multi-step Reasoning
Automatic Chain of Thought Prompting in Large Language Models
Compositional Semantic Parsing with Large Language Models
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning
Iteratively Prompt Pre-trained Language Models for Chain of Thought
ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering
Induced Natural Language Rationales and Interleaved Markup Tokens Enable Extrapolation in Large Language Models
Compression [compression]
Understanding and Improving Knowledge Distillation for Quantization Aware Training of Large Transformer Encoders
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models
Others【Other】
BBTv2: Towards a Gradient-Free Future with Large Language Models
Compositional Task Representations for Large Language Models
Just Fine-tune Twice: Selective Differential Privacy for Large Language Models
Application [application]
Multi-Modal【Multi-modal】
Visual Classification via Description from Large Language Models
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language
Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training
Code [code]
DocPrompting: Generating Code by Retrieving the Docs
Planning with Large Language Models for Code Generation
CodeT: Code Generation with Generated Tests
Language Models Can Teach Themselves to Program Better
Retrieval [retrieval]
Promptagator: Few-shot Dense Retrieval From 8 Examples
Recitation-Augmented Language Models
Generate rather than Retrieve: Large Language Models are Strong Context Generators
QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillation
Text Generation [text generation]
Generating Sequences by Learning to Self-Correct
RankGen: Improving Text Generation with Large Ranking Models
Eliciting Knowledge from Large Pre-Trained Models for Unsupervised Knowledge-Grounded Conversation
Others【Other】
Systematic Rectification of Language Models via Dead-end Analysis
Reward Design with Language Models
Bidirectional Language Models Are Also Few-shot Learners
Composing Ensembles of Pre-trained Models via Iterative Consensus
Binding Language Models in Symbolic Languages
Mind's Eye: Grounded Language Model Reasoning through Simulation
Analysis & Evaluation【Analysis and Evaluation】
WikiWhy: Answering and Explaining Cause-and-Effect Questions
ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning
Quantifying Memorization Across Neural Language Models
Mass-Editing Memory in a Transformer
Multi-lingual Evaluation of Code Generation Models
STREET: A MULTI-TASK STRUCTURED REASONING AND EXPLANATION BENCHMARK
Leveraging Large Language Models for Multiple Choice Question Answering
Broken Neural Scaling Laws
Language models are multilingual chain-of-thought reasoners
Language Models are Realistic Tabular Data Generators
Task Ambiguity in Humans and Language Models
Discovering Latent Knowledge in Language Models Without Supervision
Prompting GPT-3 To Be Reliable
Large language models are few-shot clinical information extractors
How Large Language Models are Transforming Machine-Paraphrase Plagiarism
Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs
SLING: Sino Linguistic Evaluation of Large Language Models
A Systematic Investigation of Commonsense Knowledge in Large Language Models
Lexical Generalization Improves with Larger Models and Longer Training
What do Large Language Models Learn beyond Language?
Probing for Understanding of English Verb Classes and Alternations in Large Pre-trained Language Models
Enter the NLP group —> join the NLP exchange group (remark nips/emnlp/nlpcc enters the corresponding contribution group)
Continue to release the latest information such as interpretation of natural language processing NLP daily high-quality papers, relevant first-hand information, AI algorithm positions, etc.
Join the planet, you will get:
1. Update 3-5 latest and high-quality paper speed readings every day
2. The latest introductory and advanced learning materials
4. Daily 1-3 recruitment information for AI positions such as NLP, search, promotion and promotion, and CV
