With the explosion of ChatGPT, the large language model (LLM) has received unprecedented attention. According to the "2023Q1 Employment Trend Big Data Report", the number of jobs in big language models increased by 172.53% year-on-year , which is the largest increase in jobs in all industries. Practitioners in the field of NLP and intelligent speech are moving closer to large language models.
So, what skills are needed to build your own ChatGPT from 0 to 1? In summary, the main points are as follows:
(1) Transformer and RLHF
Transformer is the underlying core architecture of ChatGPT, so you need to have a deep understanding of the details of Transformer (including Seq2Seq architecture, Attention mechanism); you also need to master the method of model fine-tuning and optimization-human feedback reinforcement learning RLHF model.
(2) Engineering skills
ChatGPT is a project that requires complex engineering capabilities such as data preprocessing, model training, and tuning.
Deep Blue Academy combines the principles of transformer and RLHF with engineering practice, and launched the " Generative Pre-Training Language Model: Theory and Practice " course. Starting from the classic language model, we will gradually go deep into the GPT model, disassemble the core module of GPT in detail, and finally lead everyone to realize their own mini-ChatGPT . While explaining the principles, the course pays great attention to code practice. From data preprocessing to model training and tuning, the code implementation of 10 practical projects is interspersed with each algorithm theory.
Deep Blue Academy mini-ChatGPT project introduction
Scan the QR code to learn more
Remark 【615 】 , priority pass!
course tutor
Huang Jia
A senior researcher of artificial intelligence at the Singapore Agency for Science, Technology and Research. His main research direction is the development and application of large NLP models, continuous learning, AI in FinTech, and AI in Spectrometry Data. He has written many best-selling books such as "Basic Machine Learning" and "Ten Chapters of Data Analysis", and has been deeply involved in the field of data science for many years. He has accumulated a wealth of scientific research projects and the implementation of AI projects in the fields of government, banking, energy, and medical care. Practical experience.
Course Outline
(Click for larger image)
Course objectives
This course adopts a thorough understanding of the principle and code implementation of the generative language model represented by ChatGPT, and independently develops its own ChatGPT.
(1) Master the classic and mainstream algorithms of language models, and their development context;
(2) Deep understanding of several core technologies of ChatGPT;
(3) Implement the mini version of ChatGPT by hand.
Harvest
1. You will get the key core and complete context of modern NLP technology (abandon all outdated things that do not need to be understood too much)
2. You will build your own simplified version of ChatGPT (generative language model)
3. You will master the basic programming skills in the NLP field and the main content of PyTorch
4. You will master the core ideas and code implementation of attention mechanism and Transformer architecture
course service
1. Assisted by three divisions
Lecturers & teaching assistants answer questions in a timely manner, and the head teacher leads the class to supervise the whole process to help you overcome procrastination and make continuous progress.
2. Regular class meeting
Teaching assistants correct homework 1 on 1, and comment and guide in class meetings; learn more skills in class meetings; gain more ideas in exchanges.
consult more
Scan the QR code to learn more
Remark 【615 】 , only through friends!
