Tencent seniors stayed up late for a month to sort out the "Linux Kernel Study Notes", which benefited a lot and did not take detours

The editor is keen to collect and organize resources, and record the process from stepping on the pit to climbing the pit. I hope that I can record what I have learned, the techniques used in actual work, learning methods, experience, and some pitfalls I have stepped on. I also hope that you, who want to be a senior engineer of the Linux kernel, can avoid some detours through my sharing, and can form a set of own methods and apply them in practice.

1. The components of the Linux kernel

1. Understand the basic concepts of the operating system. If not, you can study "Operating System: Design and Implementation", the book written by Andrew S. Tanenbaum, using MINIX as an example to explain the concept of the operating system. Very recommended.

2. With the basic concept of the operating system, you can understand the mechanism of Linux. Recommend Linux Kernel Design and Implementation written by Robert Love. This book conceptually explains what Linux is and how it works. This book should be read repeatedly.

3. With the knowledge of the Linux kernel, we also need to learn the source code of the Linux kernel in detail. The classic is "In-depth Understanding of the Linux Kernel" written by Daniel P. Bovet. When studying this book, look at the kernel code. The book is a bit of a learning curve, so there's a lot of code to dig through. However, if this book is easy to understand, then congratulations, you are already familiar with the Linux kernel.

4. If you want to develop device drivers, you can learn linux device drivers from O'Reilly Press. This book is good material for getting started with driving. There is also a very good textbook, proficient in Linux driver development, you can refer to it. Driving, it is inevitable to learn some hardware protocols and materials. If you are researching which one, you can find the corresponding hardware documentation to understand how the hardware works. I won't go into details on these.

5. For the network part, learn some Linux network part and learn "In-depth Understanding of LINUX Network Technology Insider". This book explains the network part of Linux very clearly and thoroughly. However, we generally do not do research in this area, nor do we need to do so much research. After all, there are very few relevant jobs now.

6. Most of the work related to Linux is concentrated in some embedded development fields, such as arm, mips, etc. You will learn the following information about the architecture to understand the design and working mode of the CPU. Just look at the chip manual corresponding to ARM, which is very detailed. mips Just look at MIPS running, there is a version or two. There are some differences between the two versions, it is recommended to read them all.

7. Add a little experience. Don't think that Linux is difficult to learn because it is huge and complex. If you study hard, you can learn anything. It depends on your perseverance and perseverance. In addition, don't take detours, don't read those books on Linux 0.11 on the market, just learn what you want to learn. Just like learning C language to watch Tan Haoqiang, detours and effort will seriously affect the learning effect.

Regarding the Linux kernel learning route, say a few more words about application programming, sometimes often need:

1. Learn Linux application programming. It is recommended to look at advanced programming in the Unix environment. If you have done all the examples in it, you will have a systematic understanding of the entire Linux programming.

2. For Linux, there is a Linux system programming. After learning the previous one, this one will be clear at a glance. It is mainly for Linux to understand some specific content, which is relatively complete and practical.

3. Linux network programming, systematically study "Unix network programming. Volume 1, socket network api". Basically all programs related to web applications are fine.

Second, how to learn the Linux kernel?

The knowledge of the core is like the knots below, tied one by one. To solve them, we must first find the thread head, which is the function interface in the kernel. At the beginning stage, it is generally not necessary to study the kernel code in depth, just use the interface function of the kernel.

Here's how to learn these kernel functions, like untying a rope:
insert image description here

Before we learn the Linux kernel, we first need to master the following points:

(1) How to learn the kernel, first understand what is the composition of the Linux kernel?

(2) Need to know the organizational structure of the Linux kernel source code (download link)?

(3) What are the key knowledge points that need to be learned?

(4) Finally, start our Linux kernel learning journey based on the learning materials I provided for you.

(5) Detailed explanation of the most powerful Linux kernel Makefile system file in the whole network (plain text code)

(6) The most detailed analysis of Intel CPU architecture in the whole network (kernel source code)

(7) In-depth understanding of the overall architecture of the Linux Kernel (detailed graphic explanation)

(8) QEMU debugging Linux kernel environment construction

(9) Netizens said that the Linux driver is not thorough, so this is the Linux driver

(10) This article gives you a deep understanding of the Linux kernel architecture and working principle

(11) Look at the essence of the underlying socket from the Linux kernel (IO)

(12) Communication between Linux user space and kernel space (Netlink communication mechanism)

(13) Linux kernel memory related issues, this article will give you a thorough understanding

(14) One article to understand the hierarchical structure of the CPU, this article is enough

Two, thematic module learning

2.1 Operating system principle

Linux operating system learning - start

Linux operating system learning - kernel operation

Linux operating system learning - kernel initialization

Operating system principle (1): operating system principle and overview (flow chart)

Operating system principle (2): common commands based on Linux operating system

Operating system principle (3): Linux operating system I/O mechanism principle (detailed flow chart)

Operating system principle (four): memory management RAID disk array and configuration

Principles of Operating System (5): Principles of Disk Cache Mechanism for Memory Management

Principles of Operating System (6): Page, Segment, and Segment Page Storage in Storage Management

Principle of System Operation (7): State and transition of process (five-state model)

Operating system principles (eight): several methods and basic principles of process synchronization

Principles of Operating System (9): Basic Principles and Implementation Principles of Processor Scheduling

System operation principle (10): multi-process, multi-thread, deadlock problem in concurrent execution

Principles of System Operation (11): Operating System Principles: Several Methods and Basic Principles of Process Synchronization

Principles of system operation (12): Interesting talk about operating system principles, page, segment, and segment page storage of storage management

Principle of System Operation (13): Operating System: Understanding CPU Context Switching Through Actual Combat

Principles of system operation (14): Segmentation mechanism of the operating system - paging implementation principle

Assembly language basics (1): Basic knowledge of assembly language (graphic code)

Introduction to assembly language (2): entry-level arrangement of assembly instructions, which you must know

Assembly language instructions (3): summary of all instructions in assembly language, one article is enough

Advanced Assembly Language (4): Principle and Implementation of ARM Architecture Processor Mechanism

Advanced assembly language (5): ARM instruction set and assembly language programming

2.2 Memory management topic

What is memory? This article lets you understand how memory is implemented

Essential skills for embedded development, Linux kernel source code organization structure

One article to understand Linux memory management, malloc, free implementation principle

Memory management series (1): Linux operating system memory management (detailed explanation of mind map)

Memory Management Series (2): Summary of Linux Memory Management Principles

Memory Management Series (3): After learning the operating system memory management, can you answer these 8 questions?

Memory Management Series (4): Understanding Memory barrier (memory barrier)

Memory Management Series (5): Principle of LRU Linked List Mechanism for Memory Reclamation

Memory Management Series (6): Principles of Virtual Memory and Physical Memory Mechanism

Memory Management Series (7): Summary of different situations of Malloc page fault interruption and reverse mapping RMAP

Memory Management Series (8): Implementation Principle of Malloc Function in C/C++ Development

Memory Management Series (9): In-depth understanding of glibc malloc: the principle of memory allocator implementation

Memory Management Series (10): How does the operating system manage memory, and the relationship between memory and CPU

Memory Management Series (11): Why does Linux need virtual memory, and what effect does virtual memory have on the operating system

Memory Management Series (12): Benefits of the user-mode memory memory mapping function Mmap

Memory Management Series (13): Memory Management: Detailed Explanation of Virtual Address Space-MMU

Memory Management Series (14): How does Malloc/free in C language allocate memory

Memory Management Series (15): From virtual addressing to open source projects, detailed explanation of memory management under Linux

Memory Management Series (16): This article will take you to understand, virtual memory, memory paging, segmentation, segment page memory management

Memory Management Series (Seventeen): How Much Memory Does a Linux Application Consume?

Memory management series (eighteen): virtual address to physical address, when to start mapping

Memory Management Series (19): Analysis of SLAB Allocator in Linux Memory Management (Source Code Analysis)

Memory Management Series (20): Memory Allocation Based on Linux Memory Management (Partner Algorithm and Slab Algorithm)

Memory Management Series (21): Exploring Memory Mapping Files of Memory Principles (Detailed Graphics and Texts)

Memory Management Series (22): Hanging Byte Interviewer, CPU Cache Consistency Protocol MESI

Memory Management Series (23): In-depth understanding of the principle of the Linux kernel page table mapping paging mechanism

Memory Management Series (24): Talk about the mapping between physical memory and virtual memory (super detailed~)

Memory Management Series (25): Memory Management: Implementation Principle of malloc Function in C/C++ Development

Memory Management Series (26): Stay up all night and overturn all knowledge points of Linux memory management (graphic)

Memory Management Series (27): This article lets you know about Linux-based processor cache

Memory Management Series (28): Spinlock: Deadlock Mechanism in Linux Kernel

Memory Management Series (29): Memory Management: Deep Understanding of ARMv8 Exception Handling

Memory Management Series (Thirty): Von Neumann: Some Thoughts on Memory

Memory Management Series (31): Do you really know what a system call is? This article lets you play with it

Memory Management Series (32): Enjoy reading, advance technology, explain the principle of mmap in detail

Memory management series (33): Linux lock implementation, futex_wait/wake kernel implementation general logic

Memory Management Series (34): Read and understand the usage of select, poll, epoll in one article

Memory Management Series (35): In-depth Linux Memory Principles Pages, Page Tables and Page Frames (Blocks)

2.3 Process management topic

Process management series (1): Detailed explanation of Linux process management principles (code demonstration)

Process Management Series (2): Ten minutes to let you quickly understand the process status like a boss (two models)

Process Management Series (3): As an Internet programmer, should I understand the six states of Linux processes?

Process Management Series (4): Five minutes to let you quickly understand Linux process management, real-time scheduling and SMP

Process Management Series (6): Analysis of Linux Process Priority (Code Demonstration)

Process Management Series (7): Process Management|Analysis of the relationship between concurrent synchronization, atomic operations and locks in C language

Process Management Series (8): Process Management | In-depth understanding of Linux process descriptors and process status

Process Management Series (9): An article to understand the scheduling strategy between tasks in the Linux kernel

Process Management Series (10): Creation and Derivation of Processes and Threads in the Linux Kernel

Process Management Series (11): There are several process states based on Linux

Process Management Series (12): Several CPU Scheduling Strategies of the Operating System

Process Management Series (12): Scheduling and Process Switching of Linux Process Management

Process Management Series (13): One article to understand the principles of the six major process communication mechanisms (the most detailed in the entire network)

Process Management Series (14): Super-detailed Socket Communication Principles and Examples

Process Management Series (15): This is a very complete IO basic knowledge and concepts

Process Management Series (Sixteen): In-depth understanding of the management and scheduling of Linux kernel processes (Quanzhihu is the most detailed)

Process Management Series (Seventeen): In-depth understanding of process control in Linux (super detailed~)

Process Management Series (18): From process to thread - task_struct structure analysis in Linux

Process Management Series (19): Do you really understand the Linux interrupt mechanism?

Process Management Series (20): This article gives you a thorough understanding of Linux kernel multithreading (mutex locks, condition variables, read-write locks, spin locks, semaphores)

Process Management Series (21): In-depth understanding of the principles of the Linux kernel signal processing mechanism (including source code explanation)

Process Management Series (22): An article to understand the scheduling strategy between tasks in the Linux kernel

Process Management Series (23): Playing with Linux Kernel Process Scheduling, this one is enough (all knowledge points)

Process Management Series (24): One article to understand the basic principles of Linux kernel process CPU scheduling

Process Management Series (25): This article takes you to understand the principle of process and thread communication

Process Management Series (26): An article to understand the Socket programming principle of Linux (including example analysis)

2.4 Network protocol stack topics

Interesting talk about network protocol stack (1): socket buffer principle

Interesting talk about the network protocol stack (2): how the data packet is processed

Interesting talk about network protocol stack (3): three-layer data communication under the seven-layer model

Interesting talk about the network protocol stack (4): the structure of the transmitted Arp message

Interesting talk about the network protocol stack (5): the principle and code implementation of commonly used functions in Socket programming

Interesting talk about network protocol stack (6): learning the kernel implementation of select and poll functions

Interesting talk about network protocol stack (7): Epoll process analysis from user state to kernel state

Interesting talk about network protocol stack (eight): the process of socket sending network data

Interesting talk about network protocol stack (9): In-depth understanding of network protocols, taking you to analyze HTTP step by step

Interesting talk about the network protocol stack (10): Talking about the implementation of Linux internal network, the key data structure sk_buff

2.5 Device Driver Topics

Talking about device drivers (1): Detailed explanation of operating system I/O process

Talking about device drivers (2): Character devices for Linux operating system learning

Talking about Device Drivers (3): Combined with the collection of device information, explore how devices and drivers are bound

Talking about device drivers (4): An article to understand Linux kernel network device drivers (little knowledge for white prostitutes~)

Talking about Device Drivers (5): Device Management: Detailed Explanation of Operating System I/O Process

Talking about device drivers (6): Detailed explanation of Linux kernel device drivers, block device concepts and framework principles

2.6 File system

Understanding the file system (1): This article gives you a thorough understanding of the Linux kernel file system (big summary)

Understanding the file system (2) Implementation of the DMA mechanism in the Linux network subsystem (super detailed!!)

Understanding the file system (3): Linux kernel startup and file system loading process

Understanding the file system (4): illustrations | virtual network cards commonly used in Linux

2.7 Performance optimization

Performance tool use (1): Linux performance tool ftrace use

Use of performance tools (2): Linux performance tool ftrace basics

The use of performance tools (3): the core technology of GDB debugger - ptrace system call and usage examples

Performance tool use (4): Linux performance tool ftrace framework

Performance Optimization (1): A Panoramic Guide to Linux Performance Optimization

Performance optimization (2): Talking about the real-time optimization of the Linux kernel

2.8 Interview questions/experience

Talk about the learning route of the Linux kernel, how to learn specifically?

Embedded development wants to enter a major Internet company in 2022. Are you skilled?

Embedded Linux kernel learning experience summary, an article for you to master the method

Inventory of the interview questions that Linux kernel (driver development, embedded, kernel crowd) must ask

2022 Spring Recruitment Big Factory-Completion of Classic Written Exam Interview Questions for Embedded Development

Embedded software interviews, the key points and difficulties are all covered

2.9 Kernel Books

"In-depth understanding of the Linux kernel"
"This is how Linux should be learned"
"Linux Kernel Complete Notes V3.0 Bookmark Edition"
"Linux Command Line Encyclopedia - Shotts (william E.shotts)"
"Linux Command Quick Reference Manual"
"Linux Performance Optimization Master"
"Linux Environment Programming: From Application to Kernel"
"Linux cluster and automated operation and maintenance Yu Hongchun"
"Linux Driver Development Example (Second Edition)"
"Linux Advanced Programming (3rd Edition)"
"Building Highly Available Linux Servers (4th Edition)"

2.10 Embedded Knowledge

Basic knowledge for beginners (1): The general process of embedded Linux development, allowing you to quickly understand Linux

Basic knowledge for beginners (2): After using Linux for so long, I only know these concepts

Basic knowledge for beginners (3): Understand in one article | The ins and outs of Linux drivers

Basic knowledge for beginners (4): Inventory of the interview questions that must be asked in the Linux kernel (driver development, embedded, kernel crowd)

Basic knowledge for beginners (5): seven steps recommended for embedded Linux learning route

Video recommendation: Embedded career change landed successfully! Share 138 episodes of the Linux kernel tutorial that I bought for more than 10,000 yuan for free~ Thank you for taking it away! Learn not to quit the IT world!

Three, kernel learning route

Here I recommend the exclusive [Linux Kernel Source Code] course system of ZeroSound Education. Through 20 project cases and 2W+ lines of handwritten code, it comprehensively analyzes 6 kernel technology directions.

Systematically and comprehensively explain the underlying principles and development technologies in all aspects:

The learning route mainly starts from

Process Management Topics
Memory Management Topics
Network Protocol Stack Topics
Device Driver Topics
Kernel Component Topics
File System Topics
Kernel project practical topics
and so on. . .

These contents, if you plan to learn step by step in a few years, you will become a Linux master. It is recommended that individuals participate in the training of Zero Sound Education, and the learning efficiency will be much higher. Purposeful participation in training, shortening the cycle, and rapid prototyping are the needs of the times.

Free information through train : the latest Linux kernel source code information document + video information

Kernel learning address : Linux kernel source code/memory tuning/file system/process management/device driver/network protocol stack