[linux] Introduction to debugging tools

---

foreword

During Linux kernel debugging, various tools and techniques are available to diagnose and resolve problems. The following are some commonly used Linux kernel debugging methods:

1. printk: printk is a printing function in the Linux kernel, which can insert print statements in the code to output debugging information. These messages will be sent to the kernel log buffer, which can be viewed using the dmesg command or the /var/log/messages file.

2. kdb: kdb is a debugger for the Linux kernel, which can debug the kernel at runtime. It provides a command-line interface to view and modify information such as the state of the kernel, register values, and stack traces.

3. kgdb: kgdb is a kernel-level source code debugger, which can connect the target machine with the debugging host through a serial port or a network connection. Through kgdb, you can set breakpoints on the target machine, single-step execution, view variable values, and more.

4. ftrace: ftrace is a tracing tool in the Linux kernel, which can be used to analyze the calling relationship and execution time of kernel functions. It can capture kernel execution information and output it to the kernel log buffer by configuring and enabling different trace events.

5. SystemTap: SystemTap is a dynamic tracing tool that can monitor and analyze the behavior of the kernel and applications by inserting probes at runtime. It uses a script-like language to describe tracking scripts and provides a rich set of APIs and tools to analyze tracking data.

6. GDB: GDB is a general-purpose source-level debugger that can be used to debug kernel modules and applications. By cross-compiling the kernel and debugging symbol table, you can use GDB on the development host to connect to the kernel on the target machine for debugging.

These are commonly used Linux kernel debugging methods, and each method has its own characteristics and applicable scenarios. According to the specific problems and needs, select the appropriate debugging method for kernel debugging. At the same time, multiple debugging tools and techniques can be combined to obtain more comprehensive debugging information and more efficient problem solving.

---

提示：以下是本篇文章正文内容，下面案例可供参考

1. kdb

kdb (Kernel Debugger) is a debugger for the Linux kernel that allows developers to debug the kernel at runtime. kdb provides a command-line interface to interact with the kernel and view and modify the kernel's state, register values, stack traces, and more. Here are some details about kdb:

1. Functions and Features:

   • Command-line interface: kdb provides a command-line-like interface where users can interact with the kernel by entering commands.
   • Real-time debugging: kdb can debug while the kernel is running, allowing users to perform real-time troubleshooting when there is a problem with the kernel.
   • Stack trace: kdb can display the current function call stack to help users locate the source of the problem.
   • Register viewing and modification: kdb allows users to view and modify register values ​​in the kernel for analysis and debugging of code.
   • Breakpoint setting: kdb supports setting breakpoints in the kernel code to stop execution at a specific location for further debugging.
   • Dynamic memory allocation tracking: kdb can track dynamic memory allocation and release operations in the kernel, helping users detect memory leaks and other memory-related problems.

2. Instructions:

   • Enable kdb: In the Linux kernel configuration, you need to enable the CONFIG_KDB option to compile the kernel to include kdb debugging support. After compiling, kdb can be enabled by adding "debug" to the kernel boot parameters.
   • Enter kdb: You can enter kdb by pressing the "Ctrl+Alt+SysRq+g" key combination on the console, or by connecting through the debugging serial port.
   • kdb command: After entering kdb, you can use various commands to view and modify the state of the kernel. For example, the "bt" command is used to display the current function call stack, the "regs" command is used to display register values, the "bp" command is used to set breakpoints, etc.

3. Precautions:

   • kdb is a powerful debugging tool, but it needs to be used with care. Incorrect operation may result in system crash or data corruption.
   • Kdb is usually used for kernel development and debugging. For ordinary users, using kdb for kernel debugging may require certain professional knowledge and experience.
   • In production environments, kdb is usually not enabled because it introduces additional overhead and security risks.

In short, kdb is a debugger for the Linux kernel, which provides a way to debug the kernel at runtime. It can help developers locate and solve problems in the kernel, improve debugging efficiency and code quality. However, using kdb requires caution and is typically used in development and debugging environments.

The following is a simple sample code that demonstrates how to use kdb for debugging in the Linux kernel:

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>

static int __init my_module_init(void)
{
    
    
    printk(KERN_INFO "My module is being loaded.\n");

    // 在这里插入一个断点
    kdb_trap();

    return 0;
}

static void __exit my_module_exit(void)
{
    
    
    printk(KERN_INFO "My module is being unloaded.\n");
}

module_init(my_module_init);
module_exit(my_module_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Your Name");
MODULE_DESCRIPTION("A simple example of using kdb for kernel debugging");

In the sample code above, we defined a simple kernel module. In the initialization function of the module my_module_init, we used printkthe function to output a debug message and inserted kdb_trap()the function in the code to set a breakpoint. In this way, when the module is loaded, when the code execution reaches the breakpoint, kdb will be triggered and enter the kdb debugging environment.

Note that in order to compile and load this module, you need to have a properly configured and compiled kernel and basic knowledge of kernel module development. This example is for demonstration purposes only, actual kernel debugging may involve more complex code and debugging scenarios.

Please ensure that proper kernel debugging practices and security measures are followed in actual use to avoid adverse effects on the system.

Two, ftrace

ftrace is a powerful tracing tool in the Linux kernel, which can be used to analyze and debug the execution path and performance bottleneck of the kernel. ftrace provides a lightweight trace framework that can insert trace points into the kernel and record related trace data. The following are some details about ftrace:

1. Functions and Features:

   • Dynamic Tracing: ftrace allows the dynamic insertion and removal of tracepoints at runtime in order to trace specific functions, events, or code paths in the kernel.
   • Function trace: ftrace can trace the function call path in the kernel, including the entry and exit of the function, to help analyze the execution flow of the code.
   • Event tracking: ftrace can track various events in the kernel, such as interrupts, context switches, timer triggers, etc., to help analyze the behavior and performance of the system.
   • Performance analysis: ftrace can record performance indicators such as function execution time and call times to help locate performance bottlenecks and optimize code.
   • Visualization tools: ftrace provides some visualization tools, such as trace-cmd and KernelShark, for visualizing and analyzing trace data.

2. Instructions:

   • Enable ftrace: In the Linux kernel configuration, you need to enable the CONFIG_FUNCTION_TRACER and CONFIG_DYNAMIC_FTRACE options to compile the kernel to include ftrace support.
   • Set trace points: You can use the interface provided by ftrace to insert trace points in the kernel. For example, you can tracepoint_probe_registerregister tracepoints with functions, or mark the entry and exit of functions with function_graph_enterand function_graph_exitmacros.
   • Running and collecting trace data: While the kernel is running, ftrace collects trace data and stores it in corresponding files in the tracefs file system. Trace data can be trace-cmdcollected and analyzed using the command line tool or the KernelShark visualization tool.

3. Precautions:

   • ftrace is a powerful tool, but it needs to be used with care. Tracing too many events or functions may incur performance overhead and may affect system stability.
   • In production environments, ftrace is usually disabled by default because it introduces additional overhead. Therefore, kernel debugging using ftrace is usually performed in a development and debugging environment.

In short, ftrace is a powerful tracing tool in the Linux kernel, which can be used to analyze and debug the execution path and performance bottleneck of the kernel. It provides a lightweight tracking framework that can be used to dynamically insert and delete tracking points and record related tracking data. Using ftrace can help developers gain insight into the behavior and performance of the kernel and optimize code.

---

Three, gdb

Debugging with GDB (GNU Debugger) is a common method used to debug C and C++ programs. Here are the general steps for debugging with GDB:

1. Compile the program to be debuggable: When compiling the program, make sure to use the debug info option. For example, with the GCC compiler, -goptions can be used to generate debug information. For example:gcc -g -o my_program my_program.c

2. Start GDB: Enter the command on the command line gdbfollowed by the path to the executable. For example:gdb my_program

3. Set Breakpoints: Use breakcommands to set breakpoints in the program. Breakpoints can be set on function names, line numbers, or addresses. For example: break main, break 15,break *0x4005f6

4. Run the program: Use runthe command to run the program. Command line arguments can be passed at runtime. For example:run arg1 arg2

5. Execute the program: the program will stop at the breakpoint. nextExecute the program line by line using commands. You can use stepthe command to step inside the function. For example: next,step

6. View Variables: Use printcommands to view the values ​​of variables. Can print local variables, global variables, expressions, etc. For example: print variable, print array[2],print expression

7. Modify variable: Use setcommands to modify the value of a variable. For example:set variable = value

8. Continue Execution: Use continuethe command to continue program execution until the next breakpoint or the end of the program.

9. View Stack: Use backtracethe command to view the function call stack. Commands can be used frameto switch to a specific stack frame.

10. Exit GDB: Use quitthe command to exit GDB.

This is just the basic usage of GDB, GDB provides a wealth of functions and commands for deeper debugging and analysis. You can use helpcommands to get more command help and documentation in GDB.

It should be noted that GDB is a powerful tool and needs to be used with care. During the debugging process, try to avoid unpredictable effects on the running of the program.

Ok, let's take the code of the Linux kernel as an example for analysis.

Suppose we have a my_driver.cdriver file named , which is a simple character device driver. We want to use GDB to debug this driver.

First, make sure you have compiled the driver with the debug info option. For example, compile the driver with the following command:

gcc -g -o my_driver my_driver.c

Next, start GDB and load the driver:

gdb my_driver

Then, we can set breakpoints. Suppose we want to my_driver_openset a breakpoint in a function. The following commands can be used:

break my_driver_open

Next, run the program:

run

The program my_driver_openstops at the function. Now, we can use various commands of GDB for debugging.

For example, nextto execute a program line by line using the command:

next

Use printthe command to view the value of a variable. Suppose we want to view devthe value of a variable:

print dev

Use stepthe command to enter the function:

step

Use continuethe command to continue program execution until the next breakpoint or the end of the program:

continue

Use backtracethe command to view the function call stack:

backtrace

Exit GDB with quitthe command:

quit

These are just some basic command examples of GDB, you can use other commands and functions for deeper debugging and analysis as needed.

It should be noted that the Linux kernel is a complex code base, and debugging kernel code requires a special environment and skills. Typically, it is better to debug and analyze the Linux kernel using a kernel debugger such as kdb or a tracing tool such as ftrace.