Linux-driven debugging technology reading notes (on)

Debug support in the kernel

2. Several options in the kernel

CONFIG_DEBUG_KERNEL

This option is only available for other debugging options. We should turn on this option, but it does not turn on all debugging functions.

CONFIG_DEBUG_SLAB

This is a very important option. It turns on multiple type checks in the kernel memory allocation function; after turning on the check, you can check many memory overflows
and forget to initialize errors. Before returning the allocated memory to the caller, the kernel sets each byte of it to 0xa5, and after releasing it,
it sets it to 0x6b. If the reader sees the poison string in his driver or in the oops information, he can easily determine the location.
After turning on this debugging option, the kernel will add protection values ​​before and after the object allocation, so that when the protection value changes, the kernel will know
which codes are beyond the normal access range of memory

CONFIG_DEBUG_PAGEALLOC

Upon release, all memory pages will be moved out of the kernel address space. This option will greatly reduce operating efficiency, but it can locate the location of memory corruption.

CONFIG_DEBUG_SPINLOCK

Turn on this option, the kernel will catch the uninitialized spin lock, and also the error of unlocking the same lock twice.

CONFIG_DEBUG_SPINLOCK_SLEEP

This option will have a sleep attempt during spinlock.

The symbols marked as __init in CONFIG_INIT_DEBUG will be discarded after system initialization or module uninstallation. This option can be used to detect
attempts to access the memory space used for initialization after initialization is complete

CONFIG_DEBUG_INFO

This option allows the structure of the kernel to contain complete debugging information. If you use gdb to debug, you should turn on the CONFIG_FRAME_POINTER option.

CONFIG_INIT_SYSRQ

Turn on the sysRq magic button.

CONFIG_DEBUG_STACKOVERFLOW
CONFIG_DEBUG_STACK_USAGE

This option is used to help locate the kernel stack overflow problem.

CONFIG_KALLSYMS

This option appears in the general settings, standard function menu; this option is turned on by default. This symbol information is used to debug the context; without
this symbol, the oops list only gives the hexadecimal kernel backtracking information, which is usually not very useful.

CONFIG_IKCONFIG
CONFIG_IKCONFIG_PROC

These options appear in the general settings menu, which will allow the kernel configuration state to be included in the kernel and accessed through /proc. Most kernel developers
know the configuration they use, so these two options are not very necessary. If the kernel is built by someone else, I think you need them.

CONFIG_ACPI_DEBUG

This option appears in power management, ACPI. If you suspect that your problem is related to ACPI, turn on this option.

CONFIG_DEBUG_DRIVER
in the device driver menu, he will open the driver core debugging information.

CONFIG_SCSI_CONSTANTS

Open detailed SCSI error information, if writing SCSI driver, you can open this option.

CONFIG_INPUT_EVBUG

Open the detailed input event record of the input device. If the reader wants to write a driver for the input device, you can use this option.

CONFIG_PROFILING

Found in the profiling support menu, usually used to debug performance.

2. Debug by printing

The linux kernel can be printed by printk.

printk, you can print logs of different levels

printk(KERN_DEBUG "Here I am:%s:%i\n", __FILE__, __LINE__);
printk(KERN_CRIT "Here I am:%s:%i\n", __FILE__, __LINE__);

The linux kernel provides 8 levels of error notification:

KERN_EMERG

Emergency messages, usually system crashes

CORE_ALERT

Used in situations that require immediate action

KERN_CRIT

Approaching state, generally involving serious hardware or software failure

KERN_ERR

Used to report error status. Device drivers often use KERN_ERR to report hardware problems.

KERN_WARNING

Warns of possible problems, but such problems will not cause serious system problems

KERN_NOTICE

It is necessary to make a normal situation for prompting. Many safety-related conditions are reported at this level.

CORE_INFO

Informational information. Many drivers print out their hardware information at this level when starting

KERN_DEBUG

Debug information

For the above levels, the smaller the value, the higher the priority

The default level for printk statements that do not specify a priority is DEFAULT_MESSAGE_LOGLEVEL. This macro
is specified as an integer in kernel/printk.c . In the 2.6.10 kernel, DEFAULT_MESSAGE_LOGLEVEL is KERN_WARING, but previous versions
have also taken different values.

When the priority is less than the value of the integer variable console_loglevel, the message can be displayed, and only one line will be output at a time.
If the system is running kogd and syslogd, no matter what the value of console_loglevel is, the kernel will append the message to /var/log/message
. If klogd is not running, these messages will be appended to /var/log/message. If klogd is not running, these messages
will not be delivered to user space. In this case, you can only view the /proc/kmsg file. If you use klogd, you should understand that
it will not save consecutive identical information, only the first consecutive identical row, and mark the same number of repetitions at the end.

The initial value of the variable console_loglevel is DEFAULT_CONSOLE_LOGLEVEL, and it can also be modified through the sys_syslog system call. When calling klogd
, you can specify the -c switch to modify this variable. You must kill klogd after modifying the current value. If it is set to 8, then
all messages including debugging information will be displayed.

We can also read and modify the log level of the console by accessing the file /proc/sys/kernel/printk. This file contains
4 integer values, which are the current log level, the default message level when the log level is not specified, the minimum allowed log level, and the default log level at boot time.

Test Results:

zhanglei@zhanglei-Latitude-5400:~/ourc/test$ cat /proc/sys/kernel/printk
4	4	1	7
zhanglei@zhanglei-Latitude-5400:~/ourc/test$ 

3. Redirect console messages

For the console log policy, Linux allows some flexibility: the kernel can send messages to a designated virtual console. By default, the console is the current
virtual terminal. You can call ioctl on any console to specify other consoles that receive kernel messages. This program must be run by a super user and
can be found in mics-proc .

Below is a complete list of this program. When calling this program, please attach a console number.

setconsole uses a special ioctl command: TIOCLINUX, this command can complete some special linux functions.
When using TIOCLINUX, you need to pass it a pointer parameter to the byte array. The first byte of the array specifies the numbers of all requested subcommands, and
the function of the subsequent bytes is determined by this subcommand. The subcommand in setconsole is 11, and the latter byte is used to identify the virtual console.
The complete description of TIOCLINUX is identified in the drivers/char/tty_io.c file in the kernel source code.

#include <iostream>
#include <unistd.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <cstring>
#include <sys/ioctl.h>

int main (int argc, char** argv)
{
    char bytes[2] = {11, 0};
    if (argc == 2) {
        bytes[1] = atoi(argv[1]);
    } else {
        fprintf(stderr, "%s: need a single arg\n", argv[0]);
        exit(1);
    }

    if (ioctl(STDIN_FILENO, TIOCLINUX, bytes) < 0) {
        fprintf(stderr, "%s: ioctl(stdin, TIOCLINUX):%s\n", argv[0], strerror(errno));
        exit(1);
    }
    return 0;
}

I don’t understand what this function is for

4. How is the message recorded

The printk function writes the message to a circular buffer with a length of __LOG_BUF_LEN bytes. Then, this function will wake up any
processes that are waiting for messages, that is, processes that are waiting for syslogd messages, or processes that are reading /proc/kmsg. /proc/kmsg is a circular buffer, the
read content will not continue to be saved. Generally speaking, /proc is easier, which is also the default method of klogd. dmesg can get
the contents of the buffer without flushing the buffer

Readers will find that the /proc/kmsg file is very similar to a FIFO. If there is already a process reading it, you cannot use this method to read it, and race conditions will occur.

klogd specifies the -f option to save klogd to a file, but the klogd needs to be killed.

5. Turn messages on or off

You can enable or disable each print statement by deleting or adding a letter in the name of the macro.

#undef PDEBUG
#ifdef SCULL_DEBUG
# ifdef __KERNEL__
#  define PDEBUG(fmt, args...) printk(KERN_DEBUG "scull:" fmt, ##args)
# else
#  define PDEBUG(fmt, args...) fprintf(stderr, fmt, ## args)
# endif
#else
#  define PDEBUG(fmt, args...)
#endif

#undef PDEBUGG
#define PDEBUGG(fmt, args...)

6. Speed ​​limit

Printk may generate thousands of messages to be printed to the console, causing system files to overflow. Too high message printing will slow down the system or even fail to respond.
When our system is over, unwise processes will make the system equipment keep retrying.

So we can use

int printk_ratelimit(void);

Specific case:

if (printk_ratelimit())
    printk(KERN_NOTICE "The printer is still on file");

printk_ratelimit works by tracking the number of messages sent to the console. If the output speed exceeds a threshold, printk_ratelimit will return
to 0 to avoid sending messages repeatedly.

We can
customize the behavior of printk_ratelimit by modifying /proc/sys/kernel/printk_ratelimit and /proc/sys/kernel/printk_ratelimit_burst .

7. Print the device number

When printing a message from a driver, we hope to print the device number associated with the hardware. The kernel provides some auxiliary functions, which are placed in

<linux/kdev_t.h>

int print_dev_t(char* buffer, dev_t dev);
char *format_dev_t(char *buffer, dev_t dev);

print_dev_t returns the number of printed strings, while format_dev_t returns a buffer, so you can directly call printk to use it.
Because it is very obvious to use 64-bit device numbers in future kernel versions, the size of this buffer is 20 bytes long.

Code demo

8. Debugging by query

Drivers can use the following methods to query: create files in the file system /proc, use driver ioctl control, and export attributes through sysfs.

/proc file system

/proc is a special file system created by the soft link. The kernel uses it to export information. Each file of /proc is bound to a kernel function. When the user reads the
file, this function dynamically generates the file content. For example, /proc/modules lists the current modules.

We do not recommend adding too many files under /proc, but we can use sysfs to export information.

For files implemented in /proc, you can use <linux/proc_fs.h> and define the correct function through this function.

When a process reads the /proc file, the kernel allocates a memory page, and the driver can return the data to the user space through this memory page.
This function is called the read_proc method.

#include <linux/proc_fs.h>
int (*read_proc)(char* page, char **start, off_t offset, int count, int *eof, void *data);

Create your own /proc

struct proc_dir_entry *create_proc_read_entry(const char* name, mode_t mode, struct proc_dir_entry *base, 
read_proc_t *read_proc, void* data);

Among them, name is the name of the file to be created; mode is the protection mask of the file; base specifies the directory where the file is located; read_proc is the implementation function, the kernel will ignore the data parameter, but
will pass the parameter to read_proc. The following is the code of scull calling function to create /proc file.

create_proc_read_entry("scullmem", 0, NULL, scull_read_procmem, NULL);

This function has been abandoned, the new proc_create_data can continue to be used, and there are surprises, hahahaha, there is a structure for operation

struct proc_fs_info {
    
    
        struct pid_namespace *pid_ns;
        struct dentry *proc_self;        /* For /proc/self */
        struct dentry *proc_thread_self; /* For /proc/thread-self */
        kgid_t pid_gid;
        enum proc_hidepid hide_pid;
        enum proc_pidonly pidonly;
};
#define proc_create_data(name, mode, parent, proc_ops, data) ({NULL;})

When uninstalling the module, the entry in /proc should also be deleted

remove_proc_entry("scullmem", NULL);

The most important issue is related to the deletion of the /proc entry. The delete call may occur when the file is being used, because the /proc entry does not have an associated owner,
because the /proc entry does not have an associated owner, so the use of these files does not affect the module reference Count, execute
sleep 100 </proc/myfile

It’s a bit late today, come here first, I will follow up next week /proc