Linux调试之core dump - (中篇）

更加现代的crash报告系统

一、systemd-coredump

systemd-coredump 可收集并显示内核核心转储，用于分析应用程序崩溃问题， 当进程发生crash时，能自动收集并保存相应的core dump, 同时可以用客户端工具coredumpctl来进行core dump管理，查看和分析。

• 安装systemd-coredump
  sudo apt install systemd-coredump
  • core dumps存储在/var/lib/systemd/coredump
  • 自动安装coredumpctl工具
• 配置
  • /etc/systemd/coredump.conf
    • systemd-coredump will automatically delete core dumps that exceed configurable size limits (2 GB by default).
    • It also deletes core dumps if your free disk space falls below a configurable threshold (15% free by default).
    • Additionally, systemd-tmpfiles will delete core dumps automatically after some time has passed (three days by default).

[Coredump]
#Storage=external
#Compress=yes
#ProcessSizeMax=2G
#ExternalSizeMax=2G
#JournalSizeMax=767M
#MaxUse=
#KeepFree=

• 客户端工具coredumpctl常用命令
  • coredumpctl 【列出所有core dumps】
  • coredumpctl gdb 【打开最近的core dumps】
  • coredumpctl gdb 1234【打开pid=1234进程最近的core dumps】

二、举例

• 测试环境
  • ubuntu 18.04
  • g++ 7.5.0
• 测试代码 【segfault.cpp】

#include <iostream>

void test_crash(int a, int b)
{
	std::cout << __func__ << " enter" << std::endl;
	int *ptr = nullptr;
	*ptr = 0;
	return ;
}

int main(void)
{
	std::cout << "hello crash" << std::endl;
	test_crash(0, 0);

	return 0;
}

• 编译命令
  g++ -g segfault.cpp

• 运行
  ./a.out

• coredumpctl gdb 查看崩溃stacktrace

PID: 27711 (a.out)
           UID: 1000 (ubuntu)
           GID: 1000 (ubuntu)
        Signal: 11 (SEGV)
     Timestamp: Mon 2022-08-22 19:06:31 CST (20min ago)
  Command Line: ./a.out
    Executable: /home/ubuntu/work/Linux_4.6.131/lava/sdk/a.out
 Control Group: /user.slice/user-1000.slice/session-1236.scope
          Unit: session-1236.scope
         Slice: user-1000.slice
       Session: 1236
     Owner UID: 1000 (ubuntu)
       Boot ID: 6488216f2d4c465eaff663978d90786a
    Machine ID: 350062f635cd4e9a95051d7f498230c9
      Hostname: ubuntu-Precision-3630-Tower
       Storage: /var/lib/systemd/coredump/core.a\x2eout.1000.6488216f2d4c465eaff663978d90786a.27711.1661166391000000.lz4
       Message: Process 27711 (a.out) of user 1000 dumped core.

                Stack trace of thread 27711:
                #0  0x0000556a6dc0290b n/a (/home/ubuntu/work/Linux_4.6.131/lava/sdk/a.out)
                #1  0x0000556a6dc0294f n/a (/home/ubuntu/work/Linux_4.6.131/lava/sdk/a.out)
                #2  0x00007f254a935c87 __libc_start_main (libc.so.6)
                #3  0x0000556a6dc027da n/a (/home/ubuntu/work/Linux_4.6.131/lava/sdk/a.out)

GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/ubuntu/work/Linux_4.6.131/lava/sdk/a.out...done.
[New LWP 27711]
Core was generated by `./a.out'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000556a6dc0290b in test_crash (a=0, b=0) at segfault.cpp:7
7               *ptr = 0;
(gdb)

三、结论

• 不要再使用手动coredump， 应该使用systemd-coredump
  • 手动设置ulimit -c unlimited， 仅当前shell有效
  • 手动关闭apport （ubuntu）才能生成coredump

四、参考

1. Creating Quality Backtraces for Crash Reports
2.systemd-coredump, systemd-coredump.socket, [email protected] — Acquire, save and process core dumps