1 Introduction
strace to trace system calls and signals, for the analysis of process status is very helpful.
The easiest way is to use a direct way strace plus command execution, command execution can see all system calls, parameters, return values, etc. information.
For example cat file
...
open("/dev/null", O_RDONLY) = 3
fstat(3, {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 3), ...}) = 0
read(3, "", 32768) = 0
close(3) = 0
close(1) = 0
close(2) = 0
exit_group(0) = ?
cat a file that does not exist
...
open("/dev/foo", O_RDONLY) = -1 ENOENT (No such file or directory)
...
write(2, ": No such file or directory", 27: No such file or directory) = 27
write(2, "\n", 1
) = 1
close(1) = 0
close(2) = 0
Each row is a system call, the system calls the left side of the equal sign is the function name and its parameters, the right is the return value of the call. strace display these calls and returns the value in symbolic form. strace kernel receives information from, and does not require any special way to build the kernel.
2 parameters
-c 统计每一系统调用的所执行的时间,次数和出错的次数等.
-d 输出strace关于标准错误的调试信息.
-f 跟踪由fork调用所产生的子进程.
-ff 如果提供-o filename,则所有进程的跟踪结果输出到相应的filename.pid中,pid是各进程的进程号.
-F 尝试跟踪vfork调用.在-f时,vfork不被跟踪.
-h 输出简要的帮助信息.
-i 输出系统调用的入口指针.
-q 禁止输出关于脱离的消息.
-r 打印出相对时间关于,,每一个系统调用.
-t 在输出中的每一行前加上时间信息.
-tt 在输出中的每一行前加上时间信息,微秒级.
-ttt 微秒级输出,以秒了表示时间.
-T 显示每一调用所耗的时间.
-v 输出所有的系统调用.一些调用关于环境变量,状态,输入输出等调用由于使用频繁,默认不输出.
-V 输出strace的版本信息.
-x 以十六进制形式输出非标准字符串
-xx 所有字符串以十六进制形式输出.
-a column
设置返回值的输出位置.默认 为40.
-e expr
指定一个表达式,用来控制如何跟踪.格式如下:
[qualifier=][!]value1[,value2]...
qualifier只能是 trace,abbrev,verbose,raw,signal,read,write其中之一.value是用来限定的符号或数字.默认的 qualifier是 trace.感叹号是否定符号.例如:
-eopen等价于 -e trace=open,表示只跟踪open调用.而-etrace!=open表示跟踪除了open以外的其他调用.有两个特殊的符号 all 和 none.
注意有些shell使用!来执行历史记录里的命令,所以要使用\\.
-e trace=set
只跟踪指定的系统 调用.例如:-e trace=open,close,rean,write表示只跟踪这四个系统调用.默认的为set=all.
-e trace=file
只跟踪有关文件操作的系统调用.
-e trace=process
只跟踪有关进程控制的系统调用.
-e trace=network
跟踪与网络有关的所有系统调用.
-e strace=signal
跟踪所有与系统信号有关的 系统调用
-e trace=ipc
跟踪所有与进程通讯有关的系统调用
-e abbrev=set
设定 strace输出的系统调用的结果集.-v 等与 abbrev=none.默认为abbrev=all.
-e raw=set
将指 定的系统调用的参数以十六进制显示.
-e signal=set
指定跟踪的系统信号.默认为all.如 signal=!SIGIO(或者signal=!io),表示不跟踪SIGIO信号.
-e read=set
输出从指定文件中读出 的数据.例如:
-e read=3,5
-e write=set
输出写入到指定文件中的数据.
-o filename
将strace的输出写入文件filename
-p pid
跟踪指定的进程pid.
-s strsize
指定输出的字符串的最大长度.默认为32.文件名一直全部输出.
-u username
以username 的UID和GID执行被跟踪的命令
3 Case
Mounting 3.1 pg background service process
Look at the system call table creation process postgresql
1326787 39213 0.0 0.0 357072 59172 ? S Apr28 0:00 /disk1/p01/pgsql8400/bin/postgres
1326787 39214 0.0 0.0 145208 1272 ? Ss Apr28 0:00 \_ postgres: logger process
1326787 39216 0.0 0.0 357732 140704 ? Ss Apr28 1:15 \_ postgres: checkpointer process
1326787 39217 0.0 0.0 357616 140444 ? Ss Apr28 0:04 \_ postgres: writer process
1326787 39218 0.0 0.0 357072 5688 ? Ss Apr28 0:31 \_ postgres: wal writer process
1326787 39219 0.0 0.0 145204 1440 ? Ss Apr28 0:06 \_ postgres: archiver process last was 000000010000001200000038
1326787 39220 0.0 0.0 145468 1528 ? Ss Apr28 0:00 \_ postgres: stats collector process
1326787 25516 2.0 0.0 359904 103772 ? Ss 16:38 0:04 \_ postgres: postgres postgres [local] idle
strace hanging in the 25516, which is pg background service process.
strace -T -tt -e trace=all -p 25516
# 创建表
16:42:27.255315 open("base/13003/32849", O_RDONLY) = -1 ENOENT (No such file or directory) <0.000023>
...
16:42:27.255547 open("base/13003/32849", O_RDWR|O_CREAT|O_EXCL, 0600) = 57 <0.000032>
...
3.2 Mount the returned data to verify the time
Reference Example we mount the server processes the PG, the execution end psql pg_sleep (10), the return time is determined by observing the SQL strace.
The client execution select pg_sleep(10);, you can see by PG SQL message format is received by the server process to ( "Q\0\0\0\31select pg_sleep(10);\0"). call to sleep poll system call goes to sleep.
"Q\0\0\0\31select pg_sleep(10);\0", 8192, 0, NULL, NULL) = 26 <123.716984>
16:54:38.066795 write(2, "\0\0c\0\227|\0\0t31895 [local] postgres "..., 108) = 108 <0.000012>
16:54:38.066926 read(3, 0x7fff093e8de0, 16) = -1 EAGAIN (Resource temporarily unavailable) <0.000008>
16:54:38.066963 read(3, 0x7fff093e8de0, 16) = -1 EAGAIN (Resource temporarily unavailable) <0.000007>
16:54:38.066991 poll([{fd=3, events=POLLIN}], 1, 10000
Time to, poll wake up.
) = 0 (Timeout) <10.000625>
16:54:48.067806 write(2, "\0\0Z\0\227|\0\0t31895 [local] postgres "..., 99) = 99 <0.000012>
16:54:48.067855 sendto(8, "\2\0\0\0(\0\0\0\3132\0\0\0\0\0\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 40, 0, NULL, 0) = 40 <0.000022>
16:54:48.067907 sendto(9, "T\0\0\0!\0\1pg_sleep\0\0\0\0\0\0\0\0\0\10\346\0\4\377\377\377\377"..., 65, 0, NULL, 0) = 65 <0.000090>
16:54:48.068027 recvfrom(9,
Call sendto method returns, the need to fight the string is returned. Return result set ( T\0\0\0!\0\1pg_sleep\0\0\0\0\0\0\0\0\0\10\346\0\4\377\377\377\377), beginning with T, the client receives the display:
postgres=# select pg_sleep(10);
pg_sleep
----------
(1 row)
4 Summary
The problem is that when there is a process state, strace is a very powerful tool that can analyze the current system calls to observe the behavior of the process, further inference can be applied in the upper run what kind of SQL, what kind of command execution, and so on.
Common Law:
strace -T -tt -e trace=all -p 21231
strace -T -tt -e trace=all -o file -p 21231
