Foreword, the origin of this writing
In a recent study Wei Dongshan embedded training video (3 project combat the USB camera monitoring), modeled on the video tutorial, wrote a simplified version of uvc camera driver. When commissioning the drive module in a virtual machine, into a pit, toss a long time to finally climb out. The source is due to their lack of knowledge of the load address linux module, plus a network text misleading. Do hereby organize notes recorded during the climb pit, hoping to themselves and we all have help.
First, the experimental environment
1.1 virtual machine environment
a) Vmware printed books: Vmware Workstation 12.5.7
b) Ubuntu Version: 9.10
c) kernel version: 2.6.31.14
d) gcc Version: 4.4.1
e) gdb Version: 7.0
1.2 development environment board
1.2.1 Hardware
Development Board: Hundred Questions net JZ2440 development board
Camera: Hundred Questions uvc net homemade camera
1.2.2 Software
a) kernel version: 2.6.31.14
b) toolchain version:
arm-linux-gcc 3.4.5
arm-linux-gdb 6.8
Second, the basics Introduction
2.1 linux mountable module (referred LKM)
LKM is a special ELF file. You can objdmp -h or readelf -e to view its information section.
LKM can not be run directly, but need to use insmod command dynamically loaded into the kernel to run. Loading the kernel is implemented mainly LDM load_module () in. It will allocate memory space for the LKM, calculated load address of each section, and copy them to a corresponding load address.
2.2 kgdb about how to use the debugging module can be installed, there are many online articles. To conclude, the basic routines are similar, as follows:
Kernel i) the configuration of the target machine to support debugging kgdb
ii) the configuration parameters bootstrap startup target machine to support serial debug kgdb
iii) the target machine insmod xxx.ko, and then obtaining the text section xxx.ko load address (assumed to be in some way 0xd099a000) (Note: see below 3.3)
iv) operating the target echo g> / proc / sysrq-trigger, waiting for the development of machines in the shell connected gdb
v) development machine into linux source directory in the shell, and then execute:
gdb ./vmlinux
(gdb) set remotebaud 115200
(gdb) target remote /dev/ttyS0
Remote debugging using /dev/ttyS0
kgdb_breakpoint () at kernel/kgdb.c:1674
1674 wmb(); /*Sync point after breakpoint */
The above information kgdb gdb description of the development machine and the target machine connection is successfully established. Next, run on the development machine:
(gdb) add-symbol-file /xxx_path/xxx.ko 0xd099a000
Subsequent debugging method, and to use gdb debugger application on a single machine is substantially the same.
2.3 on how to get the module load address of code section, there are online in two ways:
1) proven feasible
cat /sys/module/xxx.ko/sections/.text
Method 2) proven to be problematic
"Cat / proc / modules | grep test" (assuming that the module name is test.ko, be careful not to bring ".ko") to find the module load address, as shown below:
Find the code section (.text section) offset, as shown below:
Is the offset corresponding to the fourth row .text hexadecimal field (from left to right or sixth field) .text section is offset in the file. This will offset the load address adding module, the module can find the address of the code after the redirection. In our example, we can be obtained 0xffffffffa001b000 + 0x00000040 = 0xFFFFFFFFA001B040.
Third, why the second method is problematic
3.1 Analysis of the kernel source
(Kernel version analysis below is 2.6.31.14)
Load modules in load_module kernel / module.c of () in implementation:
static noinline struct module *load_module(void __user *umod, unsigned long len, const char __user *uargs) { . . . /* Determine total sizes, and put offsets in sh_entsize. For now this is done generically; there doesn't appear to be any special cases for the architectures. */ layout_sections(mod, hdr, sechdrs, secstrings);
. . .
/* Transfer each section which specifies SHF_ALLOC */ DEBUGP("final section addresses:\n"); for (i = 0; i < hdr->e_shnum; i++) { void *dest; if (!(sechdrs[i].sh_flags & SHF_ALLOC)) continue; if (sechdrs[i].sh_entsize & INIT_OFFSET_MASK) dest = mod->module_init + (sechdrs[i].sh_entsize & ~INIT_OFFSET_MASK); else dest = mod->module_core + sechdrs[i].sh_entsize;
if (sechdrs[i].sh_type != SHT_NOBITS)
memcpy(dest, (void *)sechdrs[i].sh_addr,
sechdrs[i].sh_size);
/* Update sh_addr to point to copy in image. */
sechdrs[i].sh_addr = (unsigned long)dest;
DEBUGP("\t0x%lx %s\n", sechdrs[i].sh_addr, secstrings + sechdrs[i].sh_name);
} }
In each section of the positioning module, mainly implemented in two or more tags.
- the role of first code layout_sections (), as said Notes: a) determining the total size of module, b) offset to each computing module section, and stores in the sechdrs.sh_entsize (as kernel modules minutes
with a Elf_Shdr type array sechdrs [], each section respectively correspond to the array element module header information)
- the role of the second segment of code, is a section of the module with the SHF_ALLOC flag, copy its contents to the final destination address.
Below the main analysis layout_sections ():
/ * Lay OUT at The SHF_ALLOC in Sections A Way How not dissimilar to LD . Might - code, the Data the Read-only, the Read-the Write the Data, the Data Tally Small sizes, and Place at The offsets INTO sh_entsize Fields: High bit means IT Belongs in the init. * / static void layout_sections ( struct Module1 MOD *, const Elf_Ehdr * HDR, Elf_Shdr sechdrs *, const char * secstrings) { static unsigned Long const Masks [] [2] = { // meaning of the mask, and yet get to the bottom, see Resources 5.8 ( Analysis of ELF format three: Sections ): * in the this Array; otherwise the Modify at the text_size * NOTE: All the MUST BE Executable code at the First Section * finder in the two loops below */ { SHF_EXECINSTR | SHF_ALLOC, ARCH_SHF_SMALL }, { SHF_ALLOC, SHF_WRITE | ARCH_SHF_SMALL }, { SHF_WRITE | SHF_ALLOC, ARCH_SHF_SMALL }, { ARCH_SHF_SMALL | SHF_ALLOC, 0 } }; unsigned int m, i; for (i = 0; i < hdr->e_shnum; i++) sechdrs[i].sh_entsize = ~0UL; //先初始化sh_entsize 为-1 DEBUGP("Core section allocation order:\n"); for (m = 0; m < ARRAY_SIZE(masks); ++m) { for (i = 0; i < hdr->e_shnum; ++i) { Elf_Shdr *s = &sechdrs[i]; if((S-> The sh_flags & Masks [m] [0])! = Masks [m] [0] || (S-> The sh_flags & Masks [m] [. 1]) || S-> sh_entsize! = ~ 0UL strstarts || (S- secstrings +>, sh_name, " .init ")) // some determination for each section, go into details yet Continue ; S-> sh_entsize = get_offset (MOD, & mod-> core_size, S, I ); // calculate the offset of each section, and stores the sechdrs.sh_entsize in DEBUGP ( " \ T% S \ n- ", secstrings S- +>, sh_name); } IF (m == 0) mod-> core_text_size mod- => core_size; } DEBUGP ( " the Init sectionTop Allocation Order: \ n- ");
... // init omitted sections and associated processing code
}
}
Visible, the core code is get_offset () function:
. / * The Update size with the this sectionTop: return offset * / static Long get_offset ( struct Module1 * MOD, unsigned int * size, Elf_Shdr * sechdr, unsigned int sectionTop) { Long RET; * size + = arch_mod_section_prepend (MOD, sectionTop); // in the non-parisc platform, arch_mod_section_prepend defined as 0 return; RET = aLIGN (* size, sechdr-> sh_addralign: 1? ); // alignment correction. Note: trinocular where expression is equivalent to sechdr-> sh_addralign sechdr-> sh_addralign:. 1? * + RET = sechdr- size> The sh_size; return RET; }
It can be seen from the code: the kernel at each section calculated address offset LKM loading offset information .ko did not use the file, but calculated according to the formula:
offset_align = ALIGN(*size, sechdr->sh_addralign ?: 1)
Wherein, offset_align offset address is calculated, * size is the total size of all previous section of the section, the ALIGN alignment is corrected, that the alignment of the section 1 or sh_addralign.
For .text section, because it is the first section of the module, so * size that is mod-> core_size, and because mod-> core_size initial value is 0 (this is my single step of determining, but I did not found their place in the source code explicitly initialized?), so offset_align text section is equal to 0, in other words, text section load address is the address of the load module.
3.2 experiments (experiments JZ2440 development board)
Step1) in get_offset () after Riga printk, compiled kernel installation
step2) insmod my_uvc.ko, then cat / proc / modules | grep my_uvc load address obtaining module, then each section is acquired by the load address sysfs
#cat /proc/modules |grep my_uvc
my_uvc 15028 0 - Live 0xbf000000
# ls -al /sys/module/my_uvc/sections/
drwxr-xr-x 2 0 0 0 Jan 1 00:04 .
drwxr-xr-x 6 0 0 0 Jan 1 00:01 ..
-r--r--r-- 1 0 0 4096 Jan 1 00:04 .bss
-r--r--r-- 1 0 0 4096 Jan 1 00:04 .data
-r--r--r-- 1 0 0 4096 Jan 1 00:04 .gnu.linkonce.this_module
-r--r--r-- 1 0 0 4096 Jan 1 00:04 .note.gnu.build-id
-r--r--r-- 1 0 0 4096 Jan 1 00:04 .rodata
-r--r--r-- 1 0 0 4096 Jan 1 00:04 .rodata.str1.1
-r--r--r-- 1 0 0 4096 Jan 1 00:04 .strtab
-r--r--r-- 1 0 0 4096 Jan 1 00:04 .symtab
-r--r--r-- 1 0 0 4096 Jan 1 00:04 .text
# cat /sys/module/my_uvc/sections/.text
0xbf000000
# cat /sys/module/my_uvc/sections/.note.gnu.build-id
0xbf0013e8
# cat /sys/module/my_uvc/sections/.rodata
0xbf00140c
# cat /sys/module/my_uvc/sections/.rodata.str1.1
0xbf0015c4
# cat /sys/module/my_uvc/sections/.symtab
0xbf001a08
# cat /sys/module/my_uvc/sections/.strtab
0xbf002438
# cat /sys/module/my_uvc/sections/.gnu.linkonce.this_module
0xbf002a70
# cat /sys/module/my_uvc/sections/.bss
0xbf002b84
Contrast printk logs (Note: The Scarlet Letter is my hand-plus settlement results)
sh_name:50, name:.text, sh_type:1, sh_flags:6, sh_offset:88, sh_size:5096, sh_addralign:4, offset_align:0 0xbf000000+0=0xbf000000 sh_name:27, name:.note.gnu.build-id, sh_type:7, sh_flags:2, sh_offset:52, sh_size:36, sh_addralign:4, offset_align:0x13e8 0xbf000000+0x13e8=0xbf0013e8 sh_name:60, name:.rodata, sh_type:1, sh_flags:2, sh_offset:5184, sh_size:440, sh_addralign:4, offset_align:0x140c 0xbf000000+0x140c=0xbf00140c sh_name:77, name:.rodata.str1.1, sh_type:1, sh_flags:50, sh_offset:5682, sh_size:1092, sh_addralign:1, offset_align:0x15c4 0xbf000000+0x15c4=0xbf0015c4 sh_name:1, name:.symtab, sh_type:2, sh_flags:2, sh_offset:130584, sh_size:2608, sh_addralign:4, offset_align:0x1a08 0xbf000000+0x1a08=0xbf001a08 sh_name:9, name:.strtab, sh_type:3, sh_flags:2, sh_offset:133192, sh_size:1397, sh_addralign:1, offset_align:0x2438 0xbf000000+0x2438=0xbf002438 i:8, sh_name:96, name:.data, sh_type:1, sh_flags:3, sh_offset:6776, sh_size:192, sh_addralign:4, offset_align:0x29b0 0xbf000000+0x29b0=0xbf0029b0 i:10, sh_name:106, name:.gnu.linkonce.this_module, sh_type:1, sh_flags:3, sh_offset:6968, sh_size:276, sh_addralign:4, offset_align:0x2a70 0xbf000000+0x2a70=0xbf002a70 i:12, sh_name:132, name:.bss, sh_type:8, sh_flags:3, sh_offset:7244, sh_size:3888, sh_addralign:4, offset_align:0x2b84 0xbf000000+0x2b84=0xbf002b84
Visible, actual results and source code analysis is consistent with the conclusions.
Fourth, if you use the second method to get the text section of LKM loading address, the actual commissioning, will show any symptoms, what is the reason
4.1 practice found in the X86 platform, and the platform under the arm, showing the actual commissioning of the symptoms, the difference is relatively large
1) In the following X86 platform, there will be a lot of strange phenomena, such as:
- Although the rate of break point, but continue to let the program up and running, but directly ignored breakpoints;
- although they would sometimes break into, but after a single step, the program ran fly out;
- even add a breakpoint, let the program continue after the run, it will be reported segmentation fault, or instruction fault
2) In the arm platform, can debug the normal (which include breakpoints, single step, monitoring, etc.), showed no symptoms of formula
4.2 Why In both platforms, there is such a difference?
To find out, try to do the following experiments were in two platforms (for simplicity, object code with the Song Baohua of "Linux device driver development explain: based on the latest Linux 4.0 kernel" globalmem Chapter VI):
Step1) by two methods (see section 2.3 above), were acquired text section LKM load address;
Step2) In GDB, respectively, these two addresses add-symbol-file, then b globalmem_read (not continue the second), a first breakpoint is hit when the continue print disassembly, the result is:
a) X86 platform (assuming cat / proc / modules | grep globalmem load address is obtained LKM 0xe0c26000, and objdump -h text section offset is obtained 40)
Get the load address text section with the second method, and then add-symbol-file, b globalmem_read:
(gdb) add-symbol-file /home/book/workspace/baohua/drivers/ch6/globalmem.ko 0xe0c26040 add symbol table from file "/home/book/workspace/baohua/drivers/ch6/globalmem.ko" at .text_addr = 0xe0c26040 (y or n) y Reading symbols from /home/book/workspace/baohua/drivers/ch6/globalmem.ko...done. (gdb) b globalmem_read Breakpoint 1 at 0xe0c261a0: file /home/book/workspace/baohua/drivers/ch6/globalmem.c, line 64.
Get the load address text section with the first method, and then add-symbol-file, b globalmem_read:
(gdb) add-symbol-file /home/book/workspace/baohua/drivers/ch6/globalmem.ko 0xe0c26000
add symbol table from file "/home/book/workspace/baohua/drivers/ch6/globalmem.ko" at
.text_addr = 0xe0c26000
(y or n) y
Reading symbols from /home/book/workspace/baohua/drivers/ch6/globalmem.ko...done.
(gdb) b globalmem_read
Breakpoint 1 at 0xe0c26174: file /home/book/workspace/baohua/drivers/ch6/globalmem.c, line 70.
(gdb) c
Continuing.
[New Thread 3101]
[Switching to Thread 3101]
Breakpoint 1, globalmem_read (filp=0xda625b00,
buf=0x9163000 <Address 0x9163000 out of bounds>, size=32768, ppos=0xd7ae9f98)
/home/book/workspace/baohua/drivers/ch6/globalmem.c:70 AT
70 if (p >= GLOBALMEM_SIZE)
(gdb)disassemble
0xe0c26160 <0 globalmem_read +>: Push% EBP
0xe0c26161 <globalmem_read. 1 +>:% MOV ESP, EBP%
0xe0c26163 <globalmem_read. 3 +>: Sub 0x18 $,% ESP
0xe0c26166 <globalmem_read. 6 +>: MOV EBX%, -0xc (% EBP)
0xe0c26169 <globalmem_read. 9 +>: MOV ESI%, -0x8 (% EBP)
0xe0c2616c <12 is globalmem_read +>: MOV EDI%, -0x4 (% EBP)
0xe0c2616f <globalmem_read + 15>: nopl 0x0 (% EAX, EAX%,. 1) 0xe0c26174 <20 is globalmem_read +>: XOR% ESI, ESI%; visible, with the first method for obtaining the address of experiments , the breakpoint is correct
0xe0c26176 <22 is globalmem_read +>: MOV 0x70 (EAX%),% EAX
0xe0c26179 <globalmem_read + 25>:% MOV EDX, EDI%
0xe0c2617b <globalmem_read + 27>:mov 0x8(%ebp),%edx
0xe0c2617e <globalmem_read + 30>: MOV (% EDX), EBX%
0xe0c26180 <globalmem_read + 32>: $ 0xFFF CMP,%ebx
0xe0c26186 <globalmem_read+38>: ja 0xe0c261a8 <globalmem_read+72>
0xe0c26188 <globalmem_read+40>: mov $0x1000,%si
0xe0c2618c <globalmem_read+44>: sub %ebx,%esi
0xe0c2618e <globalmem_read+46>: cmp %ecx,%esi
0xe0c26190 <globalmem_read+48>: ja 0xe0c261b8 <globalmem_read+88>
0xe0c26192 <globalmem_read+50>: lea 0x3c(%eax,%ebx,1),%edx
0xe0c26196 <globalmem_read+54>: mov %esi,%ecx
0xe0c26198 <globalmem_read+56>: mov %edi,%eax
0xe0c2619a <globalmem_read+58>: call 0xc0326d60 <copy_to_user>
0xe0c2619f <globalmem_read+63>: test %eax,%eax
; Visible, with the second method of obtaining the address of experiments, since the code segment, and no valid instructions 0xe0c261a0 this address, so that if a breakpoint applied thereon, will produce a variety of strange problem 0xe0c261a1 <globalmem_read +65>: JE 0xe0c261bc <globalmem_read + 92> 0xe0c261a3 <globalmem_read + 67>: MOV 0xfffffff2 $,% ESI 0xe0c261a8 <globalmem_read + 72>: MOV ESI%,% EAX 0xe0c261aa <globalmem_read + 74>: MOV -0xc (EBP% ), EBX% 0xe0c261ad <globalmem_read + 77>: MOV -0x8 (EBP%), ESI% 0xe0c261b0 <globalmem_read + 80>: MOV -0x4 (% EBP), EDI% 0xe0c261b3 <globalmem_read + 83>: MOV EBP%,% ESP 0xe0c261b5 <globalmem_read + 85>:% POP EBP 0xe0c261b6 <globalmem_read + 86>: RET
b) arm platform (assuming cat / proc / modules | grep globalmem load address is obtained LKM 0xbf000000, and objdump -h text section offset is obtained 34)
Get the load address text section with the second method, and then add-symbol-file, b globalmem_read:
(gdb) add-symbol-file /home/book/workspace/baohua/drivers/ch6/globalmem.ko 0xbf000034 add symbol table from file "/home/book/workspace/baohua/drivers/ch6/globalmem.ko" at .text_addr = 0xbf000034 (y or n) y Reading symbols from /home/book/workspace/baohua/drivers/ch6/globalmem.ko...done. (gdb) b globalmem_read Breakpoint 1 at 0xbf000200: file /home/book/workspace/baohua/drivers/ch6/globalmem.c, line 65.
Get the load address text section with the first method, and then add-symbol-file, b globalmem_read:
(gdb) add-symbol-file /home/book/workspace/baohua/drivers/ch6/globalmem.ko 0xbf000000
add symbol table from file "/home/book/workspace/baohua/drivers/ch6/globalmem.ko" at
.text_addr = 0xbf000000
(y or n) y
Reading symbols from /home/book/workspace/baohua/drivers/ch6/globalmem.ko...done.
(gdb) b globalmem_read
Breakpoint 1 at 0xbf0001cc: file /home/book/workspace/baohua/drivers/ch6/globalmem.c, line 65.
(gdb) c
Continuing.
[New Thread 787]
[Switching to Thread 787]
Breakpoint 1, globalmem_read (filp=0xc3df44e0, buf=0xbe962c80 "", size=8192,
ppos=0xc06edf80) at /home/book/workspace/baohua/drivers/ch6/globalmem.c:65
65 unsigned long p = *ppos;
(GDB) the disassemble
0xbf0001c0 <globalmem_read + 0>: MOV R12, SP
0xbf0001c4 <globalmem_read +. 4>: Push {R4, R5, R6, R7, R8, R11, R12, LR, PC}
0xbf0001c8 <globalmem_read +. 8>: Sub R11, R12, #. 4; 0x4 0xbf0001cc <12 is globalmem_read +>: LDR R7, [R3]; visible experiment, the breakpoint address with the first method of obtaining the position is correct
0xbf0001d0 <globalmem_read + 16>: mov R8, R3
0xbf0001d4 <globalmem_read + 20 is>: CMP R7, # 4096; 0x1000
0xbf0001d8 <globalmem_read + 24>: MOV R12, R1
0xbf0001dc <globalmem_read + 28>: LDR r0 of, [r0 of, # 1 16]
0xbf0001e0 <globalmem_read + 32> : movcs R6, # 0; 0x0
0xbf0001e4 <globalmem_read + 36>: BCS 0xbf00025c <globalmem_read + 156>
0xbf0001e8 <globalmem_read + 40>: MOV R1, SP
0xbf0001ec <globalmem_read+44>: bic r3, r1, #8128 ; 0x1fc0
0xbf0001f0 <globalmem_read + 48>: BIC R3, R3,#63 ; 0x3f
0xbf0001f4 <globalmem_read + 52 is>: RSB R5, R7, # 4096; 0x1000
0xbf0001f8 <globalmem_read + 56 is>: CMP R5, R2
0xbf0001fc <globalmem_read + 60>: R5 movcs, R2 0xbf000200 <globalmem_read + 64>: LDR R3, [R3, #. 8]; visible experiment with the second method the acquired address, the breakpoint location has deviated from the correct value of 0x34 bytes, since this address it is a valid address, so in fact normal or debug, and does not exhibit symptoms explicit
0xbf000204 <globalmem_read + 68>: ADDS R2, R12, R5
0xbf000208 <globalmem_read + 72>: sbcscc R2, R2, R3
0xbf00020c <globalmem_read + 76>: movcc R3, # 0; 0x0
0xbf000210 <globalmem_read + 80>: CMP R3, # 0; 0x0
0xbf000214 <globalmem_read + 84>: r0 of MOVNE, R5