File I / O
- C standard I / O library functions and Unbuffered I / O function
- C standard I / O library function printf (), putchar (), fputs (), will open up the I / O buffer in user space
- (), Read (), write (), close (), etc. located in the C standard library function I underlying system open / O buffers, also called unbuffered I \ O (Unbuffered I / O) functions
- When I call the standard library to read and write regular file / O / O is faster than Unbuffered I, and the need to manage I / O buffers
- Buffer does not need to write terminal or network device, usually direct calls Unbuffered I / O
- Each process in the Linux kernel has a task_struct structure to maintain information related to the process, called process descriptor (Process Descriptor)
- There is a pointer to the task_struct files_struct structure, known as a file description table , wherein each entry contains a pointer to open file pointer
- Users can not directly access the file description in the kernel, but can only use the file descriptor table index (0,1,2, etc.), called a file descriptor , save with a variable of type int
- When the call to open a file is opened, the kernel allocates a file descriptor and returned to the user program, the file descriptor table entry pointer to the newly opened file
- When reading and writing files, the file descriptor passed to the user program read or write, the kernel to find the corresponding entry in accordance with the file descriptor, and then locate the file table entry pointer by
- Three files will automatically open when the program starts: standard input, standard output and standard error output
- open/close
- int open(const char *pathname, int flags, ... );
- File permissions determined by the mode argument of open and umask mask of the current process
- int close(int fd);
- read/write
- Blocking (Block): When a process calls function is blocked, the process is put to sleep (Sleep) state, which is the kernel scheduler other programs until the process is waiting for an event occurs (such as a packet is received on the network) before there may continue to run
- Running (Running): being executed or ready state (ready to be executed)
- Polling (Poll): supposed to be up here, but in fact is not blocked, but directly returns an error, the caller should try to read it again
- Non-blocking I / O has a drawback, if all the devices have been no data arrives, the caller needs to repeated inquiries to do useful work, if the blockage where the operating system can schedule another process execution
- It may use non-blocking I / O, usually do not stop the query in a while loop, but did not inquire about the delay to Parliament, while waiting for a delayed schedule other processes can be executed, but the disadvantage is when the data arrives not in time
- select (2) can function simultaneously monitoring a plurality of blocking devices, you can also set blocking wait timeout
- lseek
- Move the current read position (offset)
- fcntl
- File status attribute (File Status Flag): What is the current process accesses the file or device, such as read, write, append, non-blocking, etc.
- Reset state property has been open a file without having to re-open the file
- Shell redirection syntax: open the file on which the file descriptor in the <,>, >>, <> to add a digital front, the number is expressed
- $ Command> / dev / null 2> & 1: The standard output redirection command command to / dev / null, then an error message (standard error) The command may also be generated and redirected to the standard output (& 1, not & plus it will be interpreted as a file name) of the same document, i.e., / dev / null
- Written in the / dev / null file data is not displayed, execute commands, but that is normal or not print an error message
- ioctl
- Attribute transmitting control information, a file or the device itself, such as baud rate, terminal window size
- Data read / write read and write, is the main I / O operations, known as in-band data; data can not be read / write read and write, referred to as Out-of-band data
- mmap
- A portion of the disk file is directly mapped into memory, read and write pointers may be used directly without via a file read / write functions
File system
- introduction
- How to present the user with a tree structure? How to handle user requests a file and directory operations?
- How to achieve a linear tree structure stored on disk? File storage format how to design the most efficient access to the system disk?
- What are the various file and directory operations on disk net effect?
- ext2 file system
- File system block is the smallest unit (Block), ext2 file system entire partition divided into several groups of blocks of the same size (Block Group), for each block group consists of the following components:
- Superblock (Super Block): Description excessively positive file system information area, such as block size, the version number of the file system
- Block group descriptor table (GDT, Group Descriptor Table): description information in a block group, such as where to start the inode table, where the start data block and the like. The entire partition is divided into a corresponding number of block groups to the number of the group descriptor
- Block bitmap (Block Bitmap): which describe the entire block group which blocks the free block is used
- Bitmap inode (inode Bitmap): block bitmap and the like, itself occupies one block, wherein each bit represents whether or not a free inode
- inode table (inode Table): description of the information store files, such as file type, permissions, size, creation / modification / access time, namely that information ls -l command to see
- Data block: data file storage
- The smallest unit of read and write disk is called a sector (Sector), typically 512 bytes
- Data Block Addressing
- File and directory operations system functions:
- read inode file: stat (2)
- access(2)
- chmod(2)
- chown(2)
- utime(2)
- truncate(2)
- link(2)
- rename(2)
- mkdir(2)
- rmdir(2)
- File system block is the smallest unit (Block), ext2 file system entire partition divided into several groups of blocks of the same size (Block Group), for each block group consists of the following components:
- Virtual File System (VFS, Virtual File System): Linux kernel on a variety of different file formats make an abstraction layer so that the concept of files, directories, such as read and write access abstraction layer called the concept, so the various file systems looks and with them all the same
- Linux file systems: network file system, disk file system, special file system, etc.
- VFS as a general-purpose file system, the four basic concepts of abstract file system, file, directory entry, inode, mount point
- In Linux, in addition to process all that is file
- Four basic object of VFS
- Object superblock (superblock object): a mounted file system
- Inode object (inode object): a file
- Object directory entry (dentry object): a directory
- File object (file object): Open the file by the process
-
- Each process in the PCB (Process Control Block) are saved with a file descriptor table, file descriptor is an index of the table, each entry has a pointer to the file structure in the kernel
- file structure member
- f_flags：File Status Flag
- f_pos: The current position of the reader
- f_count: reference count (Reference Count)
- Each file structure points to a file_operations structure, the structure member is a function pointers to various file operations implemented kernel functions
- Each file has a pointer to the structure dentry (directory entry directory entry) pointer structure, the kernel cache directory in the tree structure dentry Cache, each node is a dentry structure, according to the path (e.g. / home / akaedu / a) find the file's inode
- inode structure holds the information read from the disk inode to partition, such as owner, file size, type, permissions, etc.
- Each inode structure has a pointer to inode_operations structure, which is a set of functions file directory pointer to the completion of some operations kernel functions, such as add and delete files and directories, and other symbolic links follow
- inode has a pointer to a structure super_block structure which holds information on the super block from the disk to read the partition, such as the file system type, block size, etc., which is a pointer to a member s_root dentry, showing the document root of the system is to mount where
- dup and dup2 can be used to copy an existing file descriptor, so that the two file descriptors point to the same file structure
- Hard link (hard link): an inode number corresponding to the plurality of file names, i.e. a document using a plurality of aliases
- Soft link (soft link): an inode number corresponds to a file name, Trivial File
process
- introduction
- Each process in the kernel has a process control block (PCB) to maintain information related to the process, Linux is a PCB task_struct structure, including the following information
- Process id: a non-negative integer, the system each process has a unique id
- State of the process: run, suspend, stop, zombies, etc.
- Need to be saved and restored when the process of switching some CPU registers
- Information describing the virtual address space
- Description of the control terminal
- The current working directory
- umask mask
- File descriptor table, a pointer to the file structure contains many body
- And signal-related information
- User id and group id
- Control terminal, Session and process group
- Resource Capping (Resource Limit) process can be used
- fork: According to a copy of the existing process (parent process Parent Process) a new process (child process Child Process)
- exec: implementation of the new program
- Each process in the kernel has a process control block (PCB) to maintain information related to the process, Linux is a PCB task_struct structure, including the following information
- Environment Variables
- PATH: Search path for executable files can contain multiple directories, separated by a: Check the echo command value
- SHELL: current Shell, usually value / bin / bash
- TERM: current terminal type
- LANG: language and locale, decided to show the character encoding format information and time, currency, etc.
- HOME: The current path of the user's home directory
- Process Control
- fork: the system call function, after calling into the kernel, according to replicate a child process the parent process. Called once, twice returned (the parent process called once, the parent and child processes each returned once)
- gdb can only track a process (default is the parent process), but can not track multiple processes simultaneously
- exec: After you create a child process using fork, the child process calls exec needs to execute another program
- wait and waitpid: the process is terminated Shell (parent) calls wait or waitpid to get its exit status while completely remove this process, but the process terminates if the parent does not call wait or waitpid to clean it, which is the process of state zombie process
- Interprocess communication
- Each process each with a different user address space, can not see the global variables of other processes, inter-process data must be exchanged through the kernel
- An open buffer in the kernel, a process to copy data from user space to the buffer, and then process 2 from the kernel buffer to read the data away, this mechanism is called inter-process communication (IPC, InterProcess Communication)
- Pipeline is a basic IPC mechanism, created by the pipe function, the following restrictions
- Two-way communication process can only be achieved through a pipeline, if the parent is the child process writing to read, you need to open another pipe
- Read end of the pipe passing through an open file descriptor, a file descriptor to be passed through the fork
- Four kinds of special circumstances pipeline
- All point to the write end of the pipe file descriptors are closed
- There are pointing to the write end of the pipe file descriptor is not closed
- All point to the read end of the pipe file descriptors are closed
- Pointing the read end of the pipe does not turn off the file descriptor
- Other IPC mechanisms: FIFO, Unix Domain Socket (most widely used), using the file system in a special file to identify IPC channel
Shell Script
- How to Run
- Interpreted user command input one, interpreted a, manner of implementation of interactive (Interactive) and batch (Batch) (prior user to write a Shell script (Script), Shell completed the first performance) in two
- Users enter commands in the command line, Shell will fork and exec the command, but the built-in command exception, execute built-in command is equivalent to calling a function Shell process does not create a new process
- The basic syntax
- Variables: the capital letters underlined composed of sub-environment variables and local variables (Shell only exist in the current process)
- Filename substitution (Globbing): called wildcard characters for matching (Wildcard)
- Command substitution: `` or $ () is a command
- Arithmetic substitution: $ (()) in the Shell value to an integer variable
- Special meaning \ for the removal of a single character immediately following: the escape character
- Single quotes, double quotes: Keep all the characters in the literal quotes
- bash startup script
- You can set the environment variable and alias, umask settings in the startup script, take effect every time you start the Shell
- bash startup mode
- Shell started as an interactive login, or use the parameter start --login
- Interactive non-login Shel start
- Non-interactive start
- Sh command to start
- Shell script syntax
- Test conditions: test or [
- if/then/elif/else/fi
- case/esac
- for/do/done
- while/do/done
- Positional parameters and special variables
- function
- Shell scripts debug mode
- -n
- -v
- -x
Regular Expressions
- introduction
- Regular expressions: some special provisions Syntax Notation character classes, and the positional relationship between the number of qualifiers, and then use these special syntax and string together represent a common mode (Pattern)
- Message contains: a character class (Character Class), the number of qualifiers (Quantifier), the positional relationship between characters
- Application: validate user input (ip, email) format is legal
- The basic syntax
- Common Commands
- grep
- sed: stream editor (Stream Editor), Shell script used as a filter, the output of a program introduced before the input sed, edited after conversion to another output format
- awk: in rows or as a processing unit
signal
- basic concepts
- Press Ctrl-C signal is generated by the foreground process issued (hardware interrupt), CPU to switch from user mode to kernel mode processing interrupts
- Signal relative to the control flow of the process is asynchronous
- kill -l to view a list of system-defined signal, each signal has a label and a macro name, macro definitions can be found in signal.h
- Each signal has a default condition and the generation processing operation
- Generating a signal
- Blocking signal
- Performing a signal processing operation of delivery is referred to as signal (Delivery)
- Signal generated between a state referred to as signal delivery of pending (the Pending)
- Process to select a signal blocking (Block), held in a pending status signal is generated when it is blocked until the process is unblocked, before the implementation of the delivery of the action
- Capture signal
- The signal processing operation is user-defined functions (signal handler), the signal of the function call when the handover, the system automatically performs sigreturn call into the kernel mode returns again, no new signal was delivered, the main returns the context to continue
Terminal, and job control daemon
- terminal
- Serial terminal: embedded development, each port corresponding to a target terminal board
- Virtual Terminal (Virtual Terminal):
/dev/tty1~/dev/tty6 - Terminal buffer: input terminal / output buffer queue
- Terminal login: kernel module comprises a hardware processing terminal device drivers and line protocols (Line Discipline)
- Network Terminal / graphics terminal (window): Any number, achieved by pseudo-terminal (Pseudo TTY), a device composed of a master pseudo-terminal (PTY Master, corresponding to the keyboard and display) and a composition from a device (PTY Slave)
- Job control (Job Control):
- Job (Job) refers to the process group (Process Group), Shell can run multiple background and a foreground job jobs simultaneously
- Session: have the same control terminal of a group of processes
- Daemon (Daemon)
- System service process in Linux, there is no control terminal, and the user can not interact without affecting the user logon and logoff been running
Thread
- The concept of threads
- While performing in a process control multiple processes, such as downloading software, a graphical interface, and user interaction to the side, waiting for the user's mouse and keyboard event handling, while simultaneously download multiple files, wait and deal with hair from multiple network hosts to the data
- Multitasking "Wait - Processing" cycle can multithreading
- Multiple threads of the same process share the same address space
- Thread Control
- Create a thread
- Terminate the thread
- Synchronization between threads
- Mutex (Mutex, Mutual Exclusive Lock): represents the number of available resources, either 0 or 1, a thread to acquire the lock can be done, "read - modify - write" operation, and then to the other thread releases the lock, no lock thread only can not wait for access to shared data, ensure that the "read - modify - write" is an atomic operation (either perform or not perform), does not perform to the middle interrupted, this will not be parallel on the other processors operating
- Deadlock (Deadlock)
- Condition variable (Condition Variable)
- Semaphore (Semaphore): the number of resources available, may be greater than 1
- Other inter-thread synchronization mechanisms
TCP / IP protocol basis
- TCP / IP protocol stack packet encapsulation
- Packet Name: transporting layer - section (segment), the network layer - the data packets (datagram), the link layer - a frame (Frame)
- Ethernet frame format
- ARP datagram format
- IP datagram format
- IP address and routing
- Private IP address can host links to the Internet through a proxy server or NAT (network address translation)
- Native loopback test (loop back)
-
- Routing Table
- UDP segment format
- TCP protocol
socket programming
- Preliminaries
- TCP protocol-based network program
- select: a system network program calls can be blocked simultaneously monitor multiple file descriptors (such as multiple network connections), which has data on the first treatment which, without fork and thus can achieve concurrent multi-process server services
- UDP-based network protocol program
- UNIX Domain Socket IPC
- Simple Web Server
reference
Linux C Programming stop learning
http://docs.linuxtone.org/ebooks/C&CPP/c/
IBM Tutorial
https://www.ibm.com/developerworks/cn/linux/l-cn-hardandsymb-links/index.html
Soft and hard-wired connection