The Linux operating system is an open source product, as well as an open source software practice and application platform. Under this platform, there are countless open source software supports, such as our common apache, tomcat, mysql, etc.
The biggest concept of open source software is freedom and openness, so Linux, as an open source platform, ultimately needs to achieve optimal application performance at the lowest cost through the support of these open source software. Therefore, when it comes to performance issues, what is mainly achieved is the best combination of the Linux operating system and applications.
01. Summary of performance issues
The performance of the system refers to the effectiveness, stability and response speed of the operating system to complete the task.
Linux system administrators may often encounter problems such as system instability and slow response speed. For example, if a web service is built on Linux, it often occurs that the web page cannot be opened or the opening speed is slow. When encountering these problems, some people will Complaining that the Linux system is not good, in fact, these are superficial phenomena.
When the operating system completes a task, it is closely related to the system's own settings, network topology, routing devices, routing policies, access devices, physical lines, and many other aspects. Any problem in any link will affect the performance of the entire system.
Therefore, when a Linux application has a problem, it should be comprehensively checked from the application program, operating system, server hardware, network environment, etc., locate the part where the problem occurs, and then focus on solving it.
In terms of application programs, operating systems, server hardware, and network environments, the performance is most affected by the two aspects of the application program and the operating system, because problems in these two aspects are not easy to detect and are highly concealed. As long as there is a problem with hardware and network, it can generally be located immediately.
The following mainly explains the performance tuning ideas of the operating system, and the application program needs to be treated in detail.
The following introduces the general ideas and methods of optimizing Linux from four aspects: factors affecting Linux performance, personnel involved in analyzing performance, system performance optimization tools, and system performance evaluation standards.
02. Factors Affecting Linux Performance
2.1 System hardware resources (1) CPU The CPU is the basis for the stable operation of the operating system. The speed and performance of the CPU determine the overall performance of the system to a large extent. Therefore, the more CPUs and the higher the main frequency, the better the server performance. Relatively better. But that's not quite the case.
At present, most CPUs can only run one thread at the same time, and a hyperthreaded processor can run multiple threads at the same time. Therefore, the hyperthreading feature of the processor can be used to improve system performance. Under Linux, hyperthreading is only supported by running SMP cores, however, the more CPUs installed, the less performance gain you will get from hyperthreading.
In addition, the Linux kernel will recognize a multi-core processor as multiple separate CPUs. For example, two 4-core CPUs will be regarded as 8 single-core CPUs under the Lnux system. But from a performance point of view, two 4-core CPUs and 8 single-core CPUs are not completely equivalent. According to the test conclusions drawn by authoritative departments, the overall performance of the former is 25% to 30% lower than that of the latter.
Applications that may have CPU bottlenecks include db servers, dynamic web servers, etc. For such applications, the configuration and performance of the CPU should be placed in the main position.
(2) Memory The size of the memory is also an important factor affecting the performance of Linux. If the memory is too small, the system process will be blocked, and the application will become slow, or even lose response; if the memory is too large, resources will be wasted.
The Linux system uses two methods of physical memory and virtual memory. Although virtual memory can alleviate the shortage of physical memory, it will take up too much virtual memory and the performance of the application will be significantly reduced. To ensure the high-performance operation of the application, the physical memory It must be large enough; however, excessive physical memory will cause waste of memory resources. For example, on a Linux operating system with a 32-bit processor, physical memory exceeding 8GB will be wasted.
Therefore, to use a larger memory, it is recommended to install a 64-bit operating system and enable Linux's large memory kernel support.
Due to the limitation of the addressing range of the processor, on the 32-bit Linux operating system, a single application process can only use a maximum of 4GB of memory. In this way, even if the system has a larger memory, the application cannot "enjoy" it. The best way is to use a 64-bit processor and install a 64-bit operating system. Under the 64-bit operating system, the memory usage requirements of all applications can be met, and there is almost no limit.
Applications that may have memory performance bottlenecks include NOSQL servers, database servers, cache servers, etc. For such applications, the memory size should be placed in the main position.
(3) Disk I/O performance
The I/O performance of the disk directly affects the performance of the application. In an application with frequent reads and writes, if the disk I/O performance is not satisfied, the application will stall. Fortunately, today's disks use many methods to improve I/O performance, such as the common disk RAID technology.
A disk group composed of RAID technology is equivalent to a large hard disk. Users can perform operations such as partitioning, formatting, and establishing a file system on it. It is exactly the same as a single physical hard disk. The only difference is the I/O performance ratio of the RAID disk group. A single hard disk is much higher, and at the same time, the security of data is also greatly improved.
According to different disk combinations, RAID can be divided into RAID0, RAID1, RAID2, RAID3, RAID4, RAID5, RAID6, RAID7, RAID0+1, RAID10 and other levels. Commonly used RAID levels include RAID0, RAID1, RAID5, RAID0+1, Here is a brief introduction.
RAID 0: By bonding multiple hard disks into a larger-capacity hard disk group, the performance and throughput of the disk are improved. This method is low in cost and requires at least two disks, but has no fault tolerance and data repair functions, so it can only be used in environments that do not require high data security.
RAID 1: that is, disk mirroring. By mirroring the data of one disk to another disk, the reliability and repairability of disk data are guaranteed to the greatest extent. It has high data redundancy, but the disk utilization rate is only 50%. %, therefore, the cost is the highest, and it is mostly used in the occasion of saving important data.
RAID5: Disk segmentation and parity check technology are used to improve system reliability. RAID5 has high read efficiency and average write efficiency. At least 3 disks are required. A disk failure is tolerated without affecting data availability.
RAID0+1: Combining RAID0 and RAID1 technology becomes RAID0+1, which requires at least 4 hard disks. In addition to distributing data on multiple disks in this way, each disk has its own mirror disk, which provides full redundancy and allows one disk to fail without affecting data availability, and has fast read/write capabilities.
By understanding the performance of each RAID level, you can choose a suitable RAID level according to the different characteristics of the application, so as to ensure that the application program achieves the best performance in terms of disks.
(4) Internet broadband
Various applications under Linux are generally network-based, so network bandwidth is also an important factor affecting performance. A low-speed and unstable network will cause access blockage of network applications, while a stable and high-speed network bandwidth can Ensure that applications run unimpeded on the network. Fortunately, today's networks are generally gigabit bandwidth or fiber optic networks, and the impact of bandwidth issues on application performance is gradually decreasing.
2.2 Operating system related resources The performance optimization based on the operating system is also multi-faceted. It can be measured from several aspects such as system installation, system kernel parameters, network parameters, and file system. The following is a brief introduction in turn.
(1) System installation optimization System optimization can start from the installation of the operating system. When installing the Linux system, the partition of the disk and the allocation of SWAP memory will directly affect the operating performance of the system in the future.
For example, disk allocation can follow the needs of the application:
For applications with frequent write operations and low data security requirements, the disk can be made into RAID 0; for applications with high data security and no special requirements for reading and writing, the disk can be made into RAID 1; For applications that have high requirements for read operations but no special requirements for write operations, and must ensure data security, RAID 5 can be selected; for applications that have high requirements for both read and write operations, and also have high requirements for data security , you can select RAID10/01.
In this way, different RAID levels are set according to different application requirements, and the system is optimized at the bottom layer of the disk.
With the decrease of memory price and the increase of memory capacity, the setting of virtual memory SWAP now no longer requires the so-called virtual memory to be twice the physical memory, but the setting of SWAP cannot be ignored. According to experience, if The memory is small (physical memory is less than 4GB), and the size of the SWAP swap partition is generally set to twice the size of the memory;
If the physical memory is greater than 8GB but less than 16GB, you can set the SWAP size to be equal to or slightly smaller than the physical memory; have a certain effect.
(2) Kernel parameter optimization After the system installation is completed, the optimization work is not over. Next, the system kernel parameters can be optimized, but the optimization of the kernel parameters should be considered in combination with the applications deployed in the system.
For example, if the system deploys an Oracle database application, then parameters such as the system shared memory segment, system semaphore, and file handle need to be optimized; Optimization, such as modifying network kernel parameters such as net.ipv4.ip_local_port_range, net.ipv4.tcp_tw_reuse, net.core.somaxconn, etc.
(3) File system optimization
The optimization of the file system is also a focus of system resource optimization. The optional file systems under Linux include ext2, ext3, ReiserFS, ext4, and xfs. Different file systems can be selected according to different applications.
The Linux standard file system starts from VFS, then ext, and then ext2. It should be said that ext2 is a standard file system on Linux, and ext3 is formed by adding logs on the basis of ext2. From VFS to ext4, its design idea is not Too many changes are based on the design concept of the early UNIX family based on super blocks and inodes.
The XFS file system is an advanced log file system. XFS provides low-latency, high-bandwidth access to file system data by distributing disk requests, locating data, and maintaining Cache consistency. Therefore, XFS is extremely scalable and robust , has the advantages of excellent logging function, strong scalability, and fast writing performance.
At present, ext4 and xfs on the server side are the mainstream file systems. How to choose a suitable file system needs to be determined according to the characteristics of the file system and the needs of the business.
2.3 Application software resources
The optimization of the application program is actually the core of the entire optimization project. If there is a bug in an application program, even if all other aspects have reached the optimal state, the performance of the entire application system will still be low. Therefore, the optimization of the application program is the core of the performance optimization process. Most importantly, this puts forward higher requirements for program architects and program developers.
03. Personnel involved in analyzing system performance
3.1 Linux operation and maintenance personnel In the process of performance optimization, Linux operation and maintenance personnel undertake very important tasks.
First of all, Linux operation and maintenance personnel must understand and master the current operating status of the operating system, such as system load, memory status, process status, CPU load and other information, which is the basis and basis for detecting and judging system performance;
Secondly, Linux operation and maintenance personnel also master the hardware information of the system, such as disk I/O, CPU model, memory size, network card bandwidth and other parameter information, and then comprehensively evaluate the usage of system resources based on these information.
Third, as a Linux operation and maintenance personnel, you must also master the usage of system resources by applications. A more in-depth point is to understand the operating efficiency of applications, such as whether there are program bugs, memory overflow and other problems. By monitoring resources, you can find out whether there is an exception in the application. If there is indeed a problem with the application, you need to report the problem to the program developer immediately, and then improve or upgrade the program.
Performance optimization itself is a complex and cumbersome process. Only when Linux operation and maintenance personnel understand system hardware information, network information, operating system configuration information, and application program information can they optimize server performance in a targeted manner. The maintenance personnel have sufficient theoretical knowledge, rich practical experience and the mind to analyze problems carefully.
3.2 System Architecture Designers The second type of personnel involved in system performance optimization is the application program architects. If the Linux operation and maintenance personnel, after a comprehensive judgment, find that the performance is affected by the execution efficiency of the application, then the program architecture designer must intervene in time to gain an in-depth understanding of the program's running status.
First of all, the system architecture designer should track and understand the execution efficiency of the program. If there is a problem with the execution efficiency, find out where the problem has occurred; secondly, if there is a problem with the architecture design, then immediately optimize or improve the system architecture. Design better application system architecture.
3.3 Software developers The last link of system performance optimization involves program developers. After Linux operation and maintenance personnel or architecture designers find program or structural bottlenecks, program developers must immediately intervene and make corresponding program modifications. The modification of the program should take the execution efficiency of the program as the benchmark, improve the logic of the program, and optimize the code in a targeted manner.
For example, Linux operation and maintenance personnel found a SQL statement in the system that consumed a large amount of system resources, grabbed the executed SQL statement, and found that the execution efficiency of the SQL statement was too poor, which was caused by the low execution efficiency of the code written by the developer. , which requires this information to be fed back to the developer. After receiving this question, the developer can optimize the SQL in a targeted manner, and then realize the optimization of the program code.
From the above process, we can see that the process generally followed by system performance optimization is:
First of all, Linux operation and maintenance personnel check the overall status of the system, and make comprehensive judgments mainly from five aspects: system hardware, network equipment, operating system configuration, application program architecture and program code;
If it is found to be a system hardware, network device or operating system configuration problem, Linux operation and maintenance personnel can solve it independently according to the situation;
If it is found to be a program structure problem, it needs to be submitted to the program architect;
If it is found to be a program code execution problem, it will be handed over to the developer for code optimization. This completes a system performance optimization process.
04. Tuning summary
System performance optimization is a wide-ranging, tedious and long-term work. Finding the root cause of performance problems is often the most difficult part. Once the cause of the problem is found, the performance problem will be solved. Therefore, the idea of solving problems becomes very important.
For example, a website system under the Linux system, users report that the website access speed is very slow and sometimes cannot be accessed.
The first step is to detect the network. You can use the ping command to check whether the domain name resolution of the website is normal. At the same time, whether the delay of pinging the server address is too long, etc., in this way, first eliminate possible problems in the network;
If there is no problem with the network, go to the second step to check the memory usage of the Linux system. Because the response speed of the website is slow, it is generally related to the memory. Use commands such as free and vmstat to judge whether the memory resources are in short supply;
If there is no problem with memory resources, go to the third step to check the load status of the system CPU. You can comprehensively judge whether the CPU is overloaded through the output of sar, vmstat, top and other commands. If there is no problem with the CPU;
Continue to the fourth step to check whether there is a bottleneck in the disk I/O of the system. You can check the read and write performance of the disk through commands such as iostat and vmstat. What to do is to check whether there is a problem with the program itself.
Through this kind of thinking, layer-by-layer detection and step-by-step investigation, performance problems will have "nowhere to hide", and it will become very simple to find the links where performance problems occur.
The following is the supporting information. For friends who do [software testing], it should be the most comprehensive and complete preparation warehouse. This warehouse also accompanied me through the most difficult journey. I hope it can help you too!
Software testing interview applet
The software test question bank maxed out by millions of people! ! ! Who is who knows! ! ! The most comprehensive quiz mini program on the whole network, you can use your mobile phone to do the quizzes, on the subway or on the bus, roll it up!
The following interview question sections are covered:
1. Basic theory of software testing, 2. web, app, interface function testing, 3. network, 4. database, 5. linux
6. web, app, interface automation, 7. performance testing, 8. programming basics, 9. hr interview questions, 10. open test questions, 11. security testing, 12. computer basics
Information acquisition method:
