"Back-end service performance optimization guide"
table of Contents
First, the server operating system 2
Two, Web server / reverse proxy server ( Nginx ) 2
Third, the application server ( Tomcat ) 2
Fourth, the application log configuration 2
Fifth, a database ( MySQL ) 3
Sixth, the database connection pool 3
Seven, RESTfull service calls 4
First, the server operating system
1, all processes shared global limit
View System handles the maximum number of cat / proc / sys / fs / file-nr
Review: /etc/sysctl.conf
fs.file-max = 100000
net.ipv4.ip_conntrack_max = 100000
net.ipv4.netfilter.ip_conntrack_max = 100000
2, the largest single process handles
Server configuration defaults to 1024
Check the maximum number of processes ulimit -u
view the maximum number of handles processes ulimit -n
modification: /etc/security/limits.conf
* Soft nofile = 32768
* Hard nofile = 65536
The maximum open handle for all users a single thread, soft limit 32768 , hard limit 65536 ;
3, tcp connection survival time, latency connection is closed
Modify /etc/sysctl.conf , add the following lines:
# default of TIMEOUT period
net.ipv4.tcp_fin_timeout = 30
# Kai reuse, allowing TIME_WAIT sockets reused for a new TCP connection by default 0 represents off
net.ipv4.tcp_tw_reuse = 1
# Open TCP connection TIME_WAIT sockets quickly recovered by default 0 represents off
net.ipv4.tcp_tw_recycle = 1
4, using the uname -a command to view the system kernel version, make sure that the linux kernel version higher than 2.6.28 , this release linux support efficient asynchronous IO model epoll , the JDK the NIO development kit contains a pair when before the operating system version number Analyzing ( Linux> 2.6 invoked when epoll the native method), Java NiO is achieved by way of IO model is determined by the operating system.
Two, web server / reverse proxy server ( nginx )
nginx.conf profile:
5, nginx single worker process to establish the maximum number of external connections
worker_connections 16000
6, nginx maximum number of processes can open handles
worker_rlimit_nofile 65535
7, nginx worker processes
worker_processes 8 [ generally set CPU double the number of nuclei ]
Third, the application server ( Tomcat )
1, make the connector is configured to NIO mode, and set the appropriate maximum number of connections ( springboot embedded tomcat default is NIO , the maximum number of connections max-connections default 10000 )
2, the adjustment tomcat number of worker thread queue Queue Length
server.tomcat.max-threads=800
server.tomcat.accept-count=200
Fourth, the application log configuration
1, application logs the current best practice sl4j + the logback , the code should be used sl4j log facade, prohibits direct use log4j concrete realization.
2, in some highly concurrent services should consider doing log asynchronous optimized to reduce IO blocking time to improve CPU utilization:
<appender name="syslog"
class="ch.qos.logback.core.rolling.RollingFileAppender">
......
</appender>
<appender name="ASYNC_ROLLING_FILE" class="ch.qos.logback.classic.AsyncAppender">
<queueSize>256</queueSize>
<includeCallerData>true</includeCallerData>
<appender-ref ref="syslog"/>
</appender>
<root level="INFO">
<!-- <appender-ref ref="STDOUT"/> -->
<appender-ref ref="ASYNC_ROLLING_FILE"/>
</root>
Logback provided AsyncAppender using the built-in asynchronous log output queues, default 256 , in order to ensure performance, over the length of 80% of the start dropping error log outside.
| Property name |
Types of |
Defaults |
description |
| queueSize |
int |
256 |
Built-in BlockingQueue maximum capacity |
| discardingThreshold |
int |
-1 |
By default, when a capacity higher than a threshold blockingQueue (80%), discards the ERROR logging level, you do not want to discard logs (both stored each time the total amount), it can be set to 0, but if the queue is full when discards enqueue all log information, it is recommended to set 1 (the default value). Logs can be discarded as normal, it can greatly improve performance and save critical ERROR log.
|
| includeCallerData |
boolean |
false |
The cost of extracting the caller data is quite expensive. To improve performance, by default, when the event is added to the Queue, event data associated with the caller is not extracted. By default, only the "cheap" data, such as thread name. For example line number in the log if necessary should output the value set to true. Measured includeCallerData = true will bring some performance degradation, but still far tps highly concurrent synchronization log way to high. |
Fifth, a database ( MySQL )
In doing stress tests, I found the limit on the number of database connections, as well as some long slow query, the following configuration to solve the major problems in this area
The following parameters can be modified /etc/my.cnf database, the database into effect after the restart.
1, the maximum number of connections and the maximum number of users connected to the database is provided as follows,
= 1000 max_connections (default 151)
max_user_connections=1000
2 , the query generated during the temporary table size can be set tmp_table_size, max_heap_table_size join query when doing used. Parameter max_heap_table_size than tmp_table_size hours, the system will max_heap_table_size upper limit value as the maximum memory temporary tables, is larger than this, the rewritable hard disk , increasing the heap size of the table, the equivalent effect can join query speed .
Such as:
200M = the tmp_table_size (default 16M)
max_heap_table_size=500M（默认16M）
. 3 , as read_buffer_size : is MySQL read into the buffer size , a request for a table to assign a sequential scans read into the buffer, MySQL allocates some memory buffer for it. read_buffer_size variable controls the size of the buffer. If the request for a sequential scan of the table are very frequent, and such that frequent scanning too slow, its performance can be improved by increasing the value of the variable, and memory buffer size.
The following settings:
sort_buffer_size = 8M
read_buffer_size =8M
read_rnd_buffer_size = 8M
join_buffer_size , that is with the table join association , such as: join_buffer_size = 8M .
Sixth, the database connection pool
In tomcat-jdbc connection pool, for example, other connection pools as DBCP , the c3p0 , Druid and the like;
1, core portal user identity database connection pool parameters to modify several applications : user maximum connection for maxActive maximum idle maxIdle 300 , minimum idle minIdle , the initial number initialSize 60 ; Portal \ Core \ Identity : maximum connection, maximum idle 100 , minimum idle The initial number of 20 ; other applications initial 5 maximum 20
2, in a production environment testOnBorrow turn off and testOnReturn ( set to false) , mainly through the connection failure testWhileIdle guaranteed .
Seven, Restfull service call
1, hotspot service does not allow direct use jdk own java.net.URLConnection be http client calls
2, is recommended to use the Apache HttpClient , okhttp such as connection pooling mode http library rest service calls, reduce overhead repeatedly to establish the connection. Based spring technology stack applications better practice is to use restTemplate to configure the above-described implementations.
3, Apache HttpClient Fluent API underlying Based on own httpclient connection pool mode, wherein the code written dead maxTotal and maxPerRoute all requests to use a common connection pool, a total of 200 connections, each destination up to 100 connections. There will be a bottleneck in high concurrency, hot application is not recommended for use.