Recent performance has been doing the investigation, the idea is based on an analysis nginx logs, get a response time consuming url, and a request time, and then get the amount requested this time, concurrency analysis is complicated by reason of, or itself is relatively slow, if the reason is the application itself, just need to find the corresponding code, then optimize just fine
I found several reasons, the basic is the backend sql run more, a single visit can not see, but more and more people are relatively slow time, when few people 20-200 milliseconds, when many people, 200 -6000 milliseconds, then remained in the tens of milliseconds optimization, optimization strategy is to reduce unnecessary sql, plus the cache, basically solved the problem of Caton, the way to a series of commands with this record, when a summary of it
If you need to get the time of request processing, you need to add $ request_time in nginx log inside, here is my log_format
nginx.conf
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent $request_body "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" "$request_time"';
After modifying the restart nginx, nginx log to see when you can see the time it takes to process the request nginx, this time the basic is the back end of the time spent, it is possible to get the slow response of the request based on this field
Here are some that I used the command
Gets the number of pv
$ cat /usr/local/nginx/logs/access.log | wc -l
Gets the number of ip
$ cat /usr/local/nginx/logs/access.log | awk '{print $1}' | sort -k1 -r | uniq | wc -l
The most time-consuming acquisition request time, url, time-consuming, top 10, you can modify the latter figure more, all without the acquisition
$ cat /usr/local/class/logs/access.log | awk '{print $4,$7,$NF}' | awk -F '"' '{print $1,$2,$3}' | sort -k3 -rn | head -10
Get the number of requests at a certain time, the data can be removed to give the second minute, the minutes to hours to remove the data obtained, so
$ cat /usr/local/class/logs/access.log | grep 2017:13:28:55 | wc -l
Get the number of requests per minute, into a csv file output, then opened with excel, a histogram may be generated
$ cat /usr/local/class/logs/access.log | awk '{print substr($4,14,5)}' | uniq -c | awk '{print $2","$1}' > access.csv
The above diagram is excel generated, you can also use the command-line tool gnuplot generates png, and I tried it, no problem, directly report as programming, remove the manual operation section, it is convenient, but one thing is the x-axis more data, we can not excel as automatic dilution as data, so I like to use to generate excel
In fact, it is used to take up a few commands:
cat: Enter the contents of the file
grep: Filtering text
'Sort': Sorting
'Uniq': deduplication
'Awk': Text Processing
Command used in combination, a single command can be used multiple times to achieve the effect of multiple filters, the output of a preceding command after a command is input, the process flow, as long as this command to learn, how seemingly complex things, have become very simple.
The above description is a command, the following will introduce a direct output html, in fact, is the use of go-access logs to analyze nginx
cat /usr/local/nginx/logs/access.log | docker run --rm -i diyan/goaccess --time-format='%H:%M:%S' --date-format='%d/%b/%Y' --log-format='%h %^[%d:%t %^] "%r" %s %b "%R" "%u"' > index.html
go-access docker container in the form of running, as long as you install the docker, you can run directly, free installation easy
The script above, with split logs daily log, and then configure it to automatically run scripts in crontab inside, nginx can generate a report every day, the site situation at a glance, of course, there are also disadvantages, because it is not real time
Want real-time statistical data, you can use ngxtop view, it is very simple to install
$ pip install ngxtop
Running, advanced to nginx directory, then run, -c specify the configuration file, -t refresh rate in seconds
$ cd /usr/local/nginx
$ ngxtop -c conf/nginx.conf -t 1
But this real time, also need to ssh remote login, is not convenient, you can also use lua for real-time statistics, and then write an interface to display data out by lua-Module-nginx , nginx / Tengine can be used, if installed directly openresty, then it is convenient, embedded lua, without recompiling the nginx, entry openresty can see some of what I wrote articles