Detailed explanation and statistical analysis of Nginx access.log log
1. Access.log of nginx
1. Log files are generally stored under /var/log/nginx. If docker is started, you can use the host mount location, and directly use the tail -f command to view the access log.
2. The meaning of each item in access.log:
参数 说明 示例
$remote_addr 客户端地址 172.17.0.1
$remote_user 客户端用户名称 --
$time_local 访问时间和时区 [29/Dec/2022:10:17:14 +0000]
$request 请求的URI和HTTP协议 "GET /test/nginx/proxy HTTP/1.1"
$http_host 请求地址，即浏览器中你输入的地址（IP或域名） 10.1.7.33
$status HTTP请求状态 200
$upstream_status upstream状态 200
$body_bytes_sent 发送给客户端文件内容大小 38
$http_referer url跳转来源 -
$http_user_agent 用户终端浏览器等信息 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36"
$http_cookie 用户cookie信息 "grafana_session=73d13d456cb4363f8a48f5501348669e"
$ssl_protocol SSL协议版本 TLSv1
$ssl_cipher 交换数据中的算法 RC4-SHA
$upstream_addr 后台upstream的地址，即真正提供服务的主机地址 "10.1.7.33:8102"
$request_time 整个请求的总时间 0.012
$upstream_response_time 请求过程中，upstream响应时间 0.012
3. The format of access.log can be customized by yourself, and the output information format can be set in nginx.conf. You
can add a header in location to output the user agent server address
location /test/ {
#limit_req zone=allips burst=1 nodelay;
proxy_pass http://myServer/test/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header REMOTE-HOST $remote_addr;
#代理服务器地址
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 8m;
}
2. Log traffic statistics
Nginx: Anyone who makes a website with PV, UV, and independent IP
knows that they usually need to check the access data of websites such as PV and UV. Of course, if the website has a CDN, the local log of nginx is meaningless. The following is about nginx Make statistics on the log access data of the website;
- UV(Unique Visitor) : unique visitor, treat each independent Internet-connected computer (based on cookie) as a visitor, within a day (00:00-24:00), the number of visitors to your website. Visits to the same cookie within a day are only counted once
- PV (Page View) : Visits, that is, page views or clicks, each time a user visits a website is recorded once. The user visits the same page multiple times, and the number of visits is accumulated
- Statistical independent IP : the same IP address within 00:00-24:00 is only counted once, friends who do website optimization are most concerned about this
Log statistical analysis, the log content is as follows:
root@DESKTOP-0NVFL1I:/home/volumes/nginx_vts/log# tail -f access.log
172.17.0.1 - [30/Dec/2022:02:17:19 +0000] "GET /test/nginx/proxy HTTP/1.1" "10.1.7.33" 200 200 38 "-" "grafana_session=73d13d456cb4363f8a48f5501348669e" "-" "10.1.7.33:8101" 0.008 0.008
172.17.0.1 - [30/Dec/2022:02:17:20 +0000] "GET /test/nginx/proxy HTTP/1.1" "10.1.7.33" 200 200 38 "-" "grafana_session=73d13d456cb4363f8a48f5501348669e" "-" "10.1.7.33:8102" 0.016 0.016
172.17.0.1 - [30/Dec/2022:02:19:55 +0000] "GET /test/nginx/proxy HTTP/1.1" "10.1.7.33" 200 200 38 "-" "grafana_session=73d13d456cb4363f8a48f5501348669e" "-" "10.1.7.33:8101" 0.010 0.010
172.17.0.1 - [30/Dec/2022:02:19:56 +0000] "GET /test/nginx/proxy HTTP/1.1" "10.1.7.33" 200 200 38 "-" "grafana_session=73d13d456cb4363f8a48f5501348669e" "-" "10.1.7.33:8102" 0.011 0.011
Statistical interface address visits
grep /test access.log | wc -l
PV statistics
awk '{print $6}' access.log | wc -l
UV statistics
awk '{print $13}' access.log | sort -r |uniq -c |wc -l
Independent IP statistics
awk '{print $1}' access.log | sort -r |uniq -c | wc -l
3. Configure access.log to be generated on a daily basis
1. Modify the http code block in the nginx.conf configuration file to the following code
#配置按天生成access.log日志文件
map $time_iso8601 $logdate {
'~^(?<ymd>\d{4}-\d{2}-\d{2})' $ymd;
default 'date-not-found';
}
#access_log /var/log/nginx/access.log main;
access_log /var/log/nginx/access-$logdate.log main;
error_log /var/log/nginx/error.log;
2. Restart nginx, access the interface again, and check the logs, which are generated on a daily basis
root@DESKTOP-0NVFL1I:/home/volumes/nginx_vts/log# ll
-rwxrwxrwx 1 buckletime buckletime 744 Dec 30 11:14 access-2022-12-30.log
-rwxrwxrwx 1 buckletime buckletime 744 Dec 30 10:19 access.log
-rw-r--r-- 1 root root 12586 Dec 30 10:14 error.log
If the permissions are not enough, modify the log file permissions
chmod -R 777 /var/log/nginx/
Four, nginx.conf configuration
Attach the complete nginx.conf configuration
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
#开启nginx监控模块
vhost_traffic_status_zone;
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr $remote_user [$time_local] "$request" '
'"$http_host" $status $upstream_status '
'$body_bytes_sent "$http_referer" '
'"$http_cookie" "$http_x_forwarded_for" '
'"$upstream_addr" $request_time $upstream_response_time';
#配置按天生成日志文件
map $time_iso8601 $logdate {
'~^(?<ymd>\d{4}-\d{2}-\d{2})' $ymd;
default 'date-not-found';
}
#access_log /var/log/nginx/access.log main;
access_log /var/log/nginx/access-$logdate.log main;
error_log /var/log/nginx/error.log;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
#include /etc/nginx/conf.d/*.conf;
upstream myServer{
server 10.1.7.33:8101;
server 10.1.7.33:8102;
}
server {
listen 80;
server_name 10.1.7.33;
root /usr/share/nginx/html;
location /test/ {
#limit_req zone=allips burst=1 nodelay;
proxy_pass http://myServer/test/;
proxy_set_header Host $host;
#用户的真实ip
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header REMOTE-HOST $remote_addr;
proxy_set_header Cookie $http_cookie;
#用户的真实ip和经过的每一层代理服务器的ip
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 8m;
}
#nginx状态监控接口
#location /status {
# vhost_traffic_status_display;
# vhost_traffic_status_display_format html;
#}
}
}