table of Contents

Nginx common base module

Nginx common base module

Nginx log management

Nginx has a very flexible logging mode, each level configuration can have a separate access log. Log format by log_format command definition format.

Detailed log_format

In nginx default configuration file, log_format log format has been scheduled to die, but can we amend it?

1.log_format role is to define the log format syntax

配置语法: 包括: error.log access.log

Syntax: log_format name [escape=default|json] string ...;
Default: log_format combined "...";
Context: http

2.nginx default log format syntax is as follows:

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

3.Nginx built log format allows variable contains

$remote_addr        # 记录客户端IP地址
$remote_user        # 记录客户端用户名
$time_local         # 记录通用的本地时间
$time_iso8601       # 记录ISO8601标准格式下的本地时间
$request            # 记录请求的方法以及请求的http协议
$status             # 记录请求状态码(用于定位错误信息)
$body_bytes_sent    # 发送给客户端的资源字节数，不包括响应头的大小
$bytes_sent         # 发送给客户端的总字节数
$msec               # 日志写入时间。单位为秒，精度是毫秒。
$http_referer       # 记录从哪个页面链接访问过来的
$http_user_agent    # 记录客户端浏览器相关信息
$http_x_forwarded_for #记录客户端IP地址
$request_length     # 请求的长度（包括请求行， 请求头和请求正文）。
$request_time       # 请求花费的时间，单位为秒，精度毫秒
注:如果Nginx位于负载均衡器，nginx反向代理之后， web服务器无法直接获取到客 户端真实的IP地址。

$remote_addr获取的是反向代理的IP地址。 反向代理服务器在转发请求的http头信息中，

增加X-Forwarded-For信息，用来记录客户端IP地址和客户端请求的服务器地址。

4.access_log log configuration syntax

Syntax: access_log path [format [buffer=size] [gzip[=level]] [flush=time] [if=condition]];
access_log off;
Default: access_log logs/access.log combined;
Context: http, server, location, if in location, limit_except

5.Nginx Access log configuration practice

1) Configuration

[root@web01 conf.d]# vim gjy.conf
server {
    listen 80;
    server_name code.oldboy.com;    #域名

#将当前的server网站的访问日志记录至对应的目录，使用main格式
access_log /var/log/nginx/game.gjy.com.log main;
location / {
    root /code;
}

#当有人请求改favicon.ico时，不记录日志
location /favicon.ico {
   access_log off;
   return 200;
  }
}

2) Check the load and view

#检查
[root@web01 conf.d]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@web01 conf.d]# nginx -s reload
#查看日志目录，已经加载了配置的日志文件
[root@web01 conf.d]# ll /var/log/nginx/
total 56
-rw-r----- 1 nginx adm  26382 Aug 15 16:09 access.log
-rw-r----- 1 nginx adm  13135 Aug 15 16:51 error.log
-rw-r--r-- 1 root  root  3290 Aug 15 16:57 game.gjy.com_access.log      #配置的文件
-rw-r--r-- 1 root  root  1515 Aug 15 10:07 host.access.log
-rw-r--r-- 1 root  root   218 Aug 15 10:08 js.log

nginx logs cut

Use logrotate cut logs

[root@nginx conf.d]# cat /etc/logrotate.d/nginx
/var/log/nginx/*.log {
        daily                   # 每天切割日志
        missingok               # 日志丢失忽略
        rotate 52               # 日志保留52天
        compress                # 日志文件压缩
        delaycompress           # 延迟压缩日志
        notifempty              # 不切割空文件
        create 640 nginx adm    # 日志文件权限 属主nginx和属组adm
        sharedscripts
        
        postrotate      # 切割日志执行的命令
                if [ -f /var/run/nginx.pid ]; then
                        kill -USR1 `cat /var/run/nginx.pid`
                fi
        endscript
}

Effect after cutting the log

[root@oldboy ~]# ll /var/log/nginx/
total 4044
-rw-r----- 1 www adm  54438 Oct 12 03:28 access.log-20181012.gz
-rw-r----- 1 www adm  28657 Oct 13 03:48 access.log-20181013.gz
-rw-r----- 1 www adm  10135 Oct 12 03:28 error.log-20181130.gz
-rw-r----- 1 www adm   7452 Oct 13 03:48 error.log-20181201.gz

Nginx Contents Index

1. Contents Index module

ngx_http_autoindex_module`模块处理以斜杠字符（'/'）结尾的请求，并生成目录列表。
当`ngx_http_index_module`模块找不到索引文件时，通常会将请求传递给`ngx_http_autoindex_module`模块。

2. Configure

NginxThe default directory browsing is not allowed to download the entire list.

Syntax:    autoindex on | off;
Default:    autoindex off;
Context:    http, server, location
 
# autoindex常用参数
autoindex_exact_size off;
默认为on， 显示出文件的确切大小，单位是bytes。
修改为off，显示出文件的大概大小，单位是kB或者MB或者GB。
 
autoindex_localtime on;
默认为off，显示的文件时间为GMT时间。
修改为on， 显示的文件时间为文件的服务器时间。
 
charset utf-8,gbk;
默认中文目录乱码，添加上解决乱码。

Configuration example:

[root@web01 ~]# vim /etc/nginx/conf.d/module.conf
server {
    listen 80;
    server_name module.driverzeng.com;
    charset utf-8,gbk;

    localtion / {
        root /code;
        index index.html index.htm;
    }

    location /download {
        alias /module;
        autoindex on;  
        #显示出文件的大概大
        autoindex_exact_size off;  
        #显示文件时间为服务器的时间
        autoindex_localtime on;     
    }
}

Nginx status monitoring

ngx_http_stub_status_moduleModule provides access to basic status information.
By default, do not build this module, you should use --with-http_stub_status_modulethe configuration parameters to enable it

Configuration

Syntax: stub_status;
Default: —
Context: server, location

Configuration Nginx statusexample

#简单版
server {
    listen 80;
    server_name www.gjy.com;
    access_log off;
 
    location /nginx_status {
        stub_status;
    }
}


#配置目录索引，文件大小时间，格式，状态监控版
server {
        listen 80;
        server_name www.gjy.com;
        charset utf-8,gbk;

        localtion / {
                root /code;
                index index.html index.htm;
        }

        location /download {
                alias /module;
                autoindex on;
                autoindex_exact_size off;
                autoindex_localtime on;
        }

        location /nginx_status {
                stub_status;
        }
}

Open your browser and visit: http: //www.gjy.com/nginx_status

Active connections: 2 
server accepts handled requests
         5    5    8 
Reading: 0 Writing: 1 Waiting: 1 


Active connections  # 当前活动的连接数
accepts             # 当前的总连接数TCP
handled             # 成功的连接数TCP
requests            # 总的http请求数

Reading             # 请求
Writing             # 响应
Waiting             # 等待的请求数，开启了keepalive

# 注意, 一次TCP的连接，可以发起多次http的请求, 如下参数可配置进行验证
keepalive_timeout  0;   # 类似于关闭长连接
keepalive_timeout  65;  # 65s没有活动则断开连接

Nginx access control

IP-based access control http_access_module
based on user login authenticationhttp_auth_basic_module

nginx IP-based access control

#允许配置语法
Syntax:    allow address | CIDR | unix: | all;
Default:    —
Context:    http, server, location, limit_except
 
#拒绝配置语法
Syntax:    deny address | CIDR | unix: | all;
Default:    —
Context:    http, server, location, limit_except

1) Access control configuration example, the IP designated rejected, allowing all other

server {
    listen 80;
    server_name www.gjy.com;
    access_log off;
 
    location /nginx_status {
        stub_status;
        #拒绝10.0.0.1登陆
        deny 10.0.0.1; 
        #允许其他所有登陆
        allow all;   
    }
}

2) access control configuration example, only who can access, all of the other refuse

server {
    listen 80;
    server_name module.driverzeng.com;
    access_log off;
 
    location /nginx_status {
        stub_status；
        allow   10.0.0.0/24;
        allow   127.0.0.1;
        deny    all;
    }
}

Nginx based on user login authentication

1) configuration syntax based on the user login authentication

#访问提示字符串
Syntax: auth_basic string| off;
Default: auth_basic off;
Context: http, server, location, limit_except
 
#账户密码文件
Syntax: auth_basic_user_file file;
Default: -
Context: http, server, location, limit_except

2) Configure practice based on user login authentication

#1.需要安装httpd-tools，该包中携带了htpasswd命令
[root@web01 ~]# yum install httpd-tools
#2.创建新的密码文件, -c创建新文件 -b允许命令行输入密码
[root@web01 conf.d]# htpasswd -b -c /etc/nginx/mima gjy 123
Adding password for user gjy
[root@web01 conf.d]# cat /etc/nginx/mima
gjy:$apr1$5TQm/JtM$8Cy1rHl8AfWXz5eq2NVLN0

 
#3.nginx配置调用
server {
    listen 80;
    server_name www.gjy.com;
    access_log off;
 
    location /nginx_status {
        #动态监控
        stub_status;
        #相当于注释，可随意定义
        auth_basic "Auth access Blog Input your Passwd!";
        #创建的密码文件的名字
        auth_basic_user_file mima;
    }
}

Nginx access restrictions

In the enterprise often encounter this situation, server traffic anomaly, the load is too large, and so on. For high-volume access to malicious attacks, it will bring the waste of bandwidth, server stress, affect the business, often considered a number of connections to the same ip, requests, limited.

ngx_http_limit_conn_moduleThe module may define a keynumber of connections for each key is limited, as a source IP connections.

limit_conn_module Connection frequency limit

limit_req_module Request frequency limit

Nginx connection limit actual configuration

1) Nginxconnected to limit the configuration syntax

#模块名ngx_http_limit_conn_module
Syntax:     limit_conn_zone key zone=name:size;
Default: —
Context: http
 
Syntax:    limit_conn zone number;
Default: —
Context: http, server, location

2) Nginxconnected to limit the practice of the configuration

Disposed in a public network Nginx

#http层，设置
http{   
    # Limit settings
    limit_conn_zone $remote_addr zone=conn_zone:10m;
#server层调用
server{ 
    #连接限制，限制同时最高1个连接
    limit_conn conn_zone 1;
    }
}

3) Use abtools to conduct stress tests

[root@web01 ~]# yum install -y httpd-tools
[root@web01 ~]# ab -n 20 -c 2  http://127.0.0.1/index.html

4) nginx log results

2018/10/24 18:04:49 [error] 28656#28656: *1148 limiting connections by zone "conn_zone", client: 123.66.146.123, server: www.driverzeng.com, request: "GET / HTTP/1.0", host: "www.driverzeng.com"
2018/10/24 18:04:49 [error] 28656#28656: *1155 limiting connections by zone "conn_zone", client: 123.66.146.123, server: www.driverzeng.com, request: "GET / HTTP/1.0", host: "www.driverzeng.com"
2018/10/24 18:04:49 [error] 28656#28656: *1156 limiting connections by zone "conn_zone", client: 123.66.146.123, server: www.driverzeng.com, request: "GET / HTTP/1.0", host: "www.driverzeng.com"

Nginx configured requests limit combat

The company Nginx configuration file: [TP]

Enterprise real case:

1) NginxRequest Configuration syntax limits

#模块名ngx_http_limit_req_module
Syntax:     limit_req_zone key zone=name:size rate=rate;
Default: —
Context: http
 
Syntax:    limit_req zone number [burst=number] [nodelay];
Default: —
Context: http, server, location

2) Nginxconfigured requests limit combat

# http标签段定义请求限制, rate限制速率，限制一秒钟最多一个IP请求
http {
    limit_req_zone $binary_remote_addr zone=req_zone:10m rate=1r/s;
}
server {
    listen 80;
    server_name module.oldboy.com;
    # 1r/s只接收一个请求,其余请求拒绝处理并返回错误码给客户端
    #limit_req zone=req_zone;
 
    # 请求超过1r/s,剩下的将被延迟处理,请求数超过burst定义的数量, 多余的请求默认返回503
    limit_req zone=req_zone burst=3 nodelay;
    location / {
        root /code;
        index index.html;
    }
}

3) Use abtools to conduct stress tests

[root@oldboyedu ~]# yum install -y httpd-tools
[root@oldboyedu ~]# ab -n 20 -c 2  http://127.0.0.1/index.html

4) nginx log results

2018/10/24 07:38:53 [error] 81020#0: *8 limiting requests, excess: 3.998 by zone "req_zone", client: 10.0.0.10, server: module.driverzeng.com, request: "GET /index.html HTTP/1.0", host: "10.0.0.10"
2018/10/24 07:38:53 [error] 81020#0: *9 limiting requests, excess: 3.998 by zone "req_zone", client: 10.0.0.10, server: module.driverzeng.com, request: "GET /index.html HTTP/1.0", host: "10.0.0.10"
2018/10/24 07:38:53 [error] 81020#0: *10 limiting requests, excess: 3.998 by zone "req_zone", client: 10.0.0.10, server: module.driverzeng.com, request: "GET /index.html HTTP/1.0", host: "10.0.0.10"

5) nginx redirection request limit (extended)

In the process nginx request restrictions, we can customize a return value, which is the status code error pages.

The default is 503

1) modify the default return a status code

#http层配置模块
[root@web02 ~]# vim /etc/nginx/nginx.conf
http {
    limit_req_zone $binary_remote_addr zone=req_zone:10m rate=1r/s;
}

[root@web02 conf.d]# vim ce.conf 
server {
        listen 80;
        server_name www.gong.com;
        charset utf-8,gbk;
    limit_req zone=req_zone burst=1  nodelay;
       #修改返回状态码为：412
        limit_req_status 412;

        location / {
                root /code;
                index index.html index.htm;

    }
}

2) ugly page, redirects

[root@web02 conf.d]# vim ce.conf 
server {
        listen 80;
        server_name module.driverzeng.com;
        charset utf-8,gbk;

        location / {
                root /code;
                index index.html index.htm;
                limit_req zone=req_zone burst=3 nodelay;
                limit_req_status 478
                #重定错误页面
                error_page 478 /err.html;
        }
}
[root@web02 ~]# vim /code/err.html
<img style='width:100%;height:100%;' src=https://www.driverzeng.com/zenglaoshi/478_page.png>

Great ~~~~~

nginx connection limit is not valid restriction request

Let's take a look at http connection with the request of the agreement, first of all HTTP is built on TCP basis, in complete HTTP request needs to first establish a TCP three-way handshake (known as TCP connection), connected on the basis of the complete HTTP request.

所以多个HTTP请求可以建立在一次TCP连接之上, 那么我们对请求的精度限制，当然比对一个连接的限制会更加的有效，因为同一时刻只允许一个TCP连接进入, 但是同一时刻多个HTTP请求可以通过一个TCP连接进入。所以针对HTTP的请求限制才是比较优的解决方案。

Nginx Location

使用Nginx Location可以控制访问网站的路径,但一个server可以有多个location配置, 多个location的优先级该如何区分

location语法示例

location [=|^~|~|~*|!~|!~*|/] /uri/ { ...
}

location语法优先级排列

匹配符	匹配规则	优先级
=	精确匹配	1
^~	以某个字符串开头	2
~	区分大小写的正则匹配	3
~*	不区分大小写的正则匹配	4
!~	区分大小写不匹配的正则	5
!~*	不区分大小写不匹配的正则	6
/	通用匹配，任何请求都会匹配到	7

配置网站验证location优先级

[root@Nginx conf.d]# cat testserver.conf 
server {
    listen 80;
    server_name www.driverzeng.com;
    location / {
        default_type text/html;
        return 200 "location /";
    }
 
    location =/ {
        default_type text/html;
        return 200 "location =/";
    }
 
    location ~ / {
        default_type text/html;
        return 200 "location ~/";
    }
 
    # location ^~ / {
    #   default_type text/html;
    #   return 200 "location ^~";
    # }
}

测试location效果

# 优先级最高符号=
[root@Nginx conf.d]# curl www.gjy.com
location =/
 
# 注释掉精确匹配=, 重启Nginx
[root@Nginx ~]# curl www.gjy.com
location ~/
 
# 注释掉~, 重启Nginx
[root@Nginx ~]# curl www.gjy.com
location /

location应用场景

# 通用匹配，任何请求都会匹配到
location / {
    ...
}
 
# 严格区分大小写，匹配以.php结尾的都走这个location    
location ~ \.php$ {
    ...
}
 
# 严格区分大小写，匹配以.jsp结尾的都走这个location 
location ~ \.jsp$ {
    ...
}
 
# 不区分大小写匹配，只要用户访问.jpg,gif,png,js,css 都走这条location
location ~* .*\.(jpg|gif|png|js|css)$ {
    ...
}
 
# 不区分大小写匹配
location ~* "\.(sql|bak|tgz|tar.gz|.git)$" {
    ...
}