Nginx multi-level IP pass-through in reverse proxy

IP passthrough

Why do IP passthrough

In using CDNthe site do accelerate the static resources to speed up, when the user requests a static resource did not hit, then CDNwill go to the source station requested content, then the time to access the source station IPfor the CDNnode IP, not only that, probably through our WAFfirewall and front-end load balancing ( SLBback) are less likely to get to the real user IPinformation, if we are to count the user's access IPand regions becomes more troublesome, because it may not be true IP, what mechanism will have to use a user IPpassed to the back end of the final The application server job.

lab environment

Access Process

liucheng-img

Host computer	IP configuration	Remark
`Chrome`	`10.0.0.1`	`Windows`Browser
`LB-01`	`10.0.0.5`	Agent
`LB-02`	`10.0.0.6`	Two agents
`LB-03`	`10.0.0.7`	Three agents
`WEB`	`10.0.0.8`	WEB Host

Common ways

X-Real-IP

description

Each HTTPjoin request header X-Real-IPinformation, if only the presence of a proxy class, the client parameter is indeed the real IPtime, but if there are multiple level proxy, the proxy information for the parent IPinformation, and the user can not get the real IP, therefore the method has been deprecated.

nginx configuration

#LB-01一级代理配置
server {
    listen 80;
    server_name ip.test.com;
    
    location / {
        proxy_pass http://10.0.0.6;
        proxy_http_version 1.1;
        Proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}
#LB-02二级代理配置
server {
    listen 80;
    server_name ip.test.com;
    
    location / {
        proxy_pass http://10.0.0.7;
        proxy_http_version 1.1;
        Proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}
#LB-03三级代理配置
server {
    listen 80;
    server_name ip.test.com;
    
    location / {
        proxy_pass http://10.0.0.8;
        proxy_http_version 1.1;
        Proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}
#WEB主机配置
server {
    listen 80;
    server_name ip.test.com;
    
    root /wwwroot;
    index index.php;
    location ~ \.php$ {
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }
}

WEB Host Test Script

#文件名：index.php
#放到测试目录/wwwroot内
<?php
    $IP_ADDRESS = getenv("HTTP_X_REAL_IP");
    echo "SOURCE IP ADDRESS: $IP_ADDRESS";
?>

Note: After configuration, save all configuration files, and then start all nginxservices. If you windowsmust be in the test C:\Windows\System32\drivers\etc\hoststo join the local resolution records in the file, as follows:

10.0.0.5    ip.test.com

Wireshark packet capture information

Wireshark-pack

HTTP header information

GET / HTTP/1.1
Host: ip.test.com
X-Real-IP: 10.0.0.6
Connection: close
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9
Cookie: UM_distinctid=16afc8d323f94-0360d7ebcc9bae-3d644509-18e414-16afc8d324063f

chrome open test

X-Real-IP

Conclusion: the Wiresharktrace stream and Chromevisible test, X-Real-IPand can not get to the real client IPaddress, single-level proxy, if the case, you can get to the correct client IP, if there is multi-level proxy Xiecai.

X-Forwarded-For

description

在每个HTTP请求头中加入X-Forwarded-For信息，若只存在1级的代理服务器，则该参数为客户端的真实IP,若是存在多级代理时，每经过一级代理服务器，则追加上级代理服务的IP，可以获取到真实的用户IP，但若是遇到伪造的X-Forwarded-For信息或第一级代理未启用X-Forwarded-For都不能获取到真实用户IP，此法是目前比较常用的方法，但更推荐使用nginx_http_realip_module模块添加可信代理的方法。

nginx配置

#LB-01一级代理配置
server {
    listen 80;
    server_name ip.test.com;
    
    location / {
        proxy_pass http://10.0.0.6;
        proxy_http_version 1.1;
        Proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        #可以尝试注释此处的，看看最终能否获取到真实客户端IP
    }
}
#LB-02二级代理配置
server {
    listen 80;
    server_name ip.test.com;
    
    location / {
        proxy_pass http://10.0.0.7;
        proxy_http_version 1.1;
        Proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}
#LB-03三级代理配置
server {
    listen 80;
    server_name ip.test.com;
    
    location / {
        proxy_pass http://10.0.0.8;
        proxy_http_version 1.1;
        Proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}
#WEB主机配置
server {
    listen 80;
    server_name ip.test.com;
    
    root /wwwroot;
    index index.php;
    location ~ \.php$ {
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }
}

WEB主机测试脚本

#文件名：index.php
#放到测试目录/wwwroot内
<?php
    $IP_ADDRESS = getenv("HTTP_X_FORWARD_FOR");
    echo "SOURCE IP ADDRESS: $IP_ADDRESS";
?>

注：记得重载nginx配置。

Wireshark抓包信息

HTTP报头信息

GET / HTTP/1.1
Host: ip.test.com
X-Forwarded-For: 10.0.0.1, 10.0.0.5, 10.0.0.6
Connection: close
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9
Cookie: UM_distinctid=16afc8d323f94-0360d7ebcc9bae-3d644509-18e414-16afc8d324063f

chrome打开测试

结论：由Wireshark追踪流和Chrome的测试可见，X-Forwarded-For可以获取到真实的客户端IP地址，即便是在多级代理下，也可以获取到正确的客户端IP，但如果某台代理未设置X-Forwarded-For，后端应用服务器可能并不能获取到正确的客户端IP。

nginx_http_realip_module

描述

通过预定义可信任的代理主机的IP的方式（可信代理主机默认都会加入X-Forwarded-For信息），根据些X-Forwarded-For的信息，从右到左，滤除掉这些可信代理的IP信息，最终获取到的就是真实客户端IP信息，此信息替换$remote_addr这个变量，使最终客户端IP在WEB后端应用服务器的access.log第一列显示，方便取值分析。

nginx官方描述

nginx配置

#LB-01一级代理配置
server {
    listen 80;
    server_name ip.test.com;
    
    location / {
        proxy_pass http://10.0.0.6;
        proxy_http_version 1.1;
        Proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        #此为可信任的代理服务器，故需要加入这条配置
    }
}
#LB-02二级代理配置
server {
    listen 80;
    server_name ip.test.com;
    
    location / {
        proxy_pass http://10.0.0.7;
        proxy_http_version 1.1;
        Proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        #此为可信任的代理服务器，故需要加入这条配置
    }
}
#LB-03三级代理配置
server {
    listen 80;
    server_name ip.test.com;
    
    location / {
        proxy_pass http://10.0.0.8;
        proxy_http_version 1.1;
        Proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        #此为可信任的代理服务器，故需要加入这条配置
    }
}
#WEB主机配置，使用此模块是在后端WEB应用服务器添加参数
server {
    listen 80;
    server_name ip.test.com;
    #定义可信任的代理服务器地址
    set_real_ip_from  10.0.0.5;
    set_real_ip_from  10.0.0.6;
    set_real_ip_from  10.0.0.7;
    #指定从哪个HTTP报头里检索IP信息
    real_ip_header    X-Forwarded-For;
    #递归排除每个代理服务器的IP
    real_ip_recursive on;
    
    root /wwwroot;
    index index.php;
    location ~ \.php$ {
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }
}

WEB主机测试脚本

#文件名：index.php
#放到测试目录/wwwroot内
<?php
    $IP_ADDRESS = getenv("HTTP_X_FORWARD_FOR");
    echo "SOURCE IP ADDRESS: $IP_ADDRESS";
?>

注：记得重载nginx配置。

Wireshark抓包信息

HTTP报头信息

GET / HTTP/1.1
Host: ip.test.com
X-Forwarded-For: 10.0.0.1, 10.0.0.5, 10.0.0.6
Connection: close
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9
Cookie: UM_distinctid=16afc8d323f94-0360d7ebcc9bae-3d644509-18e414-16afc8d324063f

chrome打开测试

nginx-http-realip-module

WEB应用服务器的访问LOG

nginx accesslog

Conclusion: In order to solve X-Forwarded-Forcamouflage possible IPproblems, we use a back-end nginx_http_realip_modulemodule of set_real_ip_fromkeywords to add credibility IPand from the X-Forwarded-Forfilter head out, get to the ultimate success of the client IP, and also to do some of the log format to replace the the client IPis shown in access.logthe first column, to facilitate the final value analysis, to note here is that we must add the correct and absolutely credible proxy server IP, otherwise the end result is still a problem.