[All-round analysis] How to obtain the real IP of the client/server

1. Application scenarios

1. For example, in the development of the voting system, in order to prevent vote swiping, we need to limit each IP address to vote once

2. When the website is under attack such as DDoS (Distributed Denial of Service, distributed denial of service attack), we need to quickly locate the attacker's IP

3. In the process of penetration testing, it is often encountered that the website has a CDN (Content Distribution Network, content delivery network). At this time, we need to bypass the CDN to find the real IP

Two, the solution

Server (CDN)

Verify that you have a CDN

1. Use the super ping website to check whether the corresponding IP address is unique. If it is not unique, it is likely to use a CDN, such as:

http://ping.chinaz.com/
http://ping.aizhan.com/
http://ce.cloud.360.cn/

2. Use nslookup to detect. If the returned domain name resolution corresponds to multiple IP addresses, the CDN is probably used

Detection method, directly enter the command line nslookup, and then enter the corresponding domain name

There is a CDN as follows

No CDN as follows

Method 1: Query historical DNS records

View the historical records of IP and domain name binding, there may be records before using CDN, relevant query website:

DNS query: https://dnsdb.io/zh-cn/

Online domain name information query: http://toolbar.netcraft.com/site_report?url=

DNS, IP and other queries: http://viewdns.info/

CDN query IP: https://tools.ipip.net/cdn.php

Method 2: Query the subdomain name

After all, CDN is still not cheap, so many webmasters may only do CDN for the main site or sub-sites with heavy traffic, and many sub-sites of small sites are on the same server or in the same segment C as the main site. You can find the real IP of the website by querying the IP corresponding to the subdomain name.

dnsdb query (https://www.dnsdb.io/)

Just enter baidu.com type: A to collect Baidu's subdomain name and ip

Method 3: Cyberspace Search Engine

The cyberspace search engine is designed to solve the information collection process that individuals must carry out every time they conduct a penetration test. By scanning the entire network , the basic data is formatted and stored for on-demand search and use by security personnel, which improves the security personnel. work efficiency.

Commonly used cyberspace search engines: fofa , shodan , zoomeye, censys

Each search engine has a corresponding search syntax. For example, fofa, you only need to enter: title: "the title keyword of the website" or body: "the body characteristics of the website" to find out the ip domain name with these keywords included in fofa , get the real ip that can get the website, other search engine syntax please refer to the recommended reading

Method 4: Use the SSL certificate to find the real original IP

Suppose you host a service on xyz123boot.com, the origin server IP is 136.23.63.44. And CloudFlare will provide you with DDoS protection, web application firewall and some other security services to protect your services from attacks. For this to work, your web server must support SSL and have a certificate, at which point the communication between CloudFlare and your server, just like the communication between you and CloudFlare, will be encrypted (ie no flexible SSL exists). This looks secure, but the problem is that when you connect directly to the IP on port 443 https://136.23.63.44:443 the SSL certificate is exposed.

At this point, if an attacker scans 0.0.0.0/0, i.e. the entire Internet, they can get a valid certificate on xyz123boot.com on port 443, and in turn get the web server IP that was given to you.

Taking the Censys tool as an example, all we need to do is to translate the search terms described above into actual search query parameters, as follows

The search query parameters for the xyz123boot.com certificate are: parsed.names: xyz123boot.com
The query parameter to show only valid certificates is: tags.raw:trusted
The combined search parameters are: parsed.names: xyz123boot.com and tags.raw: trusted

But the disadvantage is that many illegal websites do not have SSL certificates at all and cannot obtain IP.

Method 5: Use foreign hosts to resolve domain names

Many domestic CDN manufacturers only make domestic lines for various reasons, but there may be almost no lines for foreign countries. At this time, we may be able to obtain the real IP by directly accessing foreign hosts.

Method 6: Find website vulnerabilities

1) Leakage of target sensitive files, for example: probes such as phpinfo, GitHub information leakage, etc.
2) XSS blind typing, command execution reverse shell, SSRF, etc.
3) Whether using social workers or other means, obtain the account of the target website administrator in the CDN, so as to find the real IP of the website from the configuration of the CDN.

Method 7: Website Email Subscription Search

RSS mail subscription, many websites have their own sendmail, which will send us an email. At this time, the real IP of the server will be included in the source code of the email.

Method 8: Use Zmap to scan the whole network

To find the real IP of xiaix.me website, we first obtain the IP segment from apnic, then use Zmap's banner-grab to scan out hosts with port 80 open for banner grabbing, and finally write xiaix.me in the Host in http-req.

Method 9: F5 LTM decoding method

When the server uses F5 LTM for load balancing, the real ip can also be obtained by decoding the set-cookie keyword, for example: Set-Cookie: BIGipServerpool_8.29_8030=487098378.24095.0000, first the decimal number in the first section is 487098378 Take it out, and then convert it into a hexadecimal number 1d08880a, and then from back to front, take four digits out, which is 0a.88.08.1d, and finally convert them into a decimal number 10.136.8.29, also It is the last real ip.

For more methods, please see recommended reading

client

Obtain IP method

In Java, the most direct way to get the client IP is to use request.getRemoteAddr() . This method can obtain the client IP connected to the server, and it is indeed the simplest and most effective method when there is no proxy in the middle. However, at present, Internet web applications seldom directly provide external services from application servers. Generally, there will be a layer of Nginx for reverse proxy and load balancing , and some may even have multiple layers of proxies . In the case of a reverse proxy, the IP address obtained directly using request.getRemoteAddr() is the IP address of the server where Nginx is located , not the client's IP.

Before introducing how to obtain the client IP, we need to understand the Nginx load balancing method, which is divided into the following from the protocol:

Seven-layer complex balance (HTTP/HTTPS protocol), working on the seventh layer "application layer".
Four-layer load balancing (TCP/UDP protocol), working in the "network layer" and "transport layer".

Seven layers of complex equalization

Configure Nginx (support X-Forwarded-For)

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

$proxy_add_x_forwarded_for will add the client IP directly connected to Nginx to the right of the original X-Forwarded-For value of the request. The form is as follows:

X-Forwarded-For: client1, proxy1, proxy2

X-Forwarded-For contains multiple IP addresses, and each value is separated by a comma + space. The leftmost (client1) is the IP address of the original client. If there are multiple layers of agents in the middle, each layer of agents will connect its clients End IP is appended to the right of X-Forwarded-For.

Get IP code segment

public String getClientIp(HttpServletRequest request) {
    
    
String xff = request.getHeader("X-Forwarded-For");
if (xff == null) {
    
    
return request.getRemoteAddr();
} else {
    
    
return xff.contains(",") ? xff.split(",")[0] : xff;
}
}

Disadvantage: client can forge X-Forwarded-For

A general client (such as a browser) sends an HTTP request without the X-Forwarded-For header. When the request reaches the first proxy server, the proxy server will add the X-Forwarded-For request header and set the value to The IP address of the client (that is, the first value on the far left), if there are multiple proxies later, the IP will be added to the rightmost of the X-Forwarded-For header in turn, and finally the request reaches the web application server, and the application obtains the X-Forwarded-For header. The first IP on the left of the Forwarded-For header is the real IP of the client.

However, if the client carries a forged X-Forwarded-For on the request header when initiating the request, since each subsequent layer of proxy will only append and will not overwrite, then when it finally reaches the application server, the first IP on the left will be obtained The address will be the spoofed IP of the client. That is, the IP address obtained by the getClientIp() method in the above Java code is likely to be a forged IP address

Solution: configure Nginx reverse proxy (must be directly external Nginx)

proxy_set_header X-Forwarded-For $remote_addr;

$proxy_add_x_forwarded_for will add IP to the original X-Forwarded-For, which is equivalent to giving the opportunity to forge X-Forwarded-For.

And $remote_addr is to obtain the client IP of the direct TCP connection (similar to request.getRemoteAddr() in Java), which cannot be forged , even if the client is forged, it will be overwritten instead of appended.

It should be noted that if there are multiple layers of proxies , then as long as X-Forwarded-For is configured as remoteaddr ∗ ∗ on Nginx directly accessed externally $remote_addr**, **internal layer Nginx still has to be configured as$ , otherwise the Nginx of the internal layer willoverwrite the real IP of the $client$ $.$

Layer 4 load balancing

The four-layer load balancing is based on the TCP/UDP protocol. The X-Forwarded-For way to obtain the client IP will be invalid. At this time, you can configure Nginx to enable the transparent transmission function :

stream {
    
    
	server {
    
    
		# 开启透传
        proxy_protocol on ;
    }
}

After adding this configuration, Nginx will actively send a message when establishing a TCP connection, which will contain the real IP of the client.

The format of the Proxy Protocol v2 binary header carrying the client IPv4 address is as follows:

IPv4

The format of the Proxy Protocol v2 binary header carrying the client IPv6 address is as follows:

IPv6

Replenish

Proxy IP

transparent proxy

In fact, even if the transparent proxy IP is used, the real IP will still be sent. This is generally used to break through network browsing restrictions. For example, the education network that ordinary users cannot access at will can be broken through using proxy IP.

Anonymous Proxy (Normal Anonymous Proxy and High Anonymous Proxy)

Whether the common proxy IP can find the real IP. Ordinary proxy IP is much safer than transparent proxy IP. Some servers cannot recognize the real IP, but this is not absolute. There will still be servers that recognize the use of proxy IP.

The high-anonymity proxy IP can simulate the real client browser to access the target website, it is not easy to identify the use of the proxy IP, and it will directly think that the proxy IP is the IP you use. Using a high-anonymity proxy, after advanced encryption, it is difficult to find information on the Internet, but it can still be found from reality. If you want to check the real IP, you can find out his real IP address directly from the proxy IP provider by checking the server logs and other methods.

reference article

The principle of obtaining the real ip of the client through http and the cause and prevention of using X-Forwarded-For to forge the client IP vulnerability - lovearpu - 博客园 (cnblogs.com)

11 ways to bypass CDN to find real IP - STARTURN - Blog Park (cnblogs.com)

How to Obtain the Real IP of Visitors (huaweicloud.com)

Obtain the real IP of the client through the Proxy Protocol (four-layer monitoring) (aliyun.com)

Solve the problem that Nginx proxy TCP cannot obtain the real IP of the client