openresty Development Series 40 - nginx + lua countries achieve access to information where the client ip
in order to achieve business systems for different regions IP access, show business interface contains the different areas of information. Under the system needs a lot of information based on IP user access, determine the user may access area, serving personalized content for different regions. Based on this scheme in a high performance Openresty1.13.6.1 CentOS7.6 environment.
An Introduction
To confirm the home, you can usually use some online inquiry service is by IP address, but use the online service query potential performance issues, and to access external services through lua additional amount of code. To implement the query by the local library GeoIP is a relatively good program, GeoIP offers free and fee-based services (https://www.maxmind.com/en/home), using a regularly updated database GeoIP most cases to meet the basic needs .
Therefore, it is possible to achieve fast openresty the location queries and redirects users to access interfaces through local GeopIP lua library database.
Preparing the environment
a: OpenResty installation
OpenResty easily Nginx and all kinds of commonly used lua library packaged and released, you can easily reference https://openresty.org/en/installation.html documentation from source compiler installation. The main installation steps as follows:
the tar -xvf openresty-VERSION.tar.gz
CD-VERSION openresty /
./configure -j2 --prefix = / usr / local / openresty
the make -j2
the sudo the make the install
Vim # / etc / Profile
the PATH = Export / usr / local / openresty / bin: $ the PATH
VERSION here is OpenResty specific version number, currently 1.13.6.1, compile and install the version information can be viewed through the following command:
[root @ Node5 conf] # / usr / local / openresty / bin / openresty -V
nginx Version: openresty / 1.13.6.1
Built by gcc 4.8.5 20,150,623 (Red Hat 4.8.5-36) (GCC)
Built with OpenSSL FIPS-1.0.2k 26 Jan 2017
TLS SNI Support enabled
configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt=-O2 --add-module=../ngx_devel_kit-0.3.0 --add-module=../echo-nginx-module-0.61 --add-module=../xss-nginx-module-0.05 --add-module=../ngx_coolkit-0.2rc3 --add-module=../set-misc-nginx-module-0.31 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.07 --add-module=../srcache-nginx-module-0.31 --add-module=../ngx_lua-0.10.11 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.18 --add-module=../redis2-nginx-module-0.14 --add-module=../redis-nginx-module-0.3.7 --add-module=../rds-json-nginx-module-0.15 --add-module=../rds-csv-nginx-module-0.08 --add-module=../ngx_stream_lua-0.0.3 --with-ld-opt=-Wl,-rpath,/usr/local/openresty/luajit/lib --with-pcre --with-http_gzip_static_module --with-http_realip_module --with-http_geoip_module --with-http_ssl_module --with-http_stub_status_module --with-stream --with-stream_ssl_module
openresty packet contains its own maintenance tool opm, the tool uses perl implementation depends MD5, need to perform yum install -y perl-Digest-MD5 mounted
two: GeoIP2 installation
1. From https://dev.maxmind.com/geoip/geoip2 / geolite2 / save GeoIP2 download MaxMind database format to the local server, database files will be saved to GeoLite2-City.mmdb under / usr / local / openresty directory
2.GeoIP2 lua libraries are installed, GeoIP2 lua library is located https://github.com / anjia0532 / lua-resty-maxminddb , the following command can be easy to install:
# / usr / local / openresty / bin / OPM GET / Lua-Resty-maxminddb anjia0532
3.GeoIP2 Lua library depends dynamic libraries are installed: lua implemented library dependency libmaxminddb efficient access to the mmdb. The library needs to be compiled and added to openresty access environment.
Https://github.com/maxmind/libmaxminddb/releases package downloaded from the respective source to deploy locally compiled
Basic Compiler steps:
the tar-XF libmaxminddb 1.3.2.tar.gz
CD-libmaxminddb 1.3.2
./configure
the make
the make Check
install the make
ldconfig
By default, the above operation will libmaxminddb.so deployed to the / usr / local / lib directory, in order to allow access openresty, can be copied to the directory openresty, ldconfig or updated by the following steps.
-c SH "echo / usr / local / lib >> /etc/ld.so.conf.d/local.conf"
ldconfig
Three: Configure openresty nginx environment.
1, load the appropriate configuration openresty nginx lua and dynamic library, you need to add the following command at http segments, wherein the path ;; default library represents:
lua_package_path "/usr/local/openresty/lualib/?.lua ;;";
lua_package_cpath "/usr/local/openresty/lualib/?.so ;;";
2, designated lua processing requests. For easy and intuitive, as an example of the configuration specified nginx.conf / ipinfo url start request processed by the script /usr/local/lua/getipinfo.lua, there is no complex requests and do other processing variables.
lua_code_cache off; parameter is only used for testing, production shall be set to ON;
add the location server as part of nginx.conf:
LOCATION / ipinfo {
default_type "text / HTML";
charset UTF-. 8;
/usr/local/lua/getipinfo.lua content_by_lua_file;
}
# get lua script ip belongs:
# vim /usr/local/lua/getipinfo.lua ngx.say("<br>IP location query result:<hr><br>") local cjson=require 'cjson' local geo=require 'resty.maxminddb' local arg_ip=ngx.var.arg_ip local arg_node=ngx.var.arg_node ngx.say("IP:",arg_ip,", node:",arg_node,"<br>") if not geo.initted() then geo.init("/usr/local/openresty/GeoLite2-City.mmdb") end local res,err=geo.lookup(arg_ip or ngx.var.remote_addr) if not res then ngx.say("Please check the ip address you provided: <div style='color:red'>",arg_ip,"</div>") ngx.log(ngx.ERR,' failed to lookup by ip , reason :',err) else ngx.say("Result:",cjson.encode(res)) if arg_node then ngx.say("node name:",ngx.var.arg_node, " , value:",cjson.encode(res[ngx.var.arg_node] or {})) end end
Access Interface:
http://10.11.0.215/ipinfo?ip=120.76.101.211&node=city
IP LOCATION Query the Result:
IP: 120.76.101.211, the Node: City
Result:{"city":{"geoname_id":1808926,"names":{"en":"Hangzhou","ru":"Ханчжоу","fr":"Hangzhou","pt-BR":"Hangzhou","zh-CN":"杭州","es":"Hangzhou","de":"Hangzhou","ja":"杭州市"}},"subdivisions":[{"geoname_id":1784764,"names":{"en":"Zhejiang","fr":"Province de Zhejiang","zh-CN":"浙江省"},"iso_code":"ZJ"}],"country":{"geoname_id":1814991,"names":{"en":"China","ru":"Китай","fr":"Chine","pt-BR":"China","zh-CN":"中国","es":"China","de":"China","ja":"中国"},"iso_code":"CN"},"registered_country":{"geoname_id":1814991,"names":{"en":"China","ru":"Китай","fr":"Chine","pt-BR":"China","zh-CN":"中国","es":"China","de":"China","ja":"中国"},"iso_code":"CN"},"location":{"time_zone":"Asia\/Shanghai","longitude":120.1619,"accuracy_radius":50,"latitude":30.294},"continent":{"geoname_id":6255147,"names":{"en":"Asia","ru":"Азия","fr":"Asie","pt-BR":"Ásia","zh-CN":"亚洲","es":"Asia","de":"Asien","ja":"アジア"},"code":"AS"}} node name:city , value:{"geoname_id":1808926,"names":{"en":"Hangzhou","ru":"Ханчжоу","fr":"Hangzhou","pt-BR":"Hangzhou","zh-CN":"杭州","es":"Hangzhou","de":"Hangzhou","ja":"杭州市"}}
格式化输出：
{
"city": {
"geoname_id": 1808926,
"names": {
"en": "Hangzhou",
"ru": "Ханчжоу",
"fr": "Hangzhou",
"pt-BR": "Hangzhou",
"zh-CN": "杭州",
"es": "Hangzhou",
"de": "Hangzhou",
"ja": "杭州市"
}
},
"subdivisions": [{
"geoname_id": 1784764,
"names": {
"en": "Zhejiang",
"fr": "Province de Zhejiang",
"zh-CN": "浙江省"
},
"iso_code": "ZJ"
}],
"country": {
"geoname_id": 1814991,
"names": {
"en": "China",
"ru": "Китай",
"fr": "Chine",
"pt-BR": "China",
"zh-CN": "中国",
"es": "China",
"de": "China",
"ja": "中国"
},
"iso_code": "CN"
},
"registered_country": {
"geoname_id": 1814991,
"names": {
"en": "China",
"ru": "Китай",
"fr": "Chine",
"pt-BR": "China",
"zh-CN": "中国",
"es": "China",
"de": "China",
"ja": "中国"
},
"iso_code": "CN"
},
"location": {
"time_zone": "Asia\/Shanghai",
"longitude": 120.1619,
"accuracy_radius": 50,
"latitude": 30.294
},
"continent": {
"geoname_id": 6255147,
"names": {
"en": "Asia",
"ru": "Азия",
"fr": "Asie",
"pt-BR": "Ásia",
"zh-CN": "亚洲",
"es": "Asia",
"de": "Asien",
"ja": "アジア"
},
"code": "AS"
}
}
node name: city, value: {
"geoname_id": 1808926,
"names": {
"en": "Hangzhou",
"ru": "Ханчжоу",
"fr": "Hangzhou",
"pt-BR": "Hangzhou",
"zh-CN": "杭州",
"es": "Hangzhou",
"de": "Hangzhou",
"ja": "杭州市"
}
}
Get online environment client ip country example:
nginx.conf primary configuration, introducing ip library
[root @ gdpr04: ~] # cat /usr/local/nginx/conf//nginx.conf
#user apache; worker_processes 8; #error_log logs/error.log; #error_log logs/error.log notice; #error_log logs/error.log info; #pid logs/nginx.pid; pid /data/www/logs/nginx.pid; worker_rlimit_nofile 65535; events { use epoll; worker_connections 10240; } http { include mime.types; default_type application/octet-stream; #set_real_ip_from 0.0.0.0/0; #real_ip_header X-Forwarded-For; #proxy_set_header Host $host; #proxy_set_header X-Real-IP $remote_addr; #proxy_set_header X-Forwarded-For $http_x_forwarded_for; #proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_headers_hash_max_size 51200; proxy_headers_hash_bucket_size 6400; ssl_session_cache shared:SSL:10m; ssl_session_timeout 10m; # fastcgi_cache_path /usr/local/nginx/fastcgi_cache levels=1:2 keys_zone=TEST:10m inactive=5m; fastcgi_connect_timeout 300; fastcgi_send_timeout 300; fastcgi_read_timeout 300; fastcgi_buffer_size 64k; fastcgi_buffers 4 64k; # fastcgi_busy_buffers_size 128k; fastcgi_temp_file_write_size 128k; # fastcgi_cache TEST; #fastcgi_cache_valid 200 302 1h; # fastcgi_cache_valid 301 1d; #fastcgi_cache_valid any 1m; # fastcgi_cache_min_uses 1; #geoip_country /usr/local/nginx/conf/GeoIP.dat; #fastcgi_param GEOIP_COUNTRY_CODE $geoip_country_code; geoip2 conf/GeoIP2/GeoIP2-Country.mmdb { auto_reload 5m; $geoip2_metadata_country_build metadata build_epoch; $geoip2_data_country_code source=$remote_addr country iso_code; $geoip2_data_country_name country names en; } geoip2 conf/GeoIP2/GeoIP2-City.mmdb { $geoip2_data_city_name city names en; } fastcgi_param COUNTRY_CODE $geoip2_data_country_code; fastcgi_param COUNTRY_NAME $geoip2_data_country_name; fastcgi_param CITY_NAME $geoip2_data_city_name; open_file_cache max=204800 inactive=20s; open_file_cache_min_uses 1; open_file_cache_valid 30s; #log_format main '$remote_addr - $remote_user [$time_local] "$request" ' # '$status $body_bytes_sent "$http_referer" ' # '"$http_user_agent" "$http_x_forwarded_for"'; # log_format main '[$time_local] $remote_addr $status $request_time $body_bytes_sent "$request" "$http_referer" $upstream_addr $http_x_real_ip $http_x_forwarded_for $http_user_agent $request_filename'; log_format main '$remote_addr - - [$time_local] - - "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time '; # log_format test '[$fastcgi_script_name] [$time_local] $remote_addr $status $request_time $body_bytes_sent "$request" "$http_referer" $upstream_addr $http_x_real_ip $http_x_forwarded_for $http_user_agent '; log_format error '$remote_addr - - [$time_local] - - "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time '; #access_log logs/access.log main; sendfile on; tcp_nodelay on; keepalive_timeout 70; #----for upload file client_max_body_size 8M; client_body_buffer_size 2M; #--- for resolve 400 error client_header_buffer_size 64k; large_client_header_buffers 4 64k; proxy_connect_timeout 60s; proxy_read_timeout 60s; #60s内后端服务器需要返回成功 proxy_send_timeout 60s; proxy_buffer_size 16k; proxy_buffers 4 32k; proxy_busy_buffers_size 64k; proxy_temp_file_write_size 64k; gzip on; gzip_vary off; gzip_min_length 1k; gzip_buffers 4 16k; gzip_http_version 1.0; gzip_comp_level 3; gzip_disable "MSIE [1-6]\."; gzip_types text/plain text/css text/javascript application/x-javascript text/xml application/xml; fastcgi_intercept_errors on; ssi on; ssi_silent_errors on; #ssi_types text/shtml; expires 30d; server_names_hash_bucket_size 20480; #if_modified_since before; #limit_req_zone $binary_remote_addr zone=all_zone:10m rate=3r/s; #limit_req zone=all_zone burst=2 nodelay; limit_req_zone $binary_remote_addr $host $request_uri zone=all_zone:30m rate=4r/s; geo $white_ip { ranges; default 0; 1.1.1.1-1.1.1.254 1; 192.168.254.1-192.168.254.254 2; } limit_req_whitelist geo_var_name=white_ip geo_var_value=1; limit_req_whitelist geo_var_name=white_ip geo_var_value=2; limit_req_whitelist geo_var_name=white_ip geo_var_value=3; limit_req_whitelist geo_var_name=white_ip geo_var_value=4; limit_req_whitelist geo_var_name=white_ip geo_var_value=5; limit_req_whitelist geo_var_name=white_ip geo_var_value=6; # upstream php_pool{ # ip_hash; # server unix:/tmp/php-cgi.sock; # server 192.168.254.126:9000 max_fails=0 fail_timeout=30s weight=3; # keepalive 32; # keepalive_timeout 30s; # check interval=3000 rise=2 fall=5 timeout=1000 type=tcp port=9000; # check_keepalive_requests 100; # check_http_send "HEAD / HTTP/1.1\r\nConnection: keep-alive\r\n\r\n"; # check_http_expect_alive http_2xx http_3xx; # } include vhost.d/*.conf; server { listen 80 default_server; server_name localhost; location / { root /data/www/html; index index.html index.htm; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } location /ws_status { stub_status on; access_log off; } } }
# 具体vhost的配置 # cat country-info.chinasoft.com.conf server { listen 80; server_name country-info.chinasoft.com ; #access_log /data/www/logs/nginx_log/access/country-info.chinasoft.com_access.log main ; #error_log /data/www/logs/nginx_log/error/country-info.chinasoft.com_error.log ; root /data/www/vhosts/common-info.chinasoft.com/httpdocs ; index index.html index.shtml index.php ; error_page 404 403 /404.html; location /api/v1/checkeu { add_header 'Access-Control-Allow-Origin' '*'; add_header 'Access-Control-Allow-Credentials' 'true'; add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS'; add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type'; default_type 'text/plain'; content_by_lua_file '/usr/local/nginx/conf/vhost.d/checkeu.lua'; } } server { listen 443; ssl on; ssl_certificate /usr/local/nginx/conf/cert2016/iskysoft_com.crt; ssl_certificate_key /usr/local/nginx/conf/cert2016/iskysoft_com.key; ssl_session_timeout 5m; ssl_protocols TLSv1 TLSv1.1TLSv1. 2 ; ssl_ciphers "ECDHE-RSA-AES128-GCM-SHA256: ECDHE-ECDSA-AES128-GCM-SHA256: ECDHE-RSA-AES256-GCM-SHA384: ECDHE-ECDSA-AES256-GCM-SHA384: GOD-RSA-AES128-GCM-SHA256: GOD-DSS-AES128-GCM-SHA256: + kEDH AESGCM: ECDHE-RSA-AES128, SHA256: ECDHE-ECDSA-AES128, SHA256: ECDHE-RSA-AES128-Sha: ECDHE-ECDSA-AES128-Sha: ECDHE-RSA- AES256, SHA384: ECDHE-ECDSA-AES256, SHA384: ECDHE-RSA-AES256-Sha: ECDHE-ECDSA-AES256-Sha: GOD-RSA-AES128, SHA256: GOD-RSA-AES128-Sha: GOD-DSS-AES128- SHA256: GOD-RSA-AES256, SHA256: GOD-DSS-AES256-Sha: GOD-RSA-AES256-Sha:! AES128-GCM-SHA256:! AES256-GCM-SHA384:! AES128, SHA256:! AES256, SHA256: ! AES128-Sha:! AES256-Sha: AES:! CAMELLIA: DES-CBC3-Sha:! Anull:! eNULL:! Export:! DES:! RC4:! MD5:! PSK:! aECDH:! EDH-DSS- DES-CBC3-Sha: EDH-RSA-DES-CBC3-Sha:! KRB5-DES-CBC3-Sha " ; ssl_prefer_server_ciphers from; server_name country-info.chinasoft.com; access_log /data/www/logs/nginx_log/access/country-info.chinasoft.com_access.log main ; error_log /data/www/logs/nginx_log/error/country-info.chinasoft.com_error.log ; root /data/www/vhosts/common-info.chinasoft.com/httpdocs ; index index.html ; error_page 404 403 /404.html; location /api/v1/checkeu { add_header 'Access-Control-Allow-Origin' '*'; add_header 'Access-Control-Allow-Credentials' 'true'; add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS'; add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type'; default_type 'text/plain'; content_by_lua_file '/usr/local/nginx/conf/vhost.d/checkeu.lua'; } }
# National access lua script
# cat /usr/local/nginx/conf/vhost.d/checkeu.lua
--ngx.say(" {\"c_type\":0}") local ngxmatch=ngx.re.match usercountry = ngx.var.geoip2_data_country_code --usercountry = ngx.var.geoip_country_code eopcountry = "AT|BE|BG|CY|HR|CZ|DK|EE|FI|FR|DE|GR|HU|IE|IT|LV|LT|LU|MT|NL|PL|PT|RO|SK|SI|ES|SE|GB" if not usercountry then usercountry = '' end if not usercity then usercity = '' end if ngxmatch(usercountry,eopcountry,"isjo") then ngx.say("{\"c_type\":1,\"country_code\":\""..usercountry.."\"}") else ngx.say("{\"c_type\":0,\"country_code\":\""..usercountry.."\"}") end
访问：
http://common-info.chinasoft.com/api/v1/checkeu
返回：
{"c_type":0}
{"c_type":0,"country_code":"CN"}