1. Reasons for choosing nginx
(1) Support high concurrency; (official 5w concurrency, measured 3w concurrency, because of the epoll network IO model)
(2) Less memory consumption; (nginx+php-fcgi starts 10 nginx processes under 3w concurrency, consuming 150mb of memory, and 64 nginx processes consuming 1280mb of memory)
(3) Low cost; (open source and free, can be used for business, and low cost compared to hardware f5)
(4) Save bandwidth;
(5) High stability;
(6) Built-in health check function;
(7) Support reverse proxy;
(8) Support load balancing; (application layer)
2. Network IO connection method
There are 2 types: block IO (block IO) and multiplexed IO (currently used by servers);
Blocking IO: A request comes, only this thing can be done, other things cannot be done; (For example, a classmate comes to you, tells him your address, and waits at the door of the dormitory. In this process, wait until the classmate finds you. You can’t do anything during that time)
Multiplexed IO: includes select and epoll models; (select model for example, classmates come to you, tell your classmates address and are in the dormitory, etc., classmates come to the door of the dormitory but still can't find you, they will find the building manager, one by one Finding it in the dormitory is very slow; the epoll model, for example, when a classmate comes to you, tells you the address of your classmate and is in the dormitory, etc., when classmates come to the door of the dormitory, they still can’t find you. Check the register, tell your classmates on which floor and how many rooms, so that you can be found quickly)
3.nginx virtual host and related configuration
(1) nginx configuration file structure diagram
......
event{
......
}
http{
......
server{
......
}
server{
......
}
}
(2) nginx virtual host
Define server, you can do virtual host based on port, ip, domain name.
(3) nginx matching rules
location syntax: location=[=|~|~*|^~]/uri/{...}
~ Case sensitive
~* is not case sensitive
^~ prohibit expression matching
= Exact match
For example:
location =/{
#Only match / query
}
location /{
#Matches any query starting with /, but regular and some longer strings are matched first
#./document/index.html
}
location ^~ /p_w_picpaths/{
#Match any query beginning with /p_w_picpaths/, and stop searching, do not check the regular
#./p_w_picpaths/1.gif
}
location ^* .(gif|jpg|jpeg)${
#Match any file ending with gif, jpg, jpeg, but all requests for the /p_w_picpaths/ directory will be processed above, exactly matching
#./document/1.gif
}
Explanation: url and uri
url: Uniform resource locator, such as http://www.onon.com/news/a10.html
uri: uniform resource identifier, refers to a specific path, such as news/a10.html
(4) Custom error page
error_page 403 404 /40x.html;
location = /40x.html{
root /var/web/error/;
}
Explanation: Defined in the server, where / in front of /40x.html refers to the path that defines the root of the file specified by the server.
In addition, specify a special storage path for error, such as "root /var/web/error/" above, so you can put the 40x.html page in the changed directory.
The same applies to 50x errors:
error_page 500 502 503 504 /50x.html
location = /50x.html{
root /var/web/error/;
}
(5) Automatic index and alias function
location /onon{
root /web/html;
index index.html;
autoindex on;
}
Explanation: Display the file index in the /web/html directory
location /over/{
alias /data/web3/p_w_picpaths/toto/;
}
Explanation: When accessing the /data/web3/p_w_picpaths/toto/top.jpg file, you can use /over/top.jpg to access
(6) Control site access
location /onon{
root /web/html;
index index.html;
autoindex on;
deny 192.168.0.110;
allow 192.168.0.0/24;
allow 192.168.1.12;
deny all;
}
(7) Directory authentication
/usr/local/apache/bin/htpasswd -c /home/nginx/conf/authdb webadmin
Nginx auth_basic authentication uses an apache compatible password file to create a webadmin user and enter the password.
When you access /onon, you will be prompted to enter your username and password.
location /onon{
root /web/html;
index index.html;
autoindex on;
auth_basic "Enter your user and password";
auth_basic_user_file /home/nginx/conf/authdb;
deny 192.168.0.110;
allow 192.168.0.0/24;
allow 192.168.1.12;
deny all;
}
(8) nginx status check
location /nginx_status{
stub_status on;
access_log off;
}
Explanation: No need to open access log access_log; access directory/nginx_status to add authentication, see above; need to add this module when compiling: --with-http_stub_status_module
（9）nginx rewrite
The pcre library needs to be added when compiling, and prel is compatible with regular expressions;
Nginx rewrite is to achieve url/uri rewriting;
Syntax instructions: set, if, return, break, rewrite
Grammar rules:
= And! =
~ Case sensitive
～* Not case sensitive
!~and!~*
-f and !-f determine whether the file exists
-d and !-d determine whether the directory exists
-e and !-e determine whether the file or directory exists
-x and !-x determine whether the file can be executed
$1-$9 positional parameters
Global variables (the system has been defined)
$arg_PARAMETER#This variable contains the value of the variable PARAMETER in the GET request.
$args #This variable is equal to the parameters in the request line (GET request), for example foo=123&bar=blahblah;
$binary_remote_addr# Binary customer address.
$body_bytes_sent# The number of body bytes sent in response. This data is accurate even if the connection is interrupted.
$content_length#Content-length field in the request header.
$content_type#Content-Type field in the request header.
$cookie_COOKIE#cookie COOKIE variable value
$document_root#The value specified in the root command is currently requested.
$document_uri# is the same as $uri.
$host#Request host header field, otherwise it is server name.
$hostname#Set to the machine’s hostname as returned by gethostname
$http_HEADER
$is_args#If there is a $args parameter, this variable is equal to "?", otherwise it is equal to "", null value.
$http_user_agent#Client agent information
$http_cookie#Client cookie information
$limit_rate#This variable can limit the connection rate.
$query_string# is the same as $args.
$request_body_file#The temporary file name of the client request body information.
$request_method#The action requested by the client, usually GET or POST.
$remote_addr#The IP address of the client.
$remote_port#The port of the client.
$remote_user#User name that has been verified by Auth Basic Module.
$request_completion#If the request is over, set to OK. When the request is not over or if the request is not the last in the request chain, it is empty (Empty).
$request_method#GET或POST
$request_filename#The file path of the current request, which is generated by root or alias command and URI request.
$request_uri# contains the original URI of the request parameters, not the host name, such as: "/foo/bar.php?arg=baz". Can not be modified.
$scheme#HTTP method (such as http, https).
$server_protocol#The protocol used by the request, usually HTTP/1.0 or HTTP/1.1.
$server_addr#Server address, this value can be determined after completing a system call.
$server_name#Server name.
$server_port#The port number of the request to reach the server.
$uri#The current URI without request parameters, $uri does not contain the host name, such as "/foo/bar.html". This value may be inconsistent with $request_uri. $request_uri is the value sent by the browser. This value is the value after rewrite. For example, after doing internal redirects.
if instruction:
if($http_user_agent ~MSIE){
rewrite ^(.*)$ /msie/$1 break;
}
Explanation: $http_user_agent user agent, such as browser ie, firefox, etc.;
~Case sensitive, MSIE browser ie;
^What starts with, what does $ end with, .* matches any string; the $1 position character, /msie/$1 refers to the root path pointed to by the web
/Msie/$1, such as /data/web3/p_w_picpaths/toto/msie/index.html
return instruction:
If the access URL ends with " .sh" or " .bash", status code 403 will be returned
location ~.*.(sh|bash)?$
{
return 403;
}
Explanation: "~" matches case, no matter what character is in front of ".*", ".(sh|bash)?$" is marked with sh or bas, and "\" removes the special meaning of "." after it. of. The 403 error refers to the error defined by error_page.
rewrite instruction:
The last parameter flag is marked as follows:
last: complete rewrite, which is equivalent to the [L] mark of apache, and the browser url address remains unchanged;
break: After this rule is matched, it will be completed, the match will be terminated, and the browser url address will remain unchanged;
redirect: return 302 temporary redirect, the browser displays the url after the redirect;
permanent: return 301 permanent redirection, the browser displays the url after the redirection;
Case:
1) Rewrite the /bbs directory to /data
rewrite ^/bbs/?$ /data/ permanent;
location /data{
index index.html;
}
location /bbs{
rewrite ~/bbs/?$ /data/ permanent;
}
2) Different results will be obtained according to different browsers
if($http_user_agent ~MSIE){
rewrite ^(.*)$ /msie/$1 break;
}
server{
listen 80 default_server;
server_name www.onon.com;
index index.html;
root /data/www/onon;
if($http_user_agent ~Firefox){
rewrite ^(.*)$ /firefox/$1 break;
}
if($http_user_agent ~MSIE){
rewrite ^(.*)$ /msie/$1 break;
}
}
3) Prevent hotlink
The so-called anti-hotlinking is that pictures of other people's websites refer to our website, and click on the picture on other people's websites, using the traffic of our website, only quote the pictures of our website, and will not jump to our website.
location ~ .*.(gif|jpg|jpeg|png|bmp|swf|flv)$ {
valid_referers none blocked www.onon.com *.onon.com;
if ($invalid_referer) {
rewrite ^/(.*) http://www.onon.com/block.html;
}
}
4) Realize domain name redirection
Visit www.onon.com to jump to www.onbing.com
server{
listen 80 default_server;
server_name www.onon.com;
index index.html;
root /data/www/onon;
location / {
rewrite ~(.*)$ http://www.onbing.com$1 permanent;
}
}
5) URL rewrite and reverse proxy at the same time
location /news/ {
proxy_pass http://10.0.0.10;
}
location /health/ {
proxy_pass http://10.0.0.20;
}
(10) nginx log management
The syntax is as follows: log_format name format (format ....)
The custom log format is as follows:
log_format main '$remote_addr - $remote_user [$time_local] "$request"'
'$status $body_bytes_sent "$http_referer"';
'"$http_user_agent" "$http_x_forwarded_for"';
$http_x_forwarded_for records the real IP address of the remote user, not the cache/proxy server address;
$remote_user records the user name of the remote client, which is generally not recorded;
$time_local records the access time and region, and records the local time when the user visits;
$request records the requested URL and HTTP protocol, and the specific page visited;
$status records the status of the request. The status of the first successful access is 200, the status of the page is not updated again is 304, and the status of the page cannot be found is 404
$body_bytes_sent records the size of the file body content sent to the client
$http_referer records which page link is visited from, such as Baidu link;
$http_user_agent records information about the client browser
Note: The log format defined above follows access_log, and error_log follows the error level, as follows:
access_log /data0/logs/nginx/www.onon.com-access.log main；
access_log /data0/logs/$server_name.log mylogformat buffer=32k;
Explanation: main and mylogformat are the names when defining the log format; $server_name is to automatically obtain the access address, such as www.onon.com, which consumes resources; buffer=32k is to set the size of the buffer area, and only log file descriptors are recorded.
error_log /data0/logs/nginx/www.onon.com-error.log warn;
Explanation: warn is the error level, and there are 8 log levels, namely: emerg, alert, crit, err, warn, notice, info, debug
Enable log cache
For each log record, the log file first opens the file, writes the log record, and closes it immediately.
How to improve the storage path performance of log files containing variables? Open open_log_file_cache
open_log_file_cache max=1000 inactive=20s min_uses=2 valid=1m;
Explanation: max the maximum number of file descriptors. If this file descriptor is not used within the inactive setting time, this descriptor will be deleted automatically.
Min_uses is within the specified time range of inactive. If the log file exceeds the number of times it has been used, it will be included in the cache. The default is 10 seconds. How often is the valid check to see if the log file path and file name still exist. The default is 60 seconds.
nginx log split
vim /data/sh/nginxlog_cut.sh
#! /bin/bash
#nginx log storage location
logs_path="/data/logs/nglogcut/"
#Rename the log
mkdir -p ${logs_path}$(date -d "yesterday" + "%Y")/$(date -d "yesterday" + "%m")/
mv ${logs_path}access.log ${logspath}$(date -d "yesterday" + "%Y")/$(date -d "yesterday" + "%m")/access$(date -d "yesterday" + "%Y%m%d").log
Reload nginx service
service nginx reload
Add a scheduled task
chmod +x /data/sh/nginxlog_cut.sh
#crontab -l
01 01 * /bin/bash /data/sh/nginxlog_cut.sh
(11) nginx compressed output
Enable compression in the middle of http{}
Modules: HttpGzipModule and HttpGzipStaticModule
The former is used to enable the use of gzip compression during file transfer, while the latter's function is to first check whether there is a corresponding file name ending with *.gz before transferring a file to a client that supports compression in a compressed manner. Format, which avoids repeated compression and waste of resources.
For nginx web page compression and transmission, the configuration in nginx is divided into two modes: one is dynamic, real-time compression output (output while compressing), and the other is static. Find the .gz format of the file with the same name. The file is output.
Enabling the gzip function requires the support of the zlib library, --with-http_gzip_module;
Need to specify --with-http_gzip_static_module when compiling
gzip (GNU-ZIP) is a compression technology. After gzip compression, the page size can become 30% or less of the original size, so that the user browses the page at a much faster speed. The gzip compressed page needs to be supported by both the browser and the server. In fact, it is server-side compression. After being transmitted to the browser, the browser decompresses and parses it. There is no need for us to worry about the browser, because most browsers currently support parsing gzipped pages.
#vim nginx.conf
http{
gzip on;
gzip_min_length 1k;
gzip_buffer 4 16k;
gzip_http_version 1.1;
gzip_comp_level 2;
gzip_types text/plain application/x-javascript text/css application/xml;
gzip_vary on;
}
Explanation: gzip on turns on the gzip module, turns off the use of off;
gzip_min_length sets the minimum number of bytes allowed to be compressed, and the number of page bytes is obtained from the content-length of the header. The default is 0, no matter how big the page is, it will be compressed. It is recommended to set it as 1k, if it is less than 1k, the pressure may increase;
gzip_buffer sets the system to obtain several units of buffer to store the compressed result data stream of gzip. The above setting is 4 times the application memory in 16k units;
gzip_http_version identifies the version of the http protocol. Early browsers do not support gzip compression, and users will see garbled characters, so in order to support the previous version, this option is added, and this item can basically be ignored at present;
gzip_comp_level is the gzip compression ratio, divided into 1-9 levels, 1 is the smallest compression ratio and has the fastest processing speed, 9 has the largest compression ratio but the slowest processing speed (fast transmission but more CPU consumption);
gzip_types matches mime type for compression. No matter whether it is specified or not, the "text/html" type will always be compressed;
gzip_vary is related to the http header. Add a vary header for the proxy server. Some browsers support compression, and some do not, so avoid waste and compress the unsupported ones. Therefore, judge whether it is necessary according to the HTTP header of the client compression;
(12) nginx browser local cache settings
#vim nginx.conf
location ~.*.(gif|jpg|jpeg|png|bmp|swf|flv)${
expires 30d;
}
location ~.*.(js|css|jsp)?${
expires 1h;
}
(13) Set speed limit (common streaming media)
location /download {
limit_rate 256k;
proxy_pass http://www.onbing.com;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
}
location /movie {
limit_rate_after 10m;
limit_rate 100k;
}
Explanation: There is no speed limit in the first 10m, after which the speed limit is 100k.
if($http_user_agent ~Google|Yahoo|baidu){
limit_rate 20k;
}
Explanation: The speed limit is 20k when accessed through google, yahoo or baidu search engine.
(14) Reverse proxy
Forward proxy and reverse proxy? The forward direction is a visit from the inside out, and the reverse direction is a visit from the outside to the inside. Forward and reverse are a relative concept.
Forward: Typical applications provide access to the Internet for LAN clients in the firewall;
相对于用户，就是正向代理，代理用户去请求；
让nginx成为正向代理，而将外网视为整个后端在给我们提供服务。Reverse: A typical application is to provide Internet users with access to the server behind the firewall;
相对于内部的服务那么nginx就是反向代理，代理接受用户请求；http {
client_body_buffer_size 600;
client_header_buffer_size 600;
client_max_body_size 300M;
proxy_buffering on;
proxy_cache_min_uses 3;
proxy_ignore_client_abort off;
proxy_intercept_errors on;
proxy_next_upstream error timeout invalid_header;
proxy_redirect off;
proxy_connect_timeout 60;
proxy_send_timeout 600;
proxy_read_timeout 600;
proxy_buffer_size 256k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
proxy_temp_file_write_size 256k;
proxy_temp_path /usr/local/nginx/proxy_temp;
proxy_cache_path /usr/local/nginx/proxy_cache/ levels=1:2
keys_zone=cache_onon:10m inactive=10m max_size=1000M;
location /weibo {
proxy_pass http://192.168.10.213;
proxy_set_header Host $host;
proxy_cache cache_onon;
proxy_set_header X-Forward-For $remote_addr;
}
location ~.*.(gif|jpg|jpeg|png|bmp|swf|flv|js|css|html)${
#status code
proxy_cache cache_onon;
proxy_cache_valid 200 10m;
proxy_cache_valid 304 1m;
proxy_cache_valid 301 302 1h;
proxy_cache_valid any 1m;
#Hashkey value
proxy_cache_key $host$uri$is_args$args;
proxy_set_header Host $host;
proxy_set_header X-Forward-For $remote_addr;
proxy_pass http://192.168.1.66;
}
}
(15) Load balancing
upstream news_server_pool{
server 10.0.0.21:80 weight=1 max_fails=2 fail_timeout=30s;
server 10.0.0.22:80 weight=1 max_fails=2 fail_timeout=30s;
}
upstream tv_server_pool{
server 10.0.0.31:80 weight=1 max_fails=2 fail_timeout=30s;
server 10.0.0.32:80 weight=2 max_fails=2 fail_timeout=30s;
}
Explanation: Check twice within 30s and find the next server if it fails.
location /news {
proxy_pass http://news_server_pool;
proxy_set_header Host $host;
proxy_set_header X-Forward-For $remote_addr;
}
location /tv {
proxy_pass http://tv_server_pool;
proxy_set_header Host $host;
proxy_set_header X-Forward-For $remote_addr;
}