Nginx 与 Lua

Why 80% of the code can not do farming architect? >>>  

These months, we have gradually integrated into the front end of the Lua Nginx configuration Mixlr in. Lua is a profile can be embedded in Nginx dynamic scripting language, which can request any stage of the various processing executed in Lua code Nginx. We just started using Lua route requests to the back-end server, but it is the role of architecture we exceeded our expectations. Here to talk about the work we do. Forced the search engine indexes only mixlr.com Google target domain name as a completely separate website, we do not want the crawler to fetch page subdomain, reduce our Page rank.

location / {
  header_filter_by_lua '
    if ngx.var.query_string and ngx.re.match( ngx.var.query_string, "^([0-9]{10})$" ) then
      ngx.header["Expires"] = ngx.http_time( ngx.time() + 31536000 ); 
      ngx.header["Cache-Control"] = "max-age=31536000";
    end
  ';

If the request is not mixlr.com robots.txt domain name, then the internal rewrite to robots_diallow.txt, although the standard rewrite instructions can also achieve this demand, but Lua implementation easier to understand and maintain. The head is provided in response to the program logic provides a default configuration rules Nginx than a more flexible arrangement Lua. In the following example, we want to ensure that the correct response to the first set, so that if the browser sends a request header is specified, you can indefinitely cache static files, the user is simply download once. The rewrite rule that any static document, if the request contains a timestamp parameter values, then set the appropriate response Expires and Cache-Control header.

location / {
  header_filter_by_lua '
    if ngx.var.query_string and ngx.re.match( ngx.var.query_string, "^([0-9]{10})$" ) then
      ngx.header["Expires"] = ngx.http_time( ngx.time() + 31536000 ); 
      ngx.header["Cache-Control"] = "max-age=31536000";
    end
  ';

  try_files $uri @dynamic;}

Delete jQuery JSONP request time stamp parameter when many external client requests JSONP interface will include a time stamp similar parameters, resulting in Nginx proxy cache can not hit (because they can not ignore the specified HTTP parameters). The following rules removed the time stamp parameters that can be cached upstream Nginx server response content, and reduce the load on back-end servers.

location / {
  rewrite_by_lua '
    if ngx.var.args ~= nil then
      -- /some_request?_=1346491660 becomes /some_request
      local fixed_args, count = ngx.re.sub( ngx.var.args, "&?_=[0-9]+", "" );
      if count > 0 then
        return ngx.exec(ngx.var.uri, fixed_args);
      end
    end
  ';}

The rear end of the slow log request to the error log of Nginx if the rear end of the request response is very slow, it can be recorded in the error log of Nginx, for subsequent tracing.

location / {
  log_by_lua '
    if tonumber(ngx.var.upstream_response_time) >= 1 then
      ngx.log(ngx.WARN, "[SLOW] Ngx upstream response time: " .. ngx.var.upstream_response_time .. "s from " .. ngx.var.upstream_addr);
    end
  ';}

Redis-based real-time IP banned some cases, the need to prevent rogue crawling reptiles, which can be done by specialized equipment banned, but by Lua, can achieve a simple version of the ban.

lua_shared_dict banned_ips 1m; 

location / {
  access_by_lua '
    local banned_ips = ngx.shared.banned_ips;
    local updated_at = banned_ips:get("updated_at");

    -- only update banned_ips from Redis once every ten seconds:
    if updated_at == nil or updated_at < ( ngx.now() - 10 ) then
      local redis = require "resty.redis";
      local red = redis:new();
      red:set_timeout(200);

      local ok, err = red:connect("your-redis-hostname", 6379);
      if not ok then
        ngx.log(ngx.WARN, "Redis connection error retrieving banned_ips: " .. err);
      else
        local updated_banned_ips, err = red:smembers("banned_ips");
        if err then
          ngx.log(ngx.WARN, "Redis read error retrieving banned_ips: " .. err);
        else
          -- replace the locally stored banned_ips with the updated values:
          banned_ips:flush_all();
          for index, banned_ip in ipairs(updated_banned_ips) do
            banned_ips:set(banned_ip, true);
          end
          banned_ips:set("updated_at", ngx.now());
        end
      end
    end

    if banned_ips:get(ngx.var.remote_addr) then
      ngx.log(ngx.WARN, "Banned IP detected and refused access: " .. ngx.var.remote_addr);
      return ngx.exit(ngx.HTTP_FORBIDDEN);
    end
  ';}

Now you can block specific IP access:

ruby> $redis.sadd("banned_ips", "200.1.35.4")

Nginx process once every 10 seconds to obtain the latest ban IP list from Redis. Note that, if the architecture used in Haproxy such a similar load balancing server, you need the $ remote_addr set to the correct remote IP address. This method can also be used to check the HTTP User-Agent field, required to satisfy the specified conditions. Use Nginx output CSRF (form_authenticity_token) a problem Mixlr extensive use of the page cache, thus introduced is how to give each page output session-level CSRF token. We sub Nginx request token acquired from the upstream web server, and then use the Nginx SSI (server-side include) function to output the page. This will not only solve the problem of CSRF attacks, but also to ensure that the cache can be normal use.

location /csrf_token_endpoint {
  internal;
   
  include /opt/nginx/conf/proxy.conf;
  proxy_pass "http://upstream";}

location @dynamic {
  ssi on;
  set $csrf_token '';

  rewrite_by_lua '
    -- Using a subrequest, we our upstream servers for the CSRF token for this session:
    local csrf_capture = ngx.location.capture("/csrf_token_endpoint");
    if csrf_capture.status == 200 then
      ngx.var.csrf_token = csrf_capture.body;

      -- if this is a new session, ensure it sticks by passing through the new session_id
      -- to both the subsequent upstream request, and the response:
      if not ngx.var.cookie_session then
        local match = ngx.re.match(csrf_capture.header["Set-Cookie"], "session=([a-zA-Z0-9_+=/+]+);");
        if match then
          ngx.req.set_header("Cookie", "session=" .. match[1]);
          ngx.header["Set-Cookie"] = csrf_capture.header["Set-Cookie"]; 
        end
      end
    else
      ngx.log(ngx.WARN, "No CSRF token returned from upstream, ignoring.");
    end
  ';

  try_files /maintenance.html /rails_cache$uri @thin;}

CSRF token生成 app/metal/csrf_token_endpoint.rb:

class CsrfTokenEndpoint
  def self.call(env)
    if env["PATH_INFO"] =~ /^\/csrf_token_endpoint/
      session = env["rack.session"] || {}

      token = session[:_csrf_token]
      if token.nil?
        token = SecureRandom.base64(32)
        session[:_csrf_token] = token
      end

      [ 200, { "Content-Type" => "text/plain" }, [ token ] ]
    else      
      [404, {"Content-Type" => "text/html"}, ["Not Found"]]
    end
  endend

我们的模版文件示例: <meta name="csrf-param" value="authenticity_token"/> <meta name="csrf-token" value="<!--# echo var="csrf_token" default="" encoding="none" -->"/> Again you could make use of lua_shared_dict to store in memory the CSRF token for a particular session. This minimises the number of trips made to /csrf_token_endpoint. 原文链接：http://devblog.mixlr.com/2012/09/01/nginx-lua/

Reproduced in: https: //my.oschina.net/766/blog/211004