Collecting log - log collection distal Buried

Principle Analysis

Analysis:
for collecting data on the page Buried (prepared js used to dynamically generate short tag Img then added dom page, by using the parameter request to the tag server)
can resolve the problem by cross-domain img src attribute tag the data is passed to the back-end server
performs the step of back-end servers:

  1. Accepts the request, the response picture (log.gif)
  2. Resolution parameters, save the data
  3. Set cookie

Design and Implementation

Determine the need to gather the information

name way Remark
interview time web server Nginx $msec
IP web server Nginx $remote_add
domain name JavaScript document.domain
URL JavaScript document.URL
page title JavaScript document.title
View Client web server Nginx $http_user_agent
Parameter 1 JavaScript K1
Parameter 2 JavaScript k2

Buried distal operation

//通过组装params的参数为url请求到指定IP的log.gif地址
function logOperate(params){
    var args = ''; 
    for(var i in params) {
        if(args != '') {
            args += '&';
        }   
        args += i + '=' + encodeURIComponent(params[i]);
    }
     var img = new Image(1, 1); 
    img.src = 'http://127.0.0.1/log.gif?' + args;
}

Nginx simple configuration back-end log collection

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    #日志采用|分隔符
    log_format  main  '$remote_addr|$msec|$http_user_agent|$k1|$k2';
    access_log off;
    sendfile        on;
    #连接持有时间
    keepalive_timeout  5;
    #gzip  on;
    server {
        listen       80;
        server_name  127.0.0.1;
                #拦截/log.gif路径,并且只针对这个路径才采集日志
        location /log.gif {
                        #日志记录位置且采用main格式
            access_log /var/log/nginx/access.log main;
                        #返回类型
            default_type image/gif;
                        #获取请求参数值格式为[$arg_argname],以便于日志格式解析。
            set $k1 $arg_k1;
            set $k2 $arg_k2;
                        #设置返回前端时不需要缓存
            add_header Expires "Fri, 01 Jan 1980 00:00:00 GMT";
            add_header Pragma "no-cache";
            add_header Cache-Control "no-cache, max-age=0, must-revalidate";
            #返回一个1×1的空gif图片
            empty_gif;
        }
        #拦截其他所有路径,统一返回空图片。
        location / {
            default_type image/gif;
            empty_gif;
        }
    }
}

postscript

Img url requests by the front end corresponding to the rear end of the url parameter is transmitted and parsed to intercept nginx | ​​vertical string delimiter is appended to a log file in /var/log/nginx/access.log.

Or by post flume logstat transmits the corresponding log analysis server corresponding to the data for analysis.

Guess you like

Origin www.cnblogs.com/cjunn/p/12236877.html
log
log