Author: JackTian
micro-channel public number: IT's Jiege trip (ID: Jake_Internet)
In the previous article, we introduced the basic concepts of Squid proxy server, proxy working mechanism, basic proxy types, installing Squid services, building traditional proxy, transparent proxy and ACL access control. For the article link, please refer to: You know the most commonly used open source Squid proxy server, so today, I will introduce you a commonly used Squid log analysis software, I hope it can be used by students who are helpful in their daily work in the future.
Sarg: Full name: Squid Analysis Report Generator, a Squid log analysis tool that uses HTML format to list the Internet website information, time occupancy information, ranking, number of connections, visits and other related information that users visit;
Squid log analysis software: http://www.squid-cache.org/Misc/log-analysis.html
1. Install the GD library during Sarg deployment
# yum -y install gd gd-devel
Install sarg
# mkdir /usr/local/sarg
# cd /usr/local/sarg/
# tar zxf sarg-2.3.7.tar.gz
# cd sarg-2.3.7
# ./configure --prefix=/usr/local/sarg/ -sysconfdir=/etc/sarg --enable-extraprotection && make && make installConfiguration item meaning:
-
-sysconfdir=/etc/sarg: configuration file directory
- --enable-extraprotection: add extra security protection
Configuration
# vi /etc/sarg/sarg.conf
7 access_log /usr/local/squid/var/logs/access.log // 指定 squid 的访问日志文件
25 title "Squid User Access Reports" // 网页 title 标题
120 output_dir /var/www/html/sarg // sarg 报告的输出目录
178 user_ip no // 使用用户名显示
184 topuser_sort_field connect reverse // 在 top 排序中，指定连接次数，访问字节数，采用降序排列
190 user_sort_field connect reverse // 对于用户访问记录，连接次数按降序排列
206 exclude_hosts /usr/local/sarg/noreport // 指定不计入排序的站点列表文件
257 overwrite_report no // 当那个日期报告已经存在，是否覆盖报告
289 mail_utility mailq.postfix // 发送邮件报告的命令
434 charset UTF-8 // 使用字符集
518 weekdays 0-6 // 指定 top 排序时的星期周期，0 为周日
525 hours 9-12,14-16,18-20 // 指定 top 排序时的时间周期
633 www_document_root /var/www/html // 网页根目录
Run
in the above configuration, add sites that are not included in the ranking, and need to be stored in the /usr/local/sarg/noreport file, and the added domain name will not be displayed in the ranking. Execute sarg directly to start a recording. It is recommended to set a symbolic link, and then execute sarg, and output information will be displayed.
# touch /usr/local/sarg/noreport
# ln -s /usr/local/sarg/bin/sarg /usr/local/bin/
# sarg
SARG: 纪录在文件: 0, reading: 100.00%
SARG: 没有找到纪录
SARG: 结束
2. Sarg related command help
-
-a: Specify the host name or address that is not included in the sorting
-
-b: User agent log file output
-
-c: Specify the file name of the list of sites not included in the sorting as /usr/local/sarg/norecords. For these sites accessed by the client, they will not be sorted by top;
-
-d: Specify the date range
-
-e: Specify the report recipient email
-
-f: Specify the configuration file
-
-g: Specify the input date format
-
-h: help information
-
-i: Specify the user name or IP address for client sorting
-
-l: Specify the absolute path of the squid log file.
-
-o: Specify the output path of the web report file. It is recommended to use webmaster or other non-admin users to run sarg.
-
-p: Use IP address as userid domain
- -w: Specify the temporary file directory, and confirm that the partition where the directory is located is large enough, above 1G.
3. Planned task
sarg can be made into planned task and executed regularly.
# vim /usr/local/sarg/daily.sh // 每日日报
#!/bin/bash
#Get current date
TODAY=$(date +%d/%m/%Y)
#Get one week ago today
YESTERDAY=$(date --date "1 day ago" +%d/%m/%Y)
/usr/local/bin/sqmgrlog -l /usr/local/squid/logs/access.log -o /var/www/html/sarg -z -d $YESTERDAY-$TODAY &> /dev/null
exit 0
# chmod +x /usr/local/sarg/daily.sh
# crontab -e // 添加定时任务，每天 0 点执行
00 00 * * * /usr/local/sarg/daily.sh
# chkconfig crond on
In addition to daily reports, you can also write weekly, monthly, and email reports. You can refer to the following script:
Weekly report:
-------------------------------------
#!/bin/bash
#Get current date
TODAY=$(date +%d/%m/%Y)
#Get one week ago today
YESTERDAY=$(date --date "1 week ago" +%d/%m/%Y)
/usr/local/bin/sqmgrlog -l /usr/local/squid/logs/access.log -o /usr/local/apache/htdocs/reports/weekly -z -d $YESTERDAY-$TODAY
exit 0
月报告：
-------------------------------------
#!/bin/bash
#Get current date
TODAY=$(date +%d/%m/%Y)
#Get one week ago today
YESTERDAY=$(date --date "1 month ago" +%d/%m/%Y)
/usr/local/bin/sqmgrlog -l /usr/local/squid/logs/access.log -o /usr/local/apache/htdocs/reports/monthly -z -d $YESTERDAY-$TODAY
/usr/local/squid/bin/squid -k rotate
exit 0
通过邮件发送报告：
-------------------------------------
#!/bin/bash
#Get current date
TODAY=$(date +%d/%m/%Y)
#Get one week ago today
YESTERDAY=$(date --date "1 day ago" +%d/%m/%Y)
/usr/local/bin/sqmgrlog -l /usr/local/squid/logs/access.log -e [email protected] -z -d $YESTERDAY-$TODAY
exit 0
Four, common problems
rpm package installation, can not be installed, an error is reported
warning: *.rpm: Header V3 RSA/SHA256 Signature, keykey ID c105b9de:
Solution:
Add --force --nodeps to the end of the rpm statement, and change rpm -ivh .rpm to rpm -ivh .rpm --force --nodeps.
Nodeps means to ignore dependencies. In the Linux environment, there will be more or less dependent dependencies between software. With these two setting options, you can ignore these dependencies and force installation or uninstallation.
E.g:
rpm -ivh gd-devel-2.0.35-11.el6.x86_64.rpm --force --nodeps
Or try to uninstall:
Through man rpm, I found that --allmatches can solve this problem.
E.g:
# rpm -e --allmatches --nodeps gd*
When executing the sarg command, an error is reported
# sarg
SARG: Unknown sort order "BYTES" for parameter "topuser_sort_field"
Solution:
Edit the sarg.conf configuration file, and remove the BYTES in topuser_sort_field connect BYTES reverse from line 184 in the file;
# vi /etc/sarg/sarg.conf
184 topuser_sort_field connect BYTES reverse
Then execute the command sarg;
# sarg
SARG: 纪录在文件: 0, reading: 100.00%
SARG: 没有找到纪录
SARG: 结束