日志分割
随着网站的访问量增加,默认情况下
Apache
的单个日志文件也会越来越大- 日志文件占用磁盘空间很大
- 查看相关信息不方便
- 对日志文件进行分割
- Apache自带
rotatelogs
分割工具实现 - 第三方工具
cronolog
分割
- Apache自带
rotatelogs
分割
- 配置网站的日志文件转交给
rotatelogs
分割处理 - 配置格式为
ErrorLog 4| rotatelogs命令的绝对路径-I日志文件路径/网站名-error %Y%m%d.log 86400"-
CustomLog "| rotatelogs命令路径-I日志文件路径/网站名-access %Y%m%d.log 86400" combined
配合实例
[[email protected] ~]# yum install httpd -y //安装http服务
已加载插件:fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: centos.ustc.edu.cn
* extras: centos.ustc.edu.cn
...//省略部分内容...
已安装:
httpd.x86_64 0:2.4.6-90.el7.centos
...//省略部分内容...
完毕!
[[email protected] bin]# cd /usr/sbin/ //进入sbin目录
[[email protected] sbin]# ls rotat* //查看目录下是否生成命令
rotatelogs //安装http服务后生成rotatelogs命令
[[email protected] sbin]# cd /var/log/httpd/ //进入http服务日志存放目录
[[email protected] httpd]# ls //查看是否生成日志文件
[[email protected] httpd]# //安装完成后并没有生成目录文件
[[email protected] httpd]# vim /etc/httpd/conf/httpd.conf //进入http配置文件,进行常规配置
...//省略部分内容...
# prevent Apache from glomming onto all bound IP addresses.
#
Listen 192.168.144.133:80 //修改IPv4为本地地址
#Listen 80 //注释掉IPv6监听
...//省略部分内容...
# If your host doesn't have a registered DNS name, enter its IP address here.
#
ServerName www.kgc.com:80 //开启域名服务
#
# Deny access to the entirety of your server's filesystem. You must
...//省略部分内容...
:wq
[[email protected] httpd]# systemctl stop firewalld.service //关闭防火墙
[[email protected] httpd]# setenforce 0 //关闭增强型安全功能
[[email protected] httpd]# systemctl start httpd.service //启动http服务
[[email protected] httpd]# ls //查看是否生成日志文件
access_log error_log //服务开启后成功生成日志文件
[[email protected] httpd]# vim conf/httpd.conf //进入更改主配置文件,做日志分割
...//省略部分内容...
# container, that host's errors will be logged there and not here.
#
#ErrorLog "logs/error_log" //复制此条目,并将此条目在下方粘贴
ErrorLog "| /usr/sbin/rotatelogs -l logs/www.kgc.com.error_%Y%m%log 86400"
//更改条目,引用rotatelogs命令,创建分割日志条目名称,每86400秒(一天)创建一次文件
# LogLevel: Control the number of messages logged to the error_log.
...//省略部分内容...
# (Combined Logfile Format) you can use the following directive.
#
#CustomLog "logs/access_log" combined //复制此条目,并将此条目在下方粘贴
CustomLog "| /usr/sbin/rotatelogs -l logs/www.kgc.com.access_%Y%m%dlog 86400" combined
//更改条目,引用rotatelogs命令,创建分割日志条目名称,每86400秒(一天)创建一次文件
</IfModule>
<IfModule alias_module>
...//省略部分内容...
:wq
[[email protected] httpd]# systemctl stop httpd.service //停止http服务
[[email protected] httpd]# systemctl start httpd.service //启动http服务
[[email protected] httpd]# cd ..
[[email protected] log]# cd ~
[[email protected] ~]# cd /var/log/httpd/ //重新进入站点文件
[[email protected] httpd]# ls
access_log error_log www.kgc.com.error_20191105log //显示分割除来的日志文件
[[email protected] httpd]# date //查看时间
2019年 11月 05日 星期二 19:27:50 CST
[[email protected] httpd]# date -s 11/06/19 //将时间设置后一天
2019年 11月 06日 星期三 00:00:00 CST
[[email protected] httpd]# systemctl stop httpd.service //关闭http服务
[[email protected] httpd]# systemctl start httpd.service //启动服务
[[email protected] httpd]# cd ~ //回到家目录
[[email protected] ~]# cd /var/log/httpd/ //重新今日日志存放目录
[[email protected] httpd]# ls //查看
access_log error_log www.kgc.com.error_20191105log www.kgc.com.error_20191106log
//生成新的日志文件
第三方工具cronolog
- 源码编译安装
cronolog
工具 cronolog安装包下载 - 配置网站日志文件转交给
cronolog
分割处理 - 配置格式为:
ErrorLog "| cronolog命令的绝对路径日志文件路径/网站名-error_ %Y%m%d.log'
CustomLog "| cronolog命令的绝对路径|日志文件路径/网站名%Y%m%d.log" combined
配置实例
重新开启一台Linux虚拟机,重新安装http服务
[[email protected] ~]# yum install httpd -y //安装服务
已加载插件:fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: centos.ustc.edu.cn
...//省略部分内容...
已安装:
httpd.x86_64 0:2.4.6-90.el7.centos
...//省略部分内容...
[[email protected] ~]# mount.cifs //192.168.100.10/lamp-c7 /mnt //远程挂载目录
Password for [email protected]//192.168.100.10/lamp-c7:
[[email protected] ~]# cd /mnt/ //进入挂载目录
[[email protected] mnt]# ls //查看是否成功挂载
apr-1.6.2.tar.gz cronolog-1.6.2-14.el7.x86_64.rpm httpd-2.4.29.tar.bz2 mysql-5.6.26.tar.gz
apr-util-1.6.0.tar.gz Discuz_X2.5_SC_UTF8.zip LAMP-php5.6.txt nginx-1.12.0.tar.gz
awstats-7.6.tar.gz error.png miao.jpg php-5.6.11.tar.bz2
[[email protected] mnt]# rpm -ivh cronolog-1.6.2-14.el7.x86_64.rpm //安装软件包
警告:cronolog-1.6.2-14.el7.x86_64.rpm: 头V3 RSA/SHA256 Signature, 密钥 ID 352c64e5: NOKEY
准备中... ################################# [100%]
正在升级/安装...
1:cronolog-1.6.2-14.el7 ################################# [100%]
[[email protected] mnt]# cd /usr/sbin/ //进入目录
[[email protected] sbin]# ls cronolog* //查看是否生成cronolog命令
cronolog //成功安装
[[email protected] sbin]# vim /etc/httpd/conf/httpd.conf //进入编辑配置文件
...//省略部分内容...
# prevent Apache from glomming onto all bound IP addresses.
#
Listen 192.168.144.133:80 //设置监听地址
#Listen 80
#
# Dynamic Shared Object (DSO) Support
...//省略部分内容...
# If your host doesn't have a registered DNS name, enter its IP address here.
#
ServerName www.kgc.com:80 //设置域名
#
# Deny access to the entirety of your server's filesystem. You must
...//省略部分内容...
# container, that host's errors will be logged there and not here.
#
ErrorLog "| /usr/sbin/cronolog logs/www.kgc.com.error_%Y%m%dlog"
//调用cronolog命令创建错误日志文件
#
# LogLevel: Control the number of messages logged to the error_log.
...//省略部分内容...
# (Combined Logfile Format) you can use the following directive.
#
CustomLog "| /usr/sbin/cronolog logs/www.kgc.com.access_%Y%m%dlog" combined
//调用cronolog命令创建访问日志文件
</IfModule>
...//省略部分内容...
:wq
[[email protected] sbin]# systemctl stop firewalld.service //关闭防火墙
[[email protected] sbin]# setenforce 0 //关闭增强性安全功能
[[email protected] sbin]# systemctl start httpd.service //开启http服务
[[email protected] sbin]# cd /var/log/httpd/ //进入日志文件目录
[[email protected] httpd]# ls //查看
www.kgc.com.error_20191105log //生成日志分割文件
[[email protected] httpd]# date //查看当前时间
2019年 11月 05日 星期二 20:11:57 CST
[[email protected] httpd]# date -s 11/06/19 //将时间设置为后一天
2019年 11月 06日 星期三 00:00:00 CST
[[email protected] httpd]# systemctl stop httpd.service //停止http服务
[[email protected] httpd]# systemctl start httpd.service //启动http服务
[[email protected] httpd]# cd ~ //回到家目录
[[email protected] ~]# cd /var/log/httpd/ //重新进入日志目录
[[email protected] httpd]# ls //查看
www.kgc.com.error_20191105log www.kgc.com.error_20191106log //生成新的日志文件
AWStats
日志分析系统介绍
AWStats
日志分析系统
Perl
语言开发的一款开源日志分析系统- 可用来分析
Apache、Samba、 Vsftpd、 IIS
等服务 器的访问日志 - 信息结合
crond
等计划任务服务,可对日志内容定期进行分析
部署AWStats
日志分析系统
- 重新开启一台Linux虚拟机,安装http、DNS服务,并进行常规配置
[[email protected] ~]# yum install bind httpd -y
已加载插件:fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirrors.163.com
...//省略部分内容...
已安装:
bind.x86_64 32:9.11.4-9.P2.el7 httpd.x86_64 0:2.4.6-90.el7.centos
...//省略部分内容...
[[email protected] ~]# vim /etc/named.conf
...//省略部分内容...
options {
listen-on port 53 { any; };
listen-on-v6 port 53 { ::1; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
recursing-file "/var/named/data/named.recursing";
secroots-file "/var/named/data/named.secroots";
allow-query { any; };
...//省略部分内容...
[[email protected] ~]# vim /etc/named.rfc1912.zones
...//省略部分内容...
zone "kgc.com" IN {
type master;
file "kgc.com.zone";
allow-update { none; };
};
...//省略部分内容...
[[email protected] named]# cp -p named.localhost kgc.com.zone
[[email protected] named]# vim kgc.com.zone
$TTL 1D
@ IN SOA @ rname.invalid. (
0 ; serial
1D ; refresh
1H ; retry
1W ; expire
3H ) ; minimum
NS @
A 127.0.0.1
www IN A 192.168.144.133
:wq
[[email protected] named]# vim /etc/httpd/conf/httpd.conf
...//省略部分内容...
# prevent Apache from glomming onto all bound IP addresses.
#
Listen 192.168.144.133:80
#Listen 80
#
# Dynamic Shared Object (DSO) Support
...//省略部分内容...
# If your host doesn't have a registered DNS name, enter its IP address here.
#
ServerName www.kgc.com:80
#
# Deny access to the entirety of your server's filesystem. You must
...//省略部分内容...
:wq
[[email protected] named]# systemctl stop firewalld.service
[[email protected] named]# setenforce 0
[[email protected] named]# systemctl start named
[[email protected] named]# systemctl start httpd.service
- 在客户机中测试网站是否可以访问
- 安装
AWStats
软件包
[[email protected] named]# cd /etc/httpd/
[[email protected] httpd]# mount.cifs //192.168.100.10/lamp-c7 /mnt //将软件存放目录挂载在mnt目录下
Password for [email protected]//192.168.100.10/lamp-c7: //直接回车
[[email protected] httpd]# cd /mnt/ //查看挂目录
[[email protected] mnt]# ls
apr-1.6.2.tar.gz cronolog-1.6.2-14.el7.x86_64.rpm httpd-2.4.29.tar.bz2 mysql-5.6.26.tar.gz
apr-util-1.6.0.tar.gz Discuz_X2.5_SC_UTF8.zip LAMP-php5.6.txt nginx-1.12.0.tar.gz
awstats-7.6.tar.gz error.png miao.jpg php-5.6.11.tar.bz2
[[email protected] mnt]# tar zxvf awstats-7.6.tar.gz -C /opt/ //解压awstats工具在opt目录
awstats-7.6/
awstats-7.6/tools/
awstats-7.6/tools/awstats_buildstaticpages.pl
...//省略部分内容...
[[email protected] mnt]# cd /opt/ //进入opt目录
[[email protected] opt]# ls
awstats-7.6 rh
[[email protected] opt]# mv awstats-7.6/ /usr/local/awstats //将解压的文件移动至/usr/local/目录下
[[email protected] opt]# cd /usr/local/ //进入local目录
[[email protected] local]# ls
awstats bin etc games include lib lib64 libexec sbin share src
[[email protected] local]# cd awstats/ //进入awstats目录
[[email protected] awstats]# ls
docs README.md tools wwwroot
[[email protected] awstats]# cd tools/ //进入工具存放目录
[[email protected] tools]# ls //查看可执行工具
awstats_buildstaticpages.pl awstats_updateall.pl httpd_conf nginx xslt
awstats_configure.pl dolibarr logresolvemerge.pl urlaliasbuilder.pl
awstats_exportlib.pl geoip_generator.pl maillogconvert.pl webmin
- 为要统计的站点建立配置文件
[[email protected] tools]# ./awstats_configure.pl //执行配置工具
----- AWStats awstats_configure 1.0 (build 20140126) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
...//省略部分内容...
Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
> /etc/httpd/conf/httpd.conf //输入http配置文件路径
...//省略部分内容...
-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y //输入y确认
-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> www.kgc.com //输入网页域名
-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
> //直接回车,默认存放配置文件路径即可
...//省略部分内容...
Or if you have several config files and prefer having only one command:
/usr/local/awstats/tools/awstats_updateall.pl now
Press ENTER to continue...
A SIMPLE config file has been created: /etc/awstats/awstats.www.kgc.com.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for 'www.kgc.com' with command:
> perl awstats.pl -update -config=www.kgc.com
You can also read your statistics for 'www.kgc.com' with URL:
> http://localhost/awstats/awstats.pl?config=www.kgc.com //给出分析访问网址
Press ENTER to finish...
[[email protected] tools]# cd /etc/httpd/conf //进入http配置文件目录
[[email protected] conf]# vim httpd.conf //编辑配置文件
...//省略部分内容...
# This is to permit URL access to scripts/files in AWStats directory.
#
<Directory "/usr/local/awstats/wwwroot"> //自动生成配置文件
Options None
AllowOverride None
# Order allow,deny //注释掉访问权限设置
# Allow from all
Require all granted //添加条目分为所有人都可以访问
</Directory>
:wq
- 修改站点统计配置文件
[[email protected] etc]# cd /etc/awstats/ //进入配置awstats配置文件目录
[[email protected] awstats]# ls //查看
awstats.www.kgc.com.conf //awstats配置文件
[[email protected] awstats]# ls /var/log/httpd/ //查看http服务日志文件名称
access_log error_log
[[email protected] awstats]# vim awstats.www.kgc.com.conf //进入awstats配置文件
...//省略部分内容...
# Example: "/pathtotools/logresolvemerge.pl *.log |"
#
LogFile="/var/log/httpd/access_log" //找到次条目,更改要分析的日志文件名称即可
# Enter the log file type you want to analyze.
...//省略部分内容...
# Example: "C:/awstats_data_dir"
# Default: "." (means same directory as awstats.pl)
#
DirData="/var/lib/awstats" //找到此条目,查看分析后数据的存放位置
# Relative or absolute web URL of your awstats cgi-bin directory.
...//省略部分内容...
:wq
[[email protected] awstats]# cd /var/lib/ //进入lib目录
[[email protected] lib]# mkdir awstats //创建awstats分析数据存放目录
[[email protected] lib]# ls //查看目录是否成功创建
AccountsService chrony flatpak initramfs misc PackageKit rpm-state systemd yum
alsa color fprint ipa-client mlocate plymouth rsyslog tpm
alternatives colord games iscsi net-snmp polkit-1 samba tuned
authconfig dav gdm libvirt NetworkManager postfix selinux udisks2
awstats dbus geoclue lldpad nfs pulse setroubleshoot upower
bluetooth dhclient gssproxy logrotate ntp rpcbind sss vmware
certmonger dnsmasq hyperv machines os-prober rpm stateless xkb
[[email protected] lib]# systemctl restart httpd.service //重启http服务
在客户机中测试
awstats
工具- 这个时候访问的
awstats
是没有做统计分析的,我们需要执行更新数据工具来执行统计
[[email protected] lib]# cd /usr/local/awstats/ //进入awstats工具目录
[[email protected] awstats]# ls
docs README.md tools wwwroot
[[email protected] awstats]# cd tools/ //进入工具目录
[[email protected] tools]# ls
awstats_buildstaticpages.pl awstats_updateall.pl httpd_conf nginx xslt
awstats_configure.pl dolibarr logresolvemerge.pl urlaliasbuilder.pl
awstats_exportlib.pl geoip_generator.pl maillogconvert.pl webmin
[[email protected] tools]# ./awstats_updateall.pl now //执行工具进行更新,注意不要忘记后面加now
Running '"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=www.kgc.com -configdir="/etc/awstats"' to update config www.kgc.com
Create/Update database for config "/etc/awstats/awstats.www.kgc.com.conf" by AWStats version 7.6 (build 20161204)
From data in log file "/var/log/httpd/access_log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 48
Found 0 dropped records,
Found 0 comments,
Found 0 blank records,
Found 0 corrupted records, //成功更新
Found 0 old records,
Found 48 new qualified records.
- 在客户机重新访问
awstats
- 设置周期性计划任务,自动统计访问信息,并优化访问路径
[[email protected] tools]# crontab -e //设置周期性计划任务
*/5 * * * * /usr/local/awstats/tools/./awstats_updateall.pl now //设置每5分钟执行一次
~
~
:wq
[[email protected] tools]# systemctl start crond //启动周期性计划任务
[[email protected] tools]# cd /var/www/html/ //进入http站点目录
[[email protected] html]# vim aws.html //编辑网页
<html>
<head> <meta http-equiv=refresh content="0;url=http://www.kgc.com/awstats/awstats.pl?config=www.kgc.com"> //编辑网页内容
</head>
<body></body>
</html>
:wq //保存退出
- 在客户机中测试优化的访问路径是否可以使用