使用正则表达式在日志中的时间戳之间添加换行符

在每个新时间之前设置换行符：

awk '!a[$1,$2]++ && NR>1{print ""} 1'

工作原理：在awk中，$1并且$2是第一场，在这种情况下，日期和时间。a[$1,$2]是一个关联数组，用于计算我们看到这两个字段的次数。如果之前已经看到了该日期和时间，!a[$1,$2]并且不在第一行上NR>1，那么我们将打印空白行以进行分隔print ""。最终1只是打印当前行的简写。

例子

将您的样本登录到文件中logfile：

$ awk '!a[$1,$2]++ && NR>1{print ""} 1' logfile
2017-12-03 01:35:58 [Notice] syslog: local  IP address 
2017-12-03 01:35:58 [Notice] syslog: remote IP address 
2017-12-03 01:35:58 [Notice] syslog: primary   DNS address 
2017-12-03 01:35:58 [Notice] syslog: secondary DNS address 

2017-12-03 01:35:59 [Warning] kernel: Link State: PVC_8_0 logistic interface up.
2017-12-03 01:35:59 [Informational] dnsmasq[10463]: started, version 2.52 cachesize 150
2017-12-03 01:35:59 [Informational] dnsmasq[10463]: compile time options: no-IPv6 GNU-getopt no-RTC no-DBus no-I18N no-DHCP no-TFTP
2017-12-03 01:35:59 [Warning] dnsmasq[10463]: ignoring nameserver 127.0.0.1 - local interface
2017-12-03 01:35:59 [Informational] dnsmasq[10463]: using nameserver 87.216.1.66#53(via ppp80)
2017-12-03 01:35:59 [Informational] dnsmasq[10463]: using nameserver 87.216.1.65#53(via ppp80)
2017-12-03 01:35:59 [Informational] dnsmasq[10463]: read /etc/hosts - 6 addresses

2017-12-03 01:36:00 [Informational] dnsmasq[10532]: started, version 2.52 cachesize 150
2017-12-03 01:36:00 [Informational] dnsmasq[10532]: compile time options: no-IPv6 GNU-getopt no-RTC no-DBus no-I18N no-DHCP no-TFTP
2017-12-03 01:36:00 [Informational] dnsmasq[10532]: using nameserver 87.216.1.66#53(via ppp80)

2017-12-03 01:36:01 [Warning] kernel: ^M
2017-12-03 01:36:01 [Warning] kernel: Send DNS Query : domain=ntp2.jazztel.com qType=A dnsServer=87.216.1.65
2017-12-03 01:36:01 [Warning] kernel: domain: ntp2.jazztel.com , IP: 87.216.1.241
2017-12-03 01:36:01 [Warning] kernel: sntp server=ntp2.jazztel.com: 0x5 ntpServerIP=87.216.1.241

删除重复的时间戳

$ awk '{if(a[$1,$2]++){gsub(/./," ",$1); gsub(/./," ",$2)} else if (NR>1) print""} 1' logfile
2017-12-03 01:35:58 [Notice] syslog: local  IP address 
                    [Notice] syslog: remote IP address
                    [Notice] syslog: primary DNS address
                    [Notice] syslog: secondary DNS address

2017-12-03 01:35:59 [Warning] kernel: Link State: PVC_8_0 logistic interface up.
                    [Informational] dnsmasq[10463]: started, version 2.52 cachesize 150
                    [Informational] dnsmasq[10463]: compile time options: no-IPv6 GNU-getopt no-RTC no-DBus no-I18N no-DHCP no-TFTP
                    [Warning] dnsmasq[10463]: ignoring nameserver 127.0.0.1 - local interface
                    [Informational] dnsmasq[10463]: using nameserver 87.216.1.66#53(via ppp80)
                    [Informational] dnsmasq[10463]: using nameserver 87.216.1.65#53(via ppp80)
                    [Informational] dnsmasq[10463]: read /etc/hosts - 6 addresses

2017-12-03 01:36:00 [Informational] dnsmasq[10532]: started, version 2.52 cachesize 150
                    [Informational] dnsmasq[10532]: compile time options: no-IPv6 GNU-getopt no-RTC no-DBus no-I18N no-DHCP no-TFTP
                    [Informational] dnsmasq[10532]: using nameserver 87.216.1.66#53(via ppp80)

2017-12-03 01:36:01 [Warning] kernel: ^M
                    [Warning] kernel: Send DNS Query : domain=ntp2.jazztel.com qType=A dnsServer=87.216.1.65
                    [Warning] kernel: domain: ntp2.jazztel.com , IP: 87.216.1.241
                    [Warning] kernel: sntp server=ntp2.jazztel.com: 0x5 ntpServerIP=87.216.1.241

在这种情况下，如果我们之前看到过日期$1和时间$2，那么我们将其内容替换为空格gsub(/./," ",$1); gsub(/./," ",$2)。如果不是，并且我们不在第一行，那么我们将打印空白行以进行分隔。