使用.conf文件logstash在日志文件中获取时间戳

debugcn 发表于 Dev

han

我有一个.conf文件，我想从日志文件中获取时间戳而不是系统日期。对于.log文件中从logstash到Elasticsearch的供稿，我应该做哪些更改？日志可以很好地满足ELK堆栈的问题，它是从日志文件获取系统时间而不是时间。

我的日志行如下

11812211602170772|2019-12-19 00:00:00 004|SPP_005206|[email protected]|APP_016179|prov|live|IdeaMart||caas|http|94771133726||||subs-rec-charg-notify|sms|unknown|subscriber|16.02|LKR|[{"currencyCode":"LKR", "buyingRate": 1, "sellingRate": 1}]|subscription|94771133726||freeRegistration|P9545|success|success|457d|77850|dialog||percentageFromMonthlyRevenue|70||||S1000|SUCCESS|118122116020074|prorate|0|||||||||||monthly||||Mobile Account||dialog||REG_PENDING|REGISTERED

我的.conf文件如下

input{  file{
        path => "/home/rehan/Projects/SIEM/Splunk/test3/sdp-server/*"
        type => "translog"
        start_position => "beginning"   } }

filter {     grok {
            match => { "message" => "%{NUMBER:field1}\|%{TIMESTAMP_ISO8601:field2} %{NUMBER:field3}\|%{WORD:field4}\|*\|%{WORD:field5}\@%{WORD:field6}\.com\|%{WORD:field7}\|%{WORD:field8}\|%{WORD:field9}\|%{WORD:field10}\|\|%{WORD:field11}\|%{WORD:field12}\|%{NUMBER:field13}\|\|\|\|%{WORD:field14}\-%{WORD:field15}\-%{WORD:field16}\-%{WORD:field17}\|%{WORD:field18}\|%{WORD:field19}\|%{WORD:field20}\|%{NUMBER:field21}\|%{WORD:field22}\|\[\{\"%{WORD:field23}\"\:\"%{WORD:field24}\"\,\ \"%{WORD:field25}\"\:\ %{NUMBER:field26}\, \"%{WORD:field27}\"\: %{NUMBER:field28}\}]\|%{WORD:field29}\|%{NUMBER:field30}\|\|%{WORD:field31}\|%{WORD:field32}\|%{WORD:field33}\|%{WORD:field34}\|%{WORD:field35}\|%{NUMBER:field36}\|%{WORD:field37}\|\|%{WORD:field38}\|%{NUMBER:field39}\|\|\|\|%{WORD:field40}\|%{WORD:field41}\|%{NUMBER:field42}\|%{WORD:field43}\|%{NUMBER:field43}\|\|\|\|\|\|\|\|\|\|\|%{WORD:field44}" }     }

    date{       match => [ "field2" , "yyyy-MM-dd HH:mm:ss SSS" ]       } }

output{     elasticsearch{
        hosts => "127.0.0.1:9200"
        index => "translog"     } }

任何人都给出示例建议，说明如何更改.conf文件？

sanduniYW

试试下面的答案：

input{
        file{
            path => "/home/rehan/Projects/SIEM/Splunk/test3/sdp-server/*"
            type => "translog"
            start_position => "beginning"
        }
    }

    filter {
         grok {
                match => { "message" => "%{NUMBER:field1}\|%{TIMESTAMP_ISO8601:field2} %{NUMBER:field3}\|%{WORD:field4}\|*\|%{WORD:field5}\@%{WORD:field6}\.com\|%{WORD:field7}\|%{WORD:field8}\|%{WORD:field9}\|%{WORD:field10}\|\|%{WORD:field11}\|%{WORD:field12}\|%{NUMBER:field13}\|\|\|\|%{WORD:field14}\-%{WORD:field15}\-%{WORD:field16}\-%{WORD:field17}\|%{WORD:field18}\|%{WORD:field19}\|%{WORD:field20}\|%{NUMBER:field21}\|%{WORD:field22}\|\[\{\"%{WORD:field23}\"\:\"%{WORD:field24}\"\,\ \"%{WORD:field25}\"\:\
     %{NUMBER:field26}\, \"%{WORD:field27}\"\: %{NUMBER:field28}\}]\|%{WORD:field29}\|%{NUMBER:field30}\|\|%{WORD:field31}\|%{WORD:field32}\|%{WORD:field33}\|%{WORD:field34}\|%{WORD:field35}\|%{NUMBER:field36}\|%{WORD:field37}\|\|%{WORD:field38}\|%{NUMBER:field39}\|\|\|\|%{WORD:field40}\|%{WORD:field41}\|%{NUMBER:field42}\|%{WORD:field43}\|%{NUMBER:field43}\|\|\|\|\|\|\|\|\|\|\|%{WORD:field44}" }
        }
        date {
            match => [ "field2", "yyyy-MM-dd HH:mm:ss" ]
            target => "@timestamp"
        }
    }

    output{ 
        elasticsearch{
            hosts => "127.0.0.1:9200"
            index => "translog"
        }
    }

本文收集自互联网，转载请注明来源。

如有侵权，请联系[email protected] 删除。