Merge multiple log files into a single one based on timestamp

marcobazzani

I'm trying to find a solution to fastly merge 2 log files coming from 2 application servers. The log files is like this:

00:00:00,028 DEBUG [com.acme.productionservice...

I would like something that based on the time stamp print one line of the log file or another for example:

if file one have 2 lines:

00:00:00,028 DEBUG [com.acme.productionservice...
00:00:00,128 DEBUG [com.acme.productionservice...

and file two have this 3 lines:

00:00:00,045 DEBUG [com.acme.productionservice...
00:00:00,100 DEBUG [com.acme.productionservice...
00:00:00,150 DEBUG [com.acme.productionservice...

the output should be

00:00:00,028 DEBUG [com.acme.productionservice...   (file 1)
00:00:00,045 DEBUG [com.acme.productionservice...   (file 2)
00:00:00,100 DEBUG [com.acme.productionservice...   (file 2)
00:00:00,128 DEBUG [com.acme.productionservice...   (file 1)
00:00:00,150 DEBUG [com.acme.productionservice...   (file 2)

the only way I currently know is using cat file1 file | sort but this is very slow for gb of logs I need something like reading the 2 files and compare the timestamps and decide what to print.

marcobazzani

I ended up by using

sort -m 

I also used a trick to understand from which log file the log comes from with

for a in *.log ; do 
    awk  '$0=FILENAME" "$0' $a > $a.log
do
sort -m -k 2 *.log.log 

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

How to merge multiple files based on a timestamp

From Dev

Merge all log files into one

From Dev

Paste multiple files based on the first column into one single file

From Dev

How to merge multiple csv files into an single csv file with single header also remove duplicates based on the certain column

From Dev

merge multiple dataframes based on matching timestamp

From Dev

Merge multiple txt files into one

From Dev

Merge multiple zip files in to one

From Dev

Merge multiple rows based on a single column value

From Dev

Copy multiple files into one (append, merge) in single invocation without shell redirection?

From Dev

How to merge multiple PDF files into a single file?

From Dev

Merge multiple JSON into single one (Python)

From Dev

merge two files based on one column Awk

From Dev

gulpjs - merge multiple files to make one gulpfile

From Java

Merge / convert multiple PDF files into one PDF

From Dev

Merge multiple .xls files into one sheet

From Dev

Merge multiple PDF files into one in PHP

From Dev

How to merge multiple plist files into one?

From Dev

Merge multiple csv files in one excel sheet

From Dev

Merge multiple PDF files into one in PHP

From Dev

Ssh into multiple servers and merge files into one

From Dev

Multiple files with no header to merge with one header

From Dev

Multiple ViewControllers, xib files, in one Single Storyboard

From Dev

Multiple ViewControllers, xib files, in one Single Storyboard

From Dev

Observe multiple log files in one output

From Dev

Using Python to Merge Single Line .dat Files into one .csv file

From Dev

Is it possible to merge ndf files and mdf file into one single mdf file?

From Dev

Grab one specific column from multiple csv files and merge into one

From Dev

How can I merge multiple topojson files into single topojson file

From Dev

How to read multiple files and merge them into a single pandas data frame?

Related Related

  1. 1

    How to merge multiple files based on a timestamp

  2. 2

    Merge all log files into one

  3. 3

    Paste multiple files based on the first column into one single file

  4. 4

    How to merge multiple csv files into an single csv file with single header also remove duplicates based on the certain column

  5. 5

    merge multiple dataframes based on matching timestamp

  6. 6

    Merge multiple txt files into one

  7. 7

    Merge multiple zip files in to one

  8. 8

    Merge multiple rows based on a single column value

  9. 9

    Copy multiple files into one (append, merge) in single invocation without shell redirection?

  10. 10

    How to merge multiple PDF files into a single file?

  11. 11

    Merge multiple JSON into single one (Python)

  12. 12

    merge two files based on one column Awk

  13. 13

    gulpjs - merge multiple files to make one gulpfile

  14. 14

    Merge / convert multiple PDF files into one PDF

  15. 15

    Merge multiple .xls files into one sheet

  16. 16

    Merge multiple PDF files into one in PHP

  17. 17

    How to merge multiple plist files into one?

  18. 18

    Merge multiple csv files in one excel sheet

  19. 19

    Merge multiple PDF files into one in PHP

  20. 20

    Ssh into multiple servers and merge files into one

  21. 21

    Multiple files with no header to merge with one header

  22. 22

    Multiple ViewControllers, xib files, in one Single Storyboard

  23. 23

    Multiple ViewControllers, xib files, in one Single Storyboard

  24. 24

    Observe multiple log files in one output

  25. 25

    Using Python to Merge Single Line .dat Files into one .csv file

  26. 26

    Is it possible to merge ndf files and mdf file into one single mdf file?

  27. 27

    Grab one specific column from multiple csv files and merge into one

  28. 28

    How can I merge multiple topojson files into single topojson file

  29. 29

    How to read multiple files and merge them into a single pandas data frame?

HotTag

Archive