I have a bunch of text logfiles in the following format:
ID (17 characters) Timestamp (14 characters YYYYmmddHHMMSS e.g. "20060210100040" -> 2006/02/10 10:00:40) Random data (? characters) end of line
The files are already sorted by timestamp. I need to get 1 log file with all the logs from multiple logs files, sorted by timestamp. Note that the log files are really huge, around 3-4G each (and there are dozens of them) I tried the following command:
sort -s -m -t '|' -k1n,1n +17 -o data_sort.txt *.TXT
Here is how I ended up with this command:
-s : don't bother with tie results -m : merge all logs files -t '|' : there is no | in my logs, so the whole line should be field 1 -k1n,1n: sort on the first field as a numeric value +17 : the timestamp starts at index 17 -o : output file
Actually... it fails miserably. The output file data_sort.txt is just the concatenation of all files, not sorted at all :(
I would greatly appreciate if anyone could provide any help on this problem!