Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I need to extract log data from many terrabytes worth of log files. The thing is, the data I need starts and ends with patterns I can identify, but the code between can be anything between 10 and 100+ lines.

Example:

Start
# lots of lines here
End

Currently, what I do is grep -A 50 "Start", which gives me the Start and the 50 lines thereafter. However, in almost all cases that is more or less than I need. More meaning the resulting report file grows Gigabytes larger than it needs to be and less meaning I don't get the information I need.

Is there a way to extract exactly what I need, using standard Unix / Linux tools?

share|improve this question
    
What is a regular expression? –  Ярослав Рахматуллин Apr 8 '13 at 14:57

1 Answer 1

up vote 4 down vote accepted

Try it with awk:

awk '/^Start/,/^End/' file

or if you prefer sed:

sed -n '/Start/,/End/p' file
share|improve this answer
    
Cool, thanks, I'll try and let you know –  Sean Patrick Floyd Apr 8 '13 at 14:31
    
@SeanPatrickFloyd you're welcome. I added as well a solution with sed. –  Simon Apr 8 '13 at 14:36
    
Nice code there. Do you mind explaining how sed works in this case? I tried to figure it out by myself checking the man page on ss64 and doing some trial and error experiments, but I still don't get it. :) Thanks. –  Radoo Apr 8 '13 at 14:47
    
@Radoo sed -n -> Suppresses the default output. ' -> beginning of the filter command /Start/ -> regular expression , -> separator /End/ -> regular expression p -> Print. Copy the pattern space to the standard output. ' -> end of the filter file -> file name –  Simon Apr 8 '13 at 14:59
1  
@Radoo: check the "Addresses" section of the man page -- this sed command uses two addresses (both of which are regular expressions) to select ranges of lines to apply the "p" command to. –  Gordon Davisson Apr 8 '13 at 15:39

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.