Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to extract log data from many terrabytes worth of log files. The thing is, the data I need starts and ends with patterns I can identify, but the code between can be anything between 10 and 100+ lines.

Example:

Start
# lots of lines here
End

Currently, what I do is grep -A 50 "Start", which gives me the Start and the 50 lines thereafter. However, in almost all cases that is more or less than I need. More meaning the resulting report file grows Gigabytes larger than it needs to be and less meaning I don't get the information I need.

Is there a way to extract exactly what I need, using standard Unix / Linux tools?

share|improve this question
    
What is a regular expression? – Ярослав Рахматуллин Apr 8 '13 at 14:57
up vote 3 down vote accepted

Try it with awk:

awk '/^Start/,/^End/' file

or if you prefer sed:

sed -n '/Start/,/End/p' file
share|improve this answer
    
Cool, thanks, I'll try and let you know – Sean Patrick Floyd Apr 8 '13 at 14:31
    
@SeanPatrickFloyd you're welcome. I added as well a solution with sed. – Simon Apr 8 '13 at 14:36
    
Nice code there. Do you mind explaining how sed works in this case? I tried to figure it out by myself checking the man page on ss64 and doing some trial and error experiments, but I still don't get it. :) Thanks. – user127350 Apr 8 '13 at 14:47
    
@Radoo sed -n -> Suppresses the default output. ' -> beginning of the filter command /Start/ -> regular expression , -> separator /End/ -> regular expression p -> Print. Copy the pattern space to the standard output. ' -> end of the filter file -> file name – Simon Apr 8 '13 at 14:59
1  
@Radoo: check the "Addresses" section of the man page -- this sed command uses two addresses (both of which are regular expressions) to select ranges of lines to apply the "p" command to. – Gordon Davisson Apr 8 '13 at 15:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .