How to extract lines within a range from a large textfile

Question

I have a SQL dump which is several GB in size. I'd like to extract several thousand lines, but really an arbitrary number of contiguous lines from the file to another so that I can run it in isolation.

Answer 1

There are a number of approaches

AWK

I first used the command line awk utility to rip out lines, which I found the numbers for using vim

cat large-file.sql | \
    awk '{if (NR>=14054874) print}' | \
    awk '{if (NR <=2224) print}' > large-file-portion.dump.sql

the second number was found because chaining conditions with && caused awk to do nothing.

To turn on line numbers in vim I ran the following

vi large-file.sql
:set number
/token that you are looking for/
:q!

Head | tail

Most of the other answers I found were using head or tail commands in isolation and did not represent a range of a file. I managed to combine them in the following arrangement.

cat large-file.sql | head -n 14053098 | tail -n 2224 > large-file-portion.dump.sql

Using the time utility I was able to find the best performing solution based on maximum resident set size & time

Performance

awk solution

real       162.66
user         0.23
sys          1.74
806912  maximum resident set size
     0  average shared memory size
     0  average unshared data size
     0  average unshared stack size
   208  page reclaims
     0  page faults
     0  swaps
     0  block input operations
     0  block output operations
     0  messages sent
     0  messages received
     0  signals received
172238  voluntary context switches
  3155  involuntary context switches

head | tail solution

real       136.77
user         0.22
sys          1.68
811008  maximum resident set size
     0  average shared memory size
     0  average unshared data size
     0  average unshared stack size
   209  page reclaims
     0  page faults
     0  swaps
     0  block input operations
     0  block output operations
     0  messages sent
     0  messages received
     0  signals received
159381  voluntary context switches
 22392  involuntary context switches

Hopefully using this information you'll feel informed enough to make your own choices (or propose an alternative answer I'd love to read)