Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I've got a rather sizable CSV file (75MB). I'm just trying to produce a graph of it, so I really don't need all of the data.

Rewording: I'd like to delete n lines, then keep one line, then delete n lines, and so on.

So if the file looked like this:

Line 1
Line 2
Line 3
Line 4
Line 5
Line 6

and n=2, then the output would be:

Line 3
Line 6

It seems like sed might be able to do this, but I haven't been able to figure out how. A bash command would be ideal, but I'm open to any solution.

share|improve this question
2  
Do you really want lines 1, 3, 6, etc., rather than 1, 4, 7, etc.? –  Ilmari Karonen Mar 3 '12 at 18:59
2  
Since it is a CSV file, I assume the first line contains meta data (i.e. field names.). If so, the question should be "every nth line after the first". –  iglvzx Mar 3 '12 at 19:57
4  
1, 3, 6 still doesn't make sense! –  wim Mar 5 '12 at 0:18
1  
I guess it should be 1, 3, 5 unless n=2 is a magic value for triangular numbers (1, 3, 6, 10, 15, 21 etc.) –  rjmunro Mar 7 '12 at 10:32
1  
Can you update your question to make what you're asking for ("every nth line", "n=2") and your desired output (Line 3, Line 6) consistent? Future readers are going to be confused. –  Keith Thompson Mar 8 '12 at 4:56

5 Answers 5

up vote 43 down vote accepted
~ $ awk 'NR == 1 || NR % 3 == 0' yourfile
Line 1
Line 3
Line 6

NR (number of records) variable is records number of lines because default behavior is new line for RS (record seperator). pattern and action is optional in awk's default format 'pattern {actions}'. when we give only pattern part then awk writes all the fields $0 for our pattern's true conditions.

share|improve this answer
7  
Thanks to defaults, you don't even need that much: awk 'NR == 1 || NR % 3 == 0' –  Kevin Mar 3 '12 at 20:39
    
you are right @Kevin thanks. –  Selman Ulug Mar 3 '12 at 20:42
    
@selman: If you like Kevin's solution, you might want to consider updating your answer. –  Keith Thompson Mar 3 '12 at 21:45
3  
Care to explain why it does so? That way if someone wants to slightly tweak it, then hopefully your explanation will help them do so –  Ivo Flipse Mar 4 '12 at 9:41

sed can also do this:

$ sed -n '1p;0~3p' input.txt
Line 1
Line 3
Line 6
share|improve this answer
1  
Could you explain this command? –  qed Jun 9 at 18:56

Perl can do this too:

while (<>) {
    print  if $. % 3 == 1;
}

This program will print the first line of its input, and every third line afterwards.

To explain it a bit, <> is the line input operator, which iterates over the input lines when used in a while loop like this. The special variable $. contains the number of lines read so far, and % is the modulus operator.

This code can be written even more compactly as a one-liner, using the -n and -e switches:

perl -ne 'print if $. % 3 == 1'  < input.txt  > output.txt

The -e switch takes a piece of Perl code to execute as a command line parameter, while the -n switch implicitly wraps the code in a while loop like the one shown above.


Edit: To actually get lines 1, 3, 6, 9, ... as in the example, rather than lines 1, 4, 7, 10, ... as I first assumed you wanted, replace $. % 3 == 1 with $. == 1 or $. % 3 == 0.

share|improve this answer

If you want to do it with a Bash script you can try:

#!/bin/sh

echo Please enter the file name
read fname
echo Please enter the Nth lines that you want to keep
read n

exec<$fname
value=0
while read line
do
    if [ $(( $value % $n )) -eq 0 ] ; then
        echo -e "$line" >> new_file.txt
    fi
        let value=value+1 
done
echo "Check the 'new_file.txt' that has been created in this directory";

Save it as "read_lines.sh" and remember to give +x permissions to the bash file.

chmod +x ./read_lines.sh
share|improve this answer
1  
If you made this just emit on standard out, read the no of lines to skip from the arguments and read the file from standard in, it would be simpler and more useful. You could still make new_file.txt by doing ./read_lines.sh > new_file.txt. –  rjmunro Mar 7 '12 at 10:36

A solution in pure bash, that does not spawn a process is:

{ for f in {1..2}; do read line; done;
  while read line; do
    echo $line;
    for f in {1..2}; do read line; done;
  done; } < file

The first line skip 2 lines at the beginning of file, and the while print the next line and skip 2 lines again.

If your file is small, this is a very efficient way of doing the job as it does not start a process. When your file is large, sed should be used as it is more efficient at handling io than bash.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.