Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've got a multi-GB file that has elements in 4 lines, and I would like to have every 4 lines randomized in another file, this is, keeping each four lines grouped and randomize those sets. Is there an easy way to do that?

share|improve this question
1  
Do you want to keep the four lines grouped and randomize those sets, or do you want randomize the four lines but keep the groups in order? – Patches Jul 1 '11 at 12:30
    
keep each group of four lines grouped – 719016 Jul 1 '11 at 12:49
up vote 4 down vote accepted

If you're using a reasonable new linux/unix distribution, sort comes with a -R flag which randomises lines instead of sorts them. We can use that to create this one-liner solution:

awk '{printf("%s%s",$0,(NR%4==0)?"\n":"\0")}' file.txt | sort -R | tr "\0" "\n" > sorted.txt

First, use awk to group every 4 lines by replacing \n with \0. We then shuffle the lines using sort -R and finally restore the line breaks with tr.

share|improve this answer
    
The "\n is unclosed and should be "\n". I can't make such a small edit, so I'm just putting it here. – Ian Hunter Apr 19 '13 at 19:27
    
Also, for some reason on my system, I couldn't printf the "\0" and get it to replace properly. Instead, I used printf("%s%c",$0,(NR%4==0)?"\n":0). I did not need to change the tr command. – Ian Hunter Apr 19 '13 at 19:37

This is in Python. I'm sure someone will post a Perl answer too. ;-)

#!/usr/bin/python

import random

#Change these to the desired files
infile = "/path/to/input/file"
outfile = "/path/to/output/file"

fh = file(infile)
contents = fh.readlines()
fh.close()

chunked = [contents[i:i+4] for i in xrange(0, len(contents), 4)]
random.shuffle(chunked)

fh = file(outfile, 'w')

for chunk in chunked:
    for line in chunk:
        fh.write(line)

fh.close()

IANA Programmer so somebody could probably improve this, but I tested it and it works just fine.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .