Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I've got a multi-GB file that has elements in 4 lines, and I would like to have every 4 lines randomized in another file, this is, keeping each four lines grouped and randomize those sets. Is there an easy way to do that?

share|improve this question
1  
Do you want to keep the four lines grouped and randomize those sets, or do you want randomize the four lines but keep the groups in order? –  Patches Jul 1 '11 at 12:30
    
keep each group of four lines grouped –  130490868091234 Jul 1 '11 at 12:49
add comment

2 Answers

up vote 3 down vote accepted

If you're using a reasonable new linux/unix distribution, sort comes with a -R flag which randomises lines instead of sorts them. We can use that to create this one-liner solution:

awk '{printf("%s%s",$0,(NR%4==0)?"\n":"\0")}' file.txt | sort -R | tr "\0" "\n" > sorted.txt

First, use awk to group every 4 lines by replacing \n with \0. We then shuffle the lines using sort -R and finally restore the line breaks with tr.

share|improve this answer
    
The "\n is unclosed and should be "\n". I can't make such a small edit, so I'm just putting it here. –  Ian Hunter Apr 19 '13 at 19:27
    
Also, for some reason on my system, I couldn't printf the "\0" and get it to replace properly. Instead, I used printf("%s%c",$0,(NR%4==0)?"\n":0). I did not need to change the tr command. –  Ian Hunter Apr 19 '13 at 19:37
add comment

This is in Python. I'm sure someone will post a Perl answer too. ;-)

#!/usr/bin/python

import random

#Change these to the desired files
infile = "/path/to/input/file"
outfile = "/path/to/output/file"

fh = file(infile)
contents = fh.readlines()
fh.close()

chunked = [contents[i:i+4] for i in xrange(0, len(contents), 4)]
random.shuffle(chunked)

fh = file(outfile, 'w')

for chunk in chunked:
    for line in chunk:
        fh.write(line)

fh.close()

IANA Programmer so somebody could probably improve this, but I tested it and it works just fine.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.