Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have some text files that are 100 to 300 MB in length that I want to view in Emacs, but my Emacs gets into some performance issues opening and traversing such large files. Therefore, I'm looking for an easy utility to split a file up into manageable chunks, say 50 MB each and to name the chunks based upon the original name plus some suffix to indicate their location in the sequence. Each chunk would pick up where the prior one was truncated and be no longer than 50 MB, with the last chunk possibly being shorter.

Is there any easy tool to do such on a Linux computer, perhaps something like head or tail that gives multiple results, one for each chunk?

e.g., given file test.out that is 120 MB long, break it up into test.out.1 for the first 50 MBs, test.out.2 for the second 50 MBs, and test.out.3 for the remainder 20 MBs at the end of the file.

I could use combinations of head and tail to get the pieces, but I'd like a tool that abstracts this all out, perhaps a Perl script or Python script someone has already created to do such a task?

share|improve this question

1 Answer 1

up vote 14 down vote accepted

There already is a nice tool for that: split

> man 1 split 

NAME
     split -- split a file into pieces

SYNOPSIS
     split [-l line_count] [-a suffix_length] [file [prefix]]
     split -b byte_count[K|k|M|m|G|g] [-a suffix_length] [file [prefix]]
     split -p pattern [-a suffix_length] [file [prefix]]


split --bytes 50M test.out test.out_ would split the file test.out into test.out_xaa, test.out_xab, test.out_xac, ...

A much uglier solution would be to use dd

dd if=test.out of=test.out.part1 bs=1M count=50 skip=0 creates a file named test.out.part1 with the first 50M from test.out. You can increase the value for skip to 1 to get the second chunk, to 2 for the third etc etc. Just make sure to also change the filenames or you will end up overwriting the same output file.

share|improve this answer
    
On at least my Linux, a lowercase 'm' is required for Megabytes. –  WilliamKF Sep 11 '13 at 5:32
    
Yet another new command learned ☺ –  Ramchandra Apte Sep 11 '13 at 5:39
    
@williamKF: Which flavour of Linux was that? I checked on Ubuntu and FreeBSD. FreeBSD needed the -b 50M (No option for --bytes). And Ubuntu accepted both the regular m and capital M. –  Hennes Sep 11 '13 at 10:40
    
@Hennes split (coreutils) 5.2.1 February 2011 on my Linux centos4 32 bit 2.6.9-103.ELsmp #1 SMP Fri Dec 9 04:31:51 EST 2011 i686 i686 i386 GNU/Linux –  WilliamKF Sep 11 '13 at 15:44

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.