up vote 2 down vote favorite
share [g+] share [fb]

I have a large text file, that I need to split into a series of smaller (under 1MB) files. It seems that split is what I want here.

However, if I try the following (using 900k as the limit to give some buffer before the hard limit of 1MB):

split -a 2 -b 900k total.sql createtables_

It will make each part exactly 900k (except the last part), most likely splitting lines at places where it splits the file.

So, is there a way to make it only split on line boundaries? In other words, have it make the pieces as close as possible (without going over) the limit specified in the -b argument, ensuring it only splits on line boundaries?

Or is the only option to use the -l option to split to split based on line count, and just use trial and error to find the parameter than keeps all files under the limit?

Or is split not the right tool to be using?

link|improve this question
89% accept rate
feedback

2 Answers

up vote 2 down vote accepted

split -l is the best you can do natively on Mac OS X. With GNU coreutils, you can use split -C 900k. You can install GNU coreutils on OSX, for example through Darwion Ports or Fink or MacPorts.

link|improve this answer
1  
coreutils is also available with Homebrew, which is a pretty solid package manager. – speedofmac Dec 10 '10 at 3:35
Thanks, I ended up installing GNU coreutils from Macports, and I am all set now. – pkaeding Dec 10 '10 at 15:15
feedback
up vote 1 down vote

Not sure what version of split you're using, but have you tried the --line-bytes=BYTES option? On my system it can also be entered as -C BYTES.

link|improve this answer
I am using the version that is included in Mac OS 10.6 (I couldn't find a version number, but the man page lists Aug. 21, 2005 as the date). My version doesn't accept -C or --line-bytes options. – pkaeding Dec 9 '10 at 19:03
The man page for mine shows split 5.97 February 2010. I'm not too familiar with Mac OS, but it is Unix based so you may be able to install a newer version of split. – LeoB Dec 9 '10 at 19:09
Unix-based, yes, but BSD-based, while you're using GNU-based Linux. There are many differences between dialects of Unix-like systems often falling roughly under BSD, System V and GNU. While there's plenty of overlap and cross-pollination, "newer" isn't necessarily a well-defined delineator due to philosophical differences. As Gilles points out, on a Mac you have the option of installing GNU utilities side-by-side with the native ones. – Dennis Williamson Dec 10 '10 at 18:08
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.