Open massive file on Mac?

Question

I'm looking to open a massive text file on my mac. The file is just over 10Gigs. I won't get into how it was created or what it's use is, since it isn't really relevant to the question.

Are there any mac programs which will allow me to open such a massive file, or perhaps break it up into chunks (say 200mb or something each) which would be easier to open individually.

I don't necessarily need to edit it, just view portions of it. Everything I've tried, Textmate, Vim etc etc all (understandably) crap out or take forever to load.

I also run Windows and Linux, but moving such a large file around is not easy. It currently resides on my Mac and I would rather not have the hassle of moving it.

Answer 1

split can do it -

 split -b 200m [file]

will split the file into 200MB chunks. You can split on lines with -l

 split -l 1000 [file]

will split the file into 1000 line chunks.

If that doesn't work, there is a really good hex editor for Mac, Hex Fiend which can open massive files:

Work with huge files. Hex Fiend can handle as big a file as you’re able to create. It’s been tested on files as large as 118 GB.

Hope that helps!

Answer 2

Have you tried any of the following command line tools:

• sed or awk: stream editors, but these can at times require complex expressions to get the desired output
• split: splits files into blocks of given number of bytes, lines, etc.
• csplit: splits file based on context given by regular expression

One of split or csplit might do the job, depending on how they manage memory use. Check out this page for more Info on split and csplit .

For even more info on these tools try your favourite search engine, or type man <cmd-name> at the command line in the Terminal.

If you can provide more context on how you want to split the files, I'm sure someone with expert knowledge of sed/awk or regular expressions could help.

Answer 3

I'd recommend checking out HexFiend - while also a hex editor it does read the file directly from the hard drive and doesn't try to load it into memory like most other text editors. If you want to hide the hex portion uncheck "Hexadecimal" from the View menu.

Answer 4

Another option would be to use the built-in commands head and tail.

To get the first 1000 lines:

head -1000 hugefile

To get the last 1000 lines:

tail -1000 hugefile

Or to get lines from 1,000,000 to 1,001,000:

head -1001000 hugefile | tail -1000

This might take quite a lot of time though.

Answer 5

I have need for my work to view huge text files, and have been looking for years at all alternatives, although only on Windows.

The only text editor I ever found that could truly work efficiently on huge files is V - The File Viewer.

Although commercial, at $20 it's worth it if you need to edit such files frequently.
If you only need to view one particular file, it has a fully-functional trial version.

Transferring 10 GB thru the network to a Windows machine shouldn't be unbearably slow.