Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm looking to open a massive text file on my mac. The file is just over 10Gigs. I won't get into how it was created or what it's use is, since it isn't really relevant to the question.

Are there any mac programs which will allow me to open such a massive file, or perhaps break it up into chunks (say 200mb or something each) which would be easier to open individually.

I don't necessarily need to edit it, just view portions of it. Everything I've tried, Textmate, Vim etc etc all (understandably) crap out or take forever to load.

I also run Windows and Linux, but moving such a large file around is not easy. It currently resides on my Mac and I would rather not have the hassle of moving it.

share|improve this question

migrated from stackoverflow.com Aug 28 '10 at 16:03

This question came from our site for professional and enthusiast programmers.

    
Similar to superuser.com/questions/138474/… , but this question also accepts an alternative to split the file –  Doug Harris Aug 28 '10 at 17:41

5 Answers 5

up vote 14 down vote accepted

split can do it -

 split -b 200m [file]

will split the file into 200MB chunks. You can split on lines with -l

 split -l 1000 [file]

will split the file into 1000 line chunks.

If that doesn't work, there is a really good hex editor for Mac, Hex Fiend which can open massive files:

Work with huge files. Hex Fiend can handle as big a file as you’re able to create. It’s been tested on files as large as 118 GB.

Hope that helps!

share|improve this answer
    
This works like a charm. Thanks –  Chris Salij Aug 30 '10 at 6:54
    
It's Unix, so it must work. :) –  maaartinus Feb 6 '11 at 17:54
    
Hex Friend is just awesome!! Thanks so much for the mention –  lukaswelte Mar 15 at 0:21

Have you tried any of the following command line tools:

  • sed or awk: stream editors, but these can at times require complex expressions to get the desired output
  • split: splits files into blocks of given number of bytes, lines, etc.
  • csplit: splits file based on context given by regular expression

One of split or csplit might do the job, depending on how they manage memory use. Check out this page for more Info on split and csplit .

For even more info on these tools try your favourite search engine, or type man <cmd-name> at the command line in the Terminal.

If you can provide more context on how you want to split the files, I'm sure someone with expert knowledge of sed/awk or regular expressions could help.

share|improve this answer

I'd recommend checking out HexFiend - while also a hex editor it does read the file directly from the hard drive and doesn't try to load it into memory like most other text editors. If you want to hide the hex portion uncheck "Hexadecimal" from the View menu.

share|improve this answer

Another option would be to use the built-in commands head and tail.

To get the first 1000 lines:

head -1000 hugefile

To get the last 1000 lines:

tail -1000 hugefile

Or to get lines from 1,000,000 to 1,001,000:

head -1001000 hugefile | tail -1000

This might take quite a lot of time though.

share|improve this answer
    
I like this option. If I only wanted to take it a few chunks at a time. It doesn't suit my purpose now, but I'll defiantly be using it in the future :) Thanks –  Chris Salij Aug 30 '10 at 6:54

I have need for my work to view huge text files, and have been looking for years at all alternatives, although only on Windows.

The only text editor I ever found that could truly work efficiently on huge files is V - The File Viewer.

Although commercial, at $20 it's worth it if you need to edit such files frequently.
If you only need to view one particular file, it has a fully-functional trial version.

Transferring 10 GB thru the network to a Windows machine shouldn't be unbearably slow.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.