Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

When I use "diff" to find the difference between two large data files,

diff A.txt B.txt >

it is output:

diff:memory exhausted

or add the argument "-H"

diff -H A.txt B.txt >

it also got the same message:

diff:memory exhausted

The size are about 1.5GB. How to compare large files ? Any other tools or solutions ? my PC: Redhat5.4 OS, 4GB memory. Thanks.

share|improve this question

migrated from Dec 7 '12 at 15:41

This question came from our site for professional and enthusiast programmers.

The most simple solution would be to add more RAM (which is really cheap these days), assuming a 64 bits Linux. Otherwise try to split the files in smaller chunks. Perhaps add some more swap. – Basile Starynkevitch Dec 7 '12 at 13:58
any other software or script to handle large file? – Dong Dec 7 '12 at 14:05
What exactly are you trying to do, what do the files look like? – terdon Dec 7 '12 at 15:43
up vote 4 down vote accepted

One idea would be to use split on both files and compare each chunk. It will not be exactly the same, but it all depends on what you are looking for.

share|improve this answer

try installing the textproc/2bsd-diff port

share|improve this answer
I'm trying it. Thanks – Dong Dec 7 '12 at 14:07
but how to install it? – Dong Dec 7 '12 at 14:18
Google is a good friend to know. :)… – whunmr Dec 7 '12 at 14:22

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.