When I use "diff" to find the difference between two large data files,

diff A.txt B.txt > dif.info

it is output:

diff:memory exhausted

or add the argument "-H"

diff -H A.txt B.txt > dif.info

it also got the same message:

diff:memory exhausted

The size are about 1.5GB. How to compare large files ? Any other tools or solutions ? my PC: Redhat5.4 OS, 4GB memory. Thanks.

migrated from stackoverflow.com Dec 7 '12 at 15:41

This question came from our site for professional and enthusiast programmers.

  • The most simple solution would be to add more RAM (which is really cheap these days), assuming a 64 bits Linux. Otherwise try to split the files in smaller chunks. Perhaps add some more swap. – Basile Starynkevitch Dec 7 '12 at 13:58
  • any other software or script to handle large file? – Dong Dec 7 '12 at 14:05
  • What exactly are you trying to do, what do the files look like? – terdon Dec 7 '12 at 15:43

One idea would be to use split on both files and compare each chunk. It will not be exactly the same, but it all depends on what you are looking for.


try installing the textproc/2bsd-diff port


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.