Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

When I use "diff" to find the difference between two large data files,

diff A.txt B.txt > dif.info

it is output:

diff:memory exhausted

or add the argument "-H"

diff -H A.txt B.txt > dif.info

it also got the same message:

diff:memory exhausted

The size are about 1.5GB. How to compare large files ? Any other tools or solutions ? my PC: Redhat5.4 OS, 4GB memory. Thanks.

share|improve this question

migrated from stackoverflow.com Dec 7 '12 at 15:41

This question came from our site for professional and enthusiast programmers.

    
The most simple solution would be to add more RAM (which is really cheap these days), assuming a 64 bits Linux. Otherwise try to split the files in smaller chunks. Perhaps add some more swap. –  Basile Starynkevitch Dec 7 '12 at 13:58
    
any other software or script to handle large file? –  Dong Dec 7 '12 at 14:05
    
What exactly are you trying to do, what do the files look like? –  terdon Dec 7 '12 at 15:43

2 Answers 2

up vote 4 down vote accepted

One idea would be to use split on both files and compare each chunk. It will not be exactly the same, but it all depends on what you are looking for.

share|improve this answer

try installing the textproc/2bsd-diff port

http://www.freshports.org/textproc/2bsd-diff/

share|improve this answer
    
I'm trying it. Thanks –  Dong Dec 7 '12 at 14:07
    
but how to install it? –  Dong Dec 7 '12 at 14:18
1  
Google is a good friend to know. :) dir.filewatcher.com/d/FreeBSD/5.5-release/i386/… –  whunmr Dec 7 '12 at 14:22

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.