Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have 5000 pictures and other files in a directory taking up 35 GB. I want to duplicate this directory.

Method 1: I do a simple copy and paste of the directory in explorer. I have the habit of checking the checksums after copying important files. In this case I noticed that around 2000 files failed the MD5 test. At a closer inspection of a randomly chosen JPEG with different checksums it turns out that some XMP metadata had changed. In particular, the tag

<MicrosoftPhoto:DateAcquired>

had changed the date from 2009 to today (possibly around the time I was copying the files). I have no idea what triggered this XMP data to be changed and exactly when it was changed and why for these particular files, but at least it seems to explain the checksum discrepancy.

Method 2: As I want the exact files to be duplicated, I tried the program FreeFileSync to mirror the directory, hoping no XMP metadata would mysteriously change. A checksum test in addition to a thorough file comparison test in FreeFileSync lead to two similar but yet different results: 31 files fail the checksum test, 23 files fail the file comparison test. The smaller set is not entirely contained in the bigger set, although many files occur in both. What is alarming here is that not only JPEGs are flagged as altered but also som AVIs, MPGs and a large 7-zip file. Closer inspection of a JPEG indicates that it is indeed corrupt: the bottom half of the picture is simply plain gray. Due to the size of the 7-zip file, I have not been able to pin down the discrepancy.

Note, in both methods, every file has its correct file size after being copied.

Question: Any thoughts on what is possibly going on here? I have never had this problem before, and I am now terrified that files get corrupted after simple actions like copy/paste and file sync. Even if I manage to successfully copy the files somehow, I would still like an explanation to this.

share|improve this question

3 Answers 3

up vote 2 down vote accepted

As well as checking your hard drive as ewanm89 suggested, you should check your RAM. Go to Control Panel, type in "memory" in the search box, and select "Diagnose your computer's memory problems".

You can also try other tools such as MemTest86.

Also check the System log in Event Viewer and look for errors.

Copying files really shouldn't change change them.

share|improve this answer
    
Coincidentally, I had RAM problems when I bought the computer (brand new) a few months ago. I had the RAM replaced and used MemText86. My RAM is now fine. I will check the System log, and double check memory. Thanks. –  DustByte Dec 12 '12 at 10:56
    
I declare your answer to be the solution. After many, many hours of error checking with mysterious results, it turned out to be the RAM - again! –  DustByte Dec 12 '12 at 23:29
    
@DustByte, maybe your RAM timing settings in the BIOS are wrong. You could check those. You could try setting them back to defaults if they aren't already. It's possible also that the problem is with your motherboard rather than with the RAM. –  dangph Dec 12 '12 at 23:46
    
Computer was shipped brand new only a few months ago. No BIOS settings have been changed. This set of four RAM modules were fine back in August. I run extensive memtests back then. Computer was initially shipped with faulty RAM, which I had replaced. Now I wonder what made the RAM go faulty all of a sudden? I can see there are manufacturing faults on brand new RAM, but why did this happen after a few months? Any thoughts? –  DustByte Dec 13 '12 at 10:19
    
@DustByte, I don't know why RAM goes bad. It's surprising that this second lot would go bad, unless they were from a same bad batch. Or maybe that brand is incompatible with the motherboard. You could try reseating the RAM modules. That can sometimes fix things. Ground yourself first so you don't zap anything. Memory tests aren't completely definitive if they don't find problems. You should run them for some hours. There's still the possibility that the motherboard or perhaps the power supply are faulty. –  dangph Dec 13 '12 at 12:56

Files can in theory get corrupted just sat on the disk as Earths magnetic field is constantly trying to realign the little magnets and random background radiation spike collide with the platters...

As you pointed out windows may be changing some of the metadata in the first copy.

As for the second copy attempt however in this particular case it is much more likely windows wrote them to a bad sector, I suggest running chkdsk with the right options to check the disk and update the bad sectors list. Finally activate SMART monitoring for your disks and see if the disk controller itself is reporting the disk is on the way out.

Ultimately this is why one should always keep good backups. Also this is the kind of errors raid helps avoid.

share|improve this answer
    
For the record: the disk is only a couple of months old (which doesn't mean it's not faulty). What are "the right options" for chkdsk? Could chkdsk delete/corrupt data on the disk? –  DustByte Dec 12 '12 at 8:52
    
It shouldn't hmm and the disk has never experienced a major shock? I can't remember the exact options at the moment and I'm not using Windows to look it up either. Also corruption can happen in between reading the file and writing it elsewhere. Is this basically to move the files on a single disk or between 2 disks? –  ewanm89 Dec 12 '12 at 9:00
    
I just performed copy/paste from one physical drive to another, and it seems like it works. My checksum program indicated 19 errors but a couple of random checks indicate that the files are indeed the same! I need to investigate this further. I am still baffled why the synchronisation failed. What initially lead to me check checksums is that the backup program Areca refuses to back up my files as it fails on some hashs during checking - a problem I still need to resolve. The backup in this case was onto a third hard drive. –  DustByte Dec 12 '12 at 9:46
    
@DustByte, go to Computer Management / Disk Management / Right-click on the disk / Tools tab / Check. –  dangph Dec 12 '12 at 10:15
    
On Unix systems there is a tool called rsync that can copy just changes to files there are ways to run it in windows however it is a command line tool with a lot of options that need to be got right. You might find it useful. –  ewanm89 Dec 12 '12 at 11:02

It's most probably a RAM problem. I had similar problems - moving files changed their checksums (very rarely, mostly in very big files at or above 1GB size and not always) and very often: archives were corrupt when created, mostly 7zip archives. So it seems like creating 7z archives with almost maximum RAM usage is a pretty good indicator of RAM problems. I ran Memtest86 and found a damaged bar, swapped it and had no problems since.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.