3

The Problem

I have a broken Windows Server software RAID-5 array that I would like to backup, but I want to avoid backing up all the broken files.

I'd really love for there to be some kind of script or program that can scan by HDD for corrupt files and simply delete them.

My Google-Fu returns results with how to recover or repair those files- I just want to delete them.

How I managed to break it:

I migrated my software RAID 5 from Windows Server 2008 R2 to new hardware and Windows Server 2012 R2. The move was more or less successful; I just needed to do some resynchronising. I forgot that it was resynchronising and turned it off to move it's physical location.

Oops.

After weeks of CHKDSK and SeaTools and other random attempts at recovering the drives, I've decided to give up on recovery and rebuild the whole thing whilst salvaging as much as I can.

EDIT

The work around

Using ffmeg kinda worked but it but I didn't have a clean and automated solution in Windows Server 2012. My PowerShell attempt took too long and yielded an un-readable log file.

Instead, I installed FileZilla Server and I have simply queued all of my files to be copied to my backup devices and any bad files end up in the Failed transfers tab of the FileZilla Client.

2
  • 1
    How much data and what file format is it? If it is structured data you may be able to script testing each file. Sep 5, 2015 at 0:08
  • Around 10 TB of data. All video files, mostly mkv, some avi etc. Sep 5, 2015 at 0:11

1 Answer 1

2

Good news we have a somewhat quick test for valid MKV and AVI files.

How can I check the integrity of a video file (avi, mpeg, mp4…)?

You can use a feature in ffmpeg video converter: if you will specify it to recode video to nothing it will just read input file and report any errors that will appear. This is very fast process because video frames are just being read, checked and silently dropped.

Loop though all avi and mkv files with:

ffmpeg.exe -v error -i file.avi -f null - >error.log 2>&1

Example loop for all files in one directory: (add for /R to use recursion)

FOR %%c IN (*.mkv) DO ffmpeg.exe -v error -i %%c -f null - >error.log 2>&1

Windows For Loops info

Edit Here is the Linux bash version I tested works:

for c in *.mp4; do ffmpeg -v error -i $c -f null - >error.log 2>&1; done

Now grep error log for which files are bad and delete them. I dont know what the output looks like so you may want to post sample output for assistance with that last step or split into another question.

Example output with corrupt file (chopped off end of file), errors can vary significantly:

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x161a180] moov atom not found
video-head.mp4: Invalid data found when processing input
5
  • 1
    That sounds amazing! I'll attempt when I get some time. Thanks! Sep 5, 2015 at 0:24
  • The first line works fine for individual files, but the second line returns "%%c was unexpected at this time" Sep 7, 2015 at 22:15
  • 1
    Confirmed it worked with a test on linux, will need to attempt the windows version as well. I made some tweaks which may fix it. Sep 7, 2015 at 22:45
  • Oh boy, it's really broken. I wrote a little bit of PowerShell to iterate through all mkv and avi files given a path and then pipe append to the error log. I started that before work, 8 hours later the log is 100 MB, much of it lines like: [null @ 0000000002f506c0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 47381184 >= 47376576 Sep 8, 2015 at 7:34
  • Sorry to hear you have so much corruption. You might also want to stop logging after the first 1 or 5 lines of errors on each file and kill the ffmpeg. That may clean up your log files and make it faster Sep 9, 2015 at 14:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .