Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Does anyone know of a way to check graphics files (particularly JPEG, GIF, and PNG) for corruption (preferably in an automated way)?


Explanation:

A few days ago, a command worked incorrectly and ended up deleting thousands of graphics files from a FAT32 volume that was practically out of space. I’ve used several different file/photo-recovery programs, but naturally, they are limited in how much they can recover (though fortunately the volume has 8KB clusters, which helps somewhat).

Anyway, some of the larger files, that were fragmented, are now corrupt. Some of them are not even real files at all (the recovery software merely dumped the clusters that were pointed to by now-overwritten directory entries), while others are broken because of fragmentation.

Moreover, because some picture formats embed a smaller version of the picture as a thumbnail, scanning the thumbnails for corruption is not reliable because it may be intact while the actual file (i.e., the picture when viewed full-size), could be corrupt.


Here are a couple of examples:

Here’s the second one. It’s so damaged that it doesn’t display anything.

damaged image

(A third one wouldn’t even upload because it doesn’t even have the correct header!)

share|improve this question
    
You mean visual corruption, I assume? I'd LOVE this...finally I could stop eyeballing the thumbnails of my comic books for broken jpgs. – Shinrai Apr 27 '11 at 19:24
    
Visual or structural. I found one app that supposedly did this, but it missed lots of files that didn’t even have the header! – Synetech Apr 27 '11 at 19:27
    
Oh, that stuff didn't even occur to me. Yes, please...this has to exist SOMEWHERE right? – Shinrai Apr 27 '11 at 19:55
1  
Can you upload one or more examples of such a broken file and link to them in your question? – slhck Apr 27 '11 at 20:19
    
@Shinrai, examining the thumbnails is not reliable because many picture formats include a separate thumbnail version embedded in the picture, and that may be intact. That’s why sometimes a picture whose thumbnail looks fine, is corrupt when opened. – Synetech Aug 17 '11 at 0:41

Try the jpeginfo '-c' option for your JPEG files.

I've seen the corruption you show happen with bad memory cards too.
What you want should be possible and available, check Corruption of Graphics Files;
a section from the online Encyclopedia of Graphics File Formats.

Also see File Integrity Checks in A Basic Introduction to PNG Features.

You may be interested in this Stackoverflow question,
How do I programmatically check whether an image (PNG, JPEG, or GIF) is corrupted?


Update: Source tarball for version 1.6.1 by Timo Kokkonen.
You should be able to build a binary for your machine.

share|improve this answer
    
Unfortunately, I can’t find any Windows ports. – Synetech Apr 28 '11 at 21:22
    
jpeginfo is open-source; you should be able to get the tarball and compile it on your system (maybe with Cygwin that has libjpeg). – nik May 3 '11 at 15:59
    
It’s moot either way, because I need to scan at least GIFs and PNGs as well. – Synetech Sep 9 '11 at 0:04
    
@nik - aux directory, which is part of jpeginfo's tarball cannot be created under that name on Windows, making this very hard to even extract under Windows, less alone build it. Have you managed to build it under Windows? – Rook Aug 30 '15 at 11:30

This can be done by using the Python Imaging Library's .verify() command.[1]

To run this in Windows, install Python (I installed the current latest release of Python 2), and then install Pillow (a fork of Python Imaging Library (PIL)). Then, copy the code of jpeg_corrupt.py[2] and save its contents to a .PY file, e.g. jpeg_corrupt.py.

Note that I changed the following line of code in jpeg_corrupt.py:
self.globs = ['*.jpg', '*.jpe', '*.jpeg']
to
self.globs = ['*.jpg', '*.jpe', '*.jpeg', '*.png', '*.gif']
This so .PNG and .GIF files will be scanned too.

It can then be executed through the Windows command prompt (cmd.exe) like this: C:\Python27\python.exe "C:\Directory containing the .PY file\jpeg_corrupt.py" "C:\Directory of folder to be scanned"

The first part of the command, 'C:\Python27\python.exe', might be different depending on which version of Python you installed and which directory you installed it to. In my example, it is the default installation directory of Python 2.7.

It should scan all JPG, GIF and PNG images in the specified directory and all of its subdirectories. It will show an output if it detects a corrupted image file.

I ran this on OP's sample image and it gave this error message: ...\YcB9n.png: string index out of range.

The code could also be entered in a .BAT script file, so you can easily run it a specified directory without needing to use the command prompt:

C:\Python27\python.exe "C:\Directory containing the .PY file\jpeg_corrupt.py" "%CD%"
pause



Sources:

[1]: Answer in Stack Overflow - "How do I programmatically check whether an image (PNG, JPEG, or GIF) is corrupted?" by ChristopheD
[2]: Comment by Denilson Sá in the SO answer linked in [1]

share|improve this answer
    
I deleted some parts of a jpg file randomly - the script shown nothing. It detects errors in the most worst cases only - when the header is completely missed for example... – Pavel Vlasov Aug 23 '15 at 11:14

Since I stumbled across this while trying to answer the same question I'll add another great solution I found:

Bad Peggy

Screenshot of the application

Usage
From the menu select File > Scan and then use the file dialog to browse to the folder in which the images are located. The program will then start scanning the folder and all subfolders for images (.jpg, .png, .bmp, .gif). If you want to scan a lot of pictures this will take some time, because the program needs fully load and parse the image file, so you might want to let it run overnight.

While it's scanning it'll show a progress percentage in the status bar. Any images it finds that are not perfect will show up directly in the list. If you click any image on the list, it will show a preview of what the image looks like. Quite often a image will only have a minor issue with the file format and the image will still look just fine. Other times the image will not render at all and the preview will be just black. Sometimes the image will be damaged and you'll see something like in the screenshot above.

A very handy trick is click in the column header on Reason and the images will be sorted according to how badly they're damaged (e.g. all the bad file formats that still render correctly will move to the bottom letting you focus on the more serious cases).

Also if the first scan has finished and you start another scan, the results will simply be added to the list. So if you have many different folders with images you can simply scan them sequentially without the list being cleared when you start a new scan. If you do want the list to clear, use the context menu and click Clear list.

Links
Downloads for Windows, Linux and OS X can be found here:
https://www.coderslagoon.com

Source code is here:
https://github.com/coderslagoon/badpeggy

share|improve this answer
    
Thanks for the fix. I've added a little usage information (even though the program is very self-explanatory). – Paul Jun 7 at 15:21

ImageMagick's identify program will let you know if an image is corrupt. A 'for i in find' loop testing for a none-0 return code from identify would let you script the test pretty easily to dump a list of damaged or corrupted files. It works on Windows with PowerShell too.

enter image description here

The following code with changes for your path works well in powershell

$stream = [System.IO.StreamWriter] "corrupt_jpegs.txt" 
get-childitem "c:\" -include *.jpg -recurse | foreach ($_) { 
    & "C:\Program Files\ImageMagick-6.7.1-Q16\identify.exe" $_.fullname > $null 
    if($LastExitCode -ne 0){ 
        $stream.writeline($_.fullname) 
    } 
} 
$stream.close()
share|improve this answer
    
I haven’t used ImageMagick in a while (it had bugs the last time I tried), but I’ll look into it. Thanks for the suggestion. – Synetech Aug 17 '11 at 0:39
    
The viewer tool is still buggy, but identify worked great for me with a similar problem. I used a powershell script like this to get a list of corrupt and or 0 length image files. – OldWolf Aug 17 '11 at 18:19
    
@Synetech inc. Sorry, can't update the original post with formatted code since an image was posted to it and I can't seem to get this to format nicely either. Sample Powershell script: (adjust your paths, file types etc..) $stream = [System.IO.StreamWriter] "corrupt_jpegs.txt" get-childitem "c:\" -include *.jpg -recurse | foreach ($_) { & "C:\Program Files\ImageMagick-6.7.1-Q16\identify.exe" $_.fullname > $null if($LastExitCode -ne 0){ $stream.writeline($_.fullname) } } $stream.close() – OldWolf Aug 17 '11 at 18:37
    
From the command line, identify can show corrupted JPEG data with -verbose, normally it doesn't show. – kenorb Feb 26 '15 at 18:23

I've modified the code from galacticninja's answer to do exactly what OP wanted. It is run in the same way, however it will move the files to a catch folder in the root C:\ directory instead of just listing the images on the command prompt.

You can find my modified code on Pastebin or below:

#This program will scan a directory and all it's subdirectories for corrupted jpg, png, gif, and bmp images and collect them in a Catch folder

#To run this program you will need to install Python 2.7 and PILLOW
#Once installed save this file in a notepad document with the .py extension
#Than run cmd.exe and type the following: C:\Python27\python.exe "C:\Directory this is saved in\this.py" "C:\Directory to be scanned"
#You must make a folder called Catch in your root C:\ directory for the corrupted images to be collected in


#!/usr/bin/env python2
# -*- coding: utf-8 -*-
# vi:ts=4 sw=4 et

# Okay, this code is a bit ugly, with a few "anti-patterns" and "code smell".
# But it works and I don't want to refactor it *right now*.

# TODO:
#  * Refactor it a little
#  * Add support for custom filename filter (instead of the hardcoded one)

#Big thanks to denilsonsa for writing most of this code at https://bitbucket.org/denilsonsa/small_scripts/src/542edd54d290d476603e939027ca654b25487d85/jpeg_corrupt.py?at=default


import getopt
import fnmatch
import re
import os
import os.path
import sys
import PIL.Image


available_parameters = [
    ("h", "help", "Print help"),
    ("v", "verbose", "Also print clean files"),
]


class ProgramOptions(object):
    """Holds the program options, after they are parsed by parse_options()"""

    def __init__(self):
        self.globs = ['*.jpg', '*.jpe', '*.jpeg', '*.gif', '*.png', '*.bmp']
        self.glob_re = re.compile('|'.join(
            fnmatch.translate(g) for g in self.globs
        ), re.IGNORECASE)

        self.verbose = False
        self.args = []


def print_help():
    global opt
    scriptname = os.path.basename(sys.argv[0])
    print "Usage: {0} [options] files_or_directories".format(scriptname)
    print "Recursively checks for corrupt image files"
    print ""
    print "Options:"
    long_length = 2 + max(len(long) for x,long,y in available_parameters)
    for short, long, desc in available_parameters:
        if short and long:
            comma = ", "
        else:
            comma = "  "

        if short == "":
            short = "  "
        else:
            short = "-" + short[0]

        if long:
            long = "--" + long

        print "  {0}{1}{2:{3}}  {4}".format(short,comma,long,long_length, desc)

    print ""
    print "Currently (it is hardcoded), it only checks for these files:"
    print "  " + " ".join(opt.globs)


def parse_options(argv, opt):
    """argv should be sys.argv[1:]
    opt should be an instance of ProgramOptions()"""

    try:
        opts, args = getopt.getopt(
            argv,
            "".join(short for short,x,y in available_parameters),
            [long for x,long,y in available_parameters]
        )
    except getopt.GetoptError as e:
        print str(e)
        print "Use --help for usage instructions."
        sys.exit(2)

    for o,v in opts:
        if o in ("-h", "--help"):
            print_help()
            sys.exit(0)
        elif o in ("-v", "--verbose"):
            opt.verbose = True
        else:
            print "Invalid parameter: {0}".format(o)
            print "Use --help for usage instructions."
            sys.exit(2)

    opt.args = args
    if len(args) == 0:
        print "Missing filename"
        print "Use --help for usage instructions."
        sys.exit(2)


def is_corrupt(imagefile):
    """Returns None if the file is okay, returns an error string if the file is corrupt."""
    #http://stackoverflow.com/questions/1401527/how-do-i-programmatically-check-whether-an-image-png-jpeg-or-gif-is-corrupted/1401565#1401565
    try:
        im = PIL.Image.open(imagefile)
        im.verify()
    except Exception as e:
        return str(e)
    return None


def check_files(files):
    """Receives a list of files and check each one."""
    global opt
    i = 0
    for f in files:
        # Filtering JPEG, GIF, PNG, and BMP images
        i=i+1
        if opt.glob_re.match(f):
            status = is_corrupt(f)
            if opt.verbose and status is None:
                status = "Ok"
            if status:
                file = "{0}".format(f, status)
                print file
                shorthand = file.rsplit('\\', 1)
                extention =shorthand[1]
                fullFileName = "C:\Catch" + "\\" + extention
                os.rename(file, fullFileName)


def main():
    global opt
    opt = ProgramOptions()
    parse_options(sys.argv[1:], opt)

    for pathname in opt.args:
        if os.path.isfile(pathname):
            check_files([pathname])
        elif os.path.isdir(pathname):
            for dirpath, dirnames, filenames in os.walk(pathname):
                check_files(os.path.join(dirpath, f) for f in filenames)
        else:
            print "ERROR: '{0}' is neither a file or a dir.".format(pathname)


if __name__ == "__main__":
    main()
share|improve this answer

Use identify from ImageMagick package.

Sample example:

identify -verbose -regard-warnings my_file.jpg >/dev/null && echo File is OK. || echo File is corrupted.

And the following command would identify all corrupted JPEG files in the current folder:

find . -name \*.jpg -exec identify -verbose -regard-warnings {} >/dev/null "+"
share|improve this answer

If you have Perl installed then you can use this script. You need to save the list of files to check in f.txt before you run the script. You can make this list using Irfanview. (load all thumbs from subfolders and save in txt). List of good files is saved in okf.txt and corrupted files are listed in brokenf.txt.

=====================

use Image::Magick;

open(BROKEN, ">>brokenf.txt");  # Open for appending
open(OK, ">>okf.txt");  # Open for appending
$list='f.txt';          
open(TOSORT, $list) or die("Could not open  file."); 
foreach $pic (<TOSORT>)  {     
    chomp($pic);   
    $p = new Image::Magick;
    $s = 0;    
    $error = $p->Read($pic);
        if ($error) {print BROKEN $pic . "\n";
                   }     
           else {
                  print OK $pic . "\n"; 
                }  
    }
close(TOSORT);
close(BROKEN);
close(OK);
    }

close(TOSORT);
close(BROKEN);
close(OK);
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.