37

Does anyone know of a way to check graphics files (particularly JPEG, GIF, and PNG) for corruption (preferably in an automated way)?


Explanation:

A few days ago, a command worked incorrectly and ended up deleting thousands of graphics files from a FAT32 volume that was practically out of space. I’ve used several different file/photo-recovery programs, but naturally, they are limited in how much they can recover (though fortunately the volume has 8KB clusters, which helps somewhat).

Anyway, some of the larger files, that were fragmented, are now corrupt. Some of them are not even real files at all (the recovery software merely dumped the clusters that were pointed to by now-overwritten directory entries), while others are broken because of fragmentation.

Moreover, because some picture formats embed a smaller version of the picture as a thumbnail, scanning the thumbnails for corruption is not reliable because it may be intact while the actual file (i.e., the picture when viewed full-size), could be corrupt.


Here are a couple of examples:

Here’s the second one. It’s so damaged that it doesn’t display anything.

damaged image

(A third one wouldn’t even upload because it doesn’t even have the correct header!)

10
  • You mean visual corruption, I assume? I'd LOVE this...finally I could stop eyeballing the thumbnails of my comic books for broken jpgs.
    – Shinrai
    Apr 27, 2011 at 19:24
  • Visual or structural. I found one app that supposedly did this, but it missed lots of files that didn’t even have the header!
    – Synetech
    Apr 27, 2011 at 19:27
  • Oh, that stuff didn't even occur to me. Yes, please...this has to exist SOMEWHERE right?
    – Shinrai
    Apr 27, 2011 at 19:55
  • 1
    Can you upload one or more examples of such a broken file and link to them in your question?
    – slhck
    Apr 27, 2011 at 20:19
  • 1
    I recently (re-)wrote a PHP script to scan graphics files. It is extremely promising and seems to give the most accurate results of all of my tests (other tools give lots of false positives and negatives). Once I work out the kinks, I’ll clean it up and post a version that supports graphics and archive files here. (I’ll figure something out for other types like executables later.)
    – Synetech
    Jul 20, 2013 at 22:20

10 Answers 10

21

Since I stumbled across this while trying to answer the same question I'll add another great solution I found:

Bad Peggy

Screenshot of the application

Usage
From the menu select File > Scan and then use the file dialog to browse to the folder in which the images are located. The program will then start scanning the folder and all subfolders for images (.jpg, .png, .bmp, .gif). If you want to scan a lot of pictures this will take some time, because the program needs fully load and parse the image file, so you might want to let it run overnight.

While it's scanning it'll show a progress percentage in the status bar. Any images it finds that are not perfect will show up directly in the list. If you click any image on the list, it will show a preview of what the image looks like. Quite often a image will only have a minor issue with the file format and the image will still look just fine. Other times the image will not render at all and the preview will be just black. Sometimes the image will be damaged and you'll see something like in the screenshot above.

A very handy trick is click in the column header on Reason and the images will be sorted according to how badly they're damaged (e.g. all the bad file formats that still render correctly will move to the bottom letting you focus on the more serious cases).

Also if the first scan has finished and you start another scan, the results will simply be added to the list. So if you have many different folders with images you can simply scan them sequentially without the list being cleared when you start a new scan. If you do want the list to clear, use the context menu and click Clear list.

Links
Downloads for Windows, Linux and OS X can be found here:
https://www.coderslagoon.com

Source code is here:
https://github.com/llaith/BadPeggy

2
  • Thanks for the fix. I've added a little usage information (even though the program is very self-explanatory).
    – Paul
    Jun 7, 2016 at 15:21
  • feedback: Bad Peggy did not detect a corrupted picture (neither did python/Pillow + ffmpeg).
    – JinSnow
    May 31, 2020 at 13:04
11

Try the jpeginfo '-c' option for your JPEG files.

I've seen the corruption you show happen with bad memory cards too.
What you want should be possible and available, check Corruption of Graphics Files;
a section from the online Encyclopedia of Graphics File Formats.

Also see File Integrity Checks in A Basic Introduction to PNG Features.

You may be interested in this Stackoverflow question,
How do I programmatically check whether an image (PNG, JPEG, or GIF) is corrupted?


Update: Source tarball for version 1.6.1 by Timo Kokkonen.
You should be able to build a binary for your machine.

6
  • Unfortunately, I can’t find any Windows ports.
    – Synetech
    Apr 28, 2011 at 21:22
  • jpeginfo is open-source; you should be able to get the tarball and compile it on your system (maybe with Cygwin that has libjpeg).
    – nik
    May 3, 2011 at 15:59
  • 1
    It’s moot either way, because I need to scan at least GIFs and PNGs as well.
    – Synetech
    Sep 9, 2011 at 0:04
  • 2
    @nik - aux directory, which is part of jpeginfo's tarball cannot be created under that name on Windows, making this very hard to even extract under Windows, less alone build it. Have you managed to build it under Windows?
    – Rook
    Aug 30, 2015 at 11:30
  • jpeginfo -c *.JPG | ag (WARNING|ERROR) worked for me
    – Selrond
    Aug 17, 2018 at 14:49
4

ImageMagick's identify program will let you know if an image is corrupt. A 'for i in find' loop testing for a none-0 return code from identify would let you script the test pretty easily to dump a list of damaged or corrupted files. It works on Windows with PowerShell too.

enter image description here

The following code with changes for your path works well in powershell

$stream = [System.IO.StreamWriter] "corrupt_jpegs.txt" 
get-childitem "c:\" -include *.jpg -recurse | foreach ($_) { 
    & "C:\Program Files\ImageMagick-6.7.1-Q16\identify.exe" $_.fullname > $null 
    if($LastExitCode -ne 0){ 
        $stream.writeline($_.fullname) 
    } 
} 
$stream.close()
4
  • I haven’t used ImageMagick in a while (it had bugs the last time I tried), but I’ll look into it. Thanks for the suggestion.
    – Synetech
    Aug 17, 2011 at 0:39
  • 1
    The viewer tool is still buggy, but identify worked great for me with a similar problem. I used a powershell script like this to get a list of corrupt and or 0 length image files.
    – OldWolf
    Aug 17, 2011 at 18:19
  • @Synetech inc. Sorry, can't update the original post with formatted code since an image was posted to it and I can't seem to get this to format nicely either. Sample Powershell script: (adjust your paths, file types etc..) $stream = [System.IO.StreamWriter] "corrupt_jpegs.txt" get-childitem "c:\" -include *.jpg -recurse | foreach ($_) { & "C:\Program Files\ImageMagick-6.7.1-Q16\identify.exe" $_.fullname > $null if($LastExitCode -ne 0){ $stream.writeline($_.fullname) } } $stream.close()
    – OldWolf
    Aug 17, 2011 at 18:37
  • 2
    From the command line, identify can show corrupted JPEG data with -verbose, normally it doesn't show.
    – kenorb
    Feb 26, 2015 at 18:23
4

This can be done by using the Python Imaging Library's .verify() command.[1]

To run this in Windows, install Python (I installed the current latest release of Python 2), and then install Pillow (a fork of Python Imaging Library (PIL)). Then, copy the code of jpeg_corrupt.py[2] and save its contents to a .PY file, e.g. jpeg_corrupt.py.

Note that I changed the following line of code in jpeg_corrupt.py:
self.globs = ['*.jpg', '*.jpe', '*.jpeg']
to
self.globs = ['*.jpg', '*.jpe', '*.jpeg', '*.png', '*.gif']
This so .PNG and .GIF files will be scanned too.

It can then be executed through the Windows command prompt (cmd.exe) like this: C:\Python27\python.exe "C:\Directory containing the .PY file\jpeg_corrupt.py" "C:\Directory of folder to be scanned"

The first part of the command, 'C:\Python27\python.exe', might be different depending on which version of Python you installed and which directory you installed it to. In my example, it is the default installation directory of Python 2.7.

It should scan all JPG, GIF and PNG images in the specified directory and all of its subdirectories. It will show an output if it detects a corrupted image file.

I ran this on OP's sample image and it gave this error message: ...\YcB9n.png: string index out of range.

The code could also be entered in a .BAT script file, so you can easily run it a specified directory without needing to use the command prompt:

C:\Python27\python.exe "C:\Directory containing the .PY file\jpeg_corrupt.py" "%CD%"
pause



Sources:

[1]: Answer in Stack Overflow - "How do I programmatically check whether an image (PNG, JPEG, or GIF) is corrupted?" by ChristopheD
[2]: Comment by Denilson Sá in the SO answer linked in [1]

3
  • 6
    I deleted some parts of a jpg file randomly - the script shown nothing. It detects errors in the most worst cases only - when the header is completely missed for example... Aug 23, 2015 at 11:14
  • 1
    Exactly, the same goes for jpeginfo.
    – wp78de
    Jul 15, 2017 at 4:20
  • feedback: python/Pillow + ffmpegdid not detect a corrupted picture (neither did Bad Peggy).
    – JinSnow
    May 31, 2020 at 13:05
3

I've modified the code from galacticninja's answer to do exactly what OP wanted. It is run in the same way, however it will move the files to a catch folder in the root C:\ directory instead of just listing the images on the command prompt.

You can find my modified code on Pastebin or below:

#This program will scan a directory and all it's subdirectories for corrupted jpg, png, gif, and bmp images and collect them in a Catch folder

#To run this program you will need to install Python 2.7 and PILLOW
#Once installed save this file in a notepad document with the .py extension
#Than run cmd.exe and type the following: C:\Python27\python.exe "C:\Directory this is saved in\this.py" "C:\Directory to be scanned"
#You must make a folder called Catch in your root C:\ directory for the corrupted images to be collected in


#!/usr/bin/env python2
# -*- coding: utf-8 -*-
# vi:ts=4 sw=4 et

# Okay, this code is a bit ugly, with a few "anti-patterns" and "code smell".
# But it works and I don't want to refactor it *right now*.

# TODO:
#  * Refactor it a little
#  * Add support for custom filename filter (instead of the hardcoded one)

#Big thanks to denilsonsa for writing most of this code at https://bitbucket.org/denilsonsa/small_scripts/src/542edd54d290d476603e939027ca654b25487d85/jpeg_corrupt.py?at=default


import getopt
import fnmatch
import re
import os
import os.path
import sys
import PIL.Image


available_parameters = [
    ("h", "help", "Print help"),
    ("v", "verbose", "Also print clean files"),
]


class ProgramOptions(object):
    """Holds the program options, after they are parsed by parse_options()"""

    def __init__(self):
        self.globs = ['*.jpg', '*.jpe', '*.jpeg', '*.gif', '*.png', '*.bmp']
        self.glob_re = re.compile('|'.join(
            fnmatch.translate(g) for g in self.globs
        ), re.IGNORECASE)

        self.verbose = False
        self.args = []


def print_help():
    global opt
    scriptname = os.path.basename(sys.argv[0])
    print "Usage: {0} [options] files_or_directories".format(scriptname)
    print "Recursively checks for corrupt image files"
    print ""
    print "Options:"
    long_length = 2 + max(len(long) for x,long,y in available_parameters)
    for short, long, desc in available_parameters:
        if short and long:
            comma = ", "
        else:
            comma = "  "

        if short == "":
            short = "  "
        else:
            short = "-" + short[0]

        if long:
            long = "--" + long

        print "  {0}{1}{2:{3}}  {4}".format(short,comma,long,long_length, desc)

    print ""
    print "Currently (it is hardcoded), it only checks for these files:"
    print "  " + " ".join(opt.globs)


def parse_options(argv, opt):
    """argv should be sys.argv[1:]
    opt should be an instance of ProgramOptions()"""

    try:
        opts, args = getopt.getopt(
            argv,
            "".join(short for short,x,y in available_parameters),
            [long for x,long,y in available_parameters]
        )
    except getopt.GetoptError as e:
        print str(e)
        print "Use --help for usage instructions."
        sys.exit(2)

    for o,v in opts:
        if o in ("-h", "--help"):
            print_help()
            sys.exit(0)
        elif o in ("-v", "--verbose"):
            opt.verbose = True
        else:
            print "Invalid parameter: {0}".format(o)
            print "Use --help for usage instructions."
            sys.exit(2)

    opt.args = args
    if len(args) == 0:
        print "Missing filename"
        print "Use --help for usage instructions."
        sys.exit(2)


def is_corrupt(imagefile):
    """Returns None if the file is okay, returns an error string if the file is corrupt."""
    #http://stackoverflow.com/questions/1401527/how-do-i-programmatically-check-whether-an-image-png-jpeg-or-gif-is-corrupted/1401565#1401565
    try:
        im = PIL.Image.open(imagefile)
        im.verify()
    except Exception as e:
        return str(e)
    return None


def check_files(files):
    """Receives a list of files and check each one."""
    global opt
    i = 0
    for f in files:
        # Filtering JPEG, GIF, PNG, and BMP images
        i=i+1
        if opt.glob_re.match(f):
            status = is_corrupt(f)
            if opt.verbose and status is None:
                status = "Ok"
            if status:
                file = "{0}".format(f, status)
                print file
                shorthand = file.rsplit('\\', 1)
                extention =shorthand[1]
                fullFileName = "C:\Catch" + "\\" + extention
                os.rename(file, fullFileName)


def main():
    global opt
    opt = ProgramOptions()
    parse_options(sys.argv[1:], opt)

    for pathname in opt.args:
        if os.path.isfile(pathname):
            check_files([pathname])
        elif os.path.isdir(pathname):
            for dirpath, dirnames, filenames in os.walk(pathname):
                check_files(os.path.join(dirpath, f) for f in filenames)
        else:
            print "ERROR: '{0}' is neither a file or a dir.".format(pathname)


if __name__ == "__main__":
    main()
1
3

Install imagemagick, if you're on Mac you can use Homebrew.

brew update && brew install imagemagick

Then you can use this small Python script.

import os
from subprocess import Popen, PIPE

def checkImage(fn):
    proc = Popen(['identify', '-verbose', fn], stdout=PIPE, stderr=PIPE)
    out, err = proc.communicate()
    exitcode = proc.returncode

    return exitcode, out, err

for directory, subdirectories, files in os.walk('/Your/Path/To/Files/'):
    for file in files:
        filePath = os.path.join(directory, file)
        code, output, error = checkImage(filePath)
        if code != 0 or error != '':
            print(str(code)+' '+error)
            #os.remove(filePath)

Replace /Your/Path/To/Files/ and uncomment the last line if you want to delete the corrupted images.

2

Use identify from ImageMagick package.

Sample example:

identify -verbose -regard-warnings my_file.jpg >/dev/null && echo File is OK. || echo File is corrupted.

And the following command would identify all corrupted JPEG files in the current folder:

find . -name \*.jpg -exec identify -verbose -regard-warnings {} >/dev/null "+"
2

My open source Pyhton script check-media-integrity checks integrity of pictures, and video/audio files. It uses Pillow modules, ImageMagick and FFmpeg wrappers to try decoding the files.

Pillow image.verify does not see all the defects (e.g., ignores truncation) for this reason I also performed image/decode + manipulation.

1

This blog post lists five tools that can (detect and) repair corrupted image files. The only free one among them is File Repair 2.1.

0

If you have Perl installed then you can use this script. You need to save the list of files to check in f.txt before you run the script. You can make this list using Irfanview. (load all thumbs from subfolders and save in txt). List of good files is saved in okf.txt and corrupted files are listed in brokenf.txt.

=====================

use Image::Magick;

open(BROKEN, ">>brokenf.txt");  # Open for appending
open(OK, ">>okf.txt");  # Open for appending
$list='f.txt';          
open(TOSORT, $list) or die("Could not open  file."); 
foreach $pic (<TOSORT>)  {     
    chomp($pic);   
    $p = new Image::Magick;
    $s = 0;    
    $error = $p->Read($pic);
        if ($error) {print BROKEN $pic . "\n";
                   }     
           else {
                  print OK $pic . "\n"; 
                }  
    }
close(TOSORT);
close(BROKEN);
close(OK);
    }

close(TOSORT);
close(BROKEN);
close(OK);

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .