Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

My ultimate goal is to get meaningful snapshots from videos that are either 30 min or 1 hour long. "Meaningful" is a bit ambitious, so I have simplified my requirements.

The image should be crisp - not blurry.

Initially, I thought this meant getting a "keyframe". Since there are many keyframes, I decided to choose the keyframe closest to the third minute of the video, which was generally "meaningful" enough for me. I followed the advice at: FFmpeg command to find key frame closest to 3rd minute

But the problem is that these keyframes are often (not always) blurry. An example is:

enter image description here

I then tried, Meaningful thumbnails for a Video using FFmpeg which did help get more meaningful snapshots, but I still often (not always) got blurry frames like the above.

You will notice that this sort of image is essentially an overlap of 2 different scenes. Sometimes, however, I get images that work for me – like this:

enter image description here

The above image is not very meaningful, but it is crisp.

Ideally, I would like to FFmpeg not to return blurry frames. Alternatively, I would like to use a script to detect blurry frames and select the least blurry from say 5 frames. Does anyone know how to do this?

share|improve this question
    
I tried using Face Detection as well. I theorized that if the identified face area overlaps then the image is blurry. I wasn't quite successful because it seems that faces are not easy to detect in video frames. The software requires almost "perfect" portrait photos. –  cnfcnf Feb 14 '13 at 11:19
    
I found this paper that might be relevant: research.microsoft.com/pubs/68802/… but it seems somewhat complex. –  cnfcnf Feb 14 '13 at 12:42
    
Does "blurry" for you mean that there are two overlapping images? Or do you have an example of a "blurry" thumbnail without any overlapping image? Note that there might be motion blur in the actual video itself—you can't get rid of that. If the person in the first image is actually moving, every still frame will exhibit a little motion blur. –  slhck Feb 14 '13 at 14:38
    
The paper you found doesn't really have a lot to do with your problem. It describes a blurring metric, i.e. a metric to determine how blurry an image is (at the coding layer). But that won't help you a lot since you'd have to implement it yourself. Also the paper seems quite old and way the authors prove the validity of the metric doesn't look too convincing to me. –  slhck Feb 14 '13 at 14:41
    
By "blurry", my top priority is overlapping images. That's the biggest problem. You pose an interesting question on whether there are blurry images that are not overlapping, but I haven't noticed that yet. I have found 2 other links: 1) stackoverflow.com/questions/5180327/… and 2) stackoverflow.com/questions/4470107/… The second link might be incorrect because my keyframes are from H.264 video. –  cnfcnf Feb 14 '13 at 14:57

1 Answer 1

"Meaningful" is quite subjective but "blurry" is fairly objective and simple to detect.

I had a similar issue and after a little bit of researching ended up with the following algorithm:

  • Determine video file length in seconds.
  • Generate 10 png thumbnails using ffmpeg and scene change detecion with something like:

    ffmpeg -ss {$skip} -i {$input} -vf "select=gt(scene\,0.3)" -r 1 -frames:v 1 {$output}
    

On each iteration $skip is increased by 10% on the video length in seconds. The parameter 0.3 may not be most appropriate for you, but you can play arround with it. This solved the problem with the overlapping scenes and/or completely blurred images.

  • Detect the edges of those 10 thumbnails and downscale them for better performance with ImageMagick:

    convert {$input} -thumbnail {$w}x{$h} -colorspace Gray -edge 1 {$path_to_downscaled_image}
    

I'm using -thumbnail to scale the image. Then I'm adding a black and white filter. Finally I do the edge detection with a radius of 1. This produces a black and white image only with the edges marked as white. This means that the more white the image has the more edges there are in it. You will get something like this:

enter image description here

  • Identify the resulting black and white images with ImageMagick:

    identify -format '{$format}' {$path_to_downscaled_image}
    

What you should be interested is %[mean] and %[standard-deviation]. Play arround with those and see what suits you best. Just sort all the resulting images by "%[mean] + %[standard-deviation]" for example. Find the same image before we started resizing and filtering it.

  • Finally pick the most "edgy" one, find its original and optionally convert it again:

    convert {$input} -thumbnail {$w}x{$h} -adaptive-sharpen 1.25x0.75 {$final_output}
    

I've found out that -adaptive-sharpen really helps with the end result because it sharpens the images only around those same edges. I tried different geometries and found out that 1.25x0.75 works best for me when I'm scaling down to a quarter of the original resolution.

I've done this in PHP and it takes about 25 seconds for a 12 minute movie to execute which is fine by me.

I hope this was helpful.

share|improve this answer
    
I haven't tried this yet, but the logic seems sound and I am overall quite impressed with how thorough this has been. Thank you! –  cnfcnf Jan 4 at 10:27
    
Give it a try. I would be more than happy to see what your results are. –  matthewd Jan 6 at 9:07

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.