Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

What would be the best tool / method to automatically find a position from an video file, where the frame approximately matches* a given image?

Basically, a command to: "Find the position where [image.jpg] appears in the [video.mpg]"

Preferably with ffmpeg, or some other linux command-line tool.

* - or maybe the frame which best of all of the frames in the video, matches the image

share|improve this question
If something like this exists, then it's really cool – That Brazilian Guy Oct 22 '13 at 20:58
I would think there are some image matching libraries somewhere - combining those with a tool that extracts frames from video shouldn't really be that hard. If such doesn't exist, I'm quite tempted to write one myself. :) – Ilari Kajaste Oct 22 '13 at 21:06
There are some amazing image matching tools out there! (Although this one is not for linux, it will compare different image formats, dimensions, color depths, and even partial images; so maybe there is a similar CLI tool or library for linux that you can pipe outputs?) – That Brazilian Guy Oct 23 '13 at 0:56
So, have you wrote such a tool yourself? Would've been really useful. – user Jun 2 '14 at 14:32
@user3075942 Not yet, but the concept is still on my todo. :) – Ilari Kajaste Jun 2 '14 at 14:33

You can use the md5 and framemd5 muxers if the image and frame are exactly the same. Example:

  1. Get MD5 sum of target image:

    $ ffmpeg -i frame.jpg -f md5 - 2>&1 | grep MD5
  2. Find exact image frame in video:

    $ ffmpeg -i -f framemd5 - 2>&1 | grep b7fb5124a65108ebb067129d9d81ed57
      0,        62,        62,         1,  1424400, b7fb5124a65108ebb067129d9d81ed57

The output numbers refer to: stream_index, packet_dts, packet_pts, packet_duration, packet_size, MD5.

However, this is probably not what you're looking for. Conceivably this will work if is mjpeg; such as if you make a video from jpg inputs and stream copy them to the output:

ffmpeg -pattern_type glob -i "*.jpg" -codec copy output.mkv
share|improve this answer
You're right, exact match is not what I'm looking for. – Ilari Kajaste Oct 23 '13 at 4:26
@IlariKajaste Consider submitting a feature request to the FFmpeg Bug Tracker. Mention any libraries that may do what you want if you find any. – LordNeckbeard Oct 23 '13 at 5:51
A feature like this might be a bit out of scope as direct feature of FFmpeg. But with a small script and some piping this might be possible. I don't know, though, which is why I'm asking. :) – Ilari Kajaste Oct 23 '13 at 6:58
@IlariKajaste There are already several "detect" filters in ffmpeg: silencedetect, volumedetect, blackdetect, blackframe, cropdetect, edgedetect, so a "framedetect" may not be out of scope. – LordNeckbeard Oct 23 '13 at 17:54
Ah, yes, good point! – Ilari Kajaste Oct 24 '13 at 5:51

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.