FFmpeg can capture images from videos that can be used as thumbnails to represent the video. Most common ways of doing that are captured in the FFmpeg Wiki.
But, I don't want to pick random frames at some intervals. I found some options using filters on FFmpeg to capture scene changes:
thumbnail tries to find the most representative frames in the video:
ffmpeg -i input.mp4 -vf "thumbnail,scale=640:360" -frames:v 1 thumb.png
and the following command selects only frames that have more than 40% of changes compared to previous (and so probably are scene changes) and generates a sequence of 5 PNGs.
ffmpeg -i input.mp4 -vf "select=gt(scene\,0.4),scale=640:360" -frames:v 5 thumb%03d.png
Info credit for the above commands to Fabio Sonnati. The second one seemed better as I could get n images and pick the best. I tried it and it generated the same image 5 times.
Some more investigation led me to:
ffmpeg -i input.mp4 -vf "select=gt(scene\,0.5)" -frames:v 5 -vsync vfr out%02d.png
-vsync vfr ensures that you get different images. This still always picks the first frame of the video, in most cases the first frame is credits/logo and not meaningful, so I added a
-ss 3 to discard first 3 seconds of the video.
My final command looks like this:
ffmpeg -ss 3 -i input.mp4 -vf "select=gt(scene\,0.5)" -frames:v 5 -vsync vfr out%02d.jpg
This was the best I could do. I have noticed that since I pick only 5 videos , all of them are mostly from beginning of the video and may miss out on important scenes that occur later in the video
I would like to pick your brains for any other better options.