How can I use ffmpeg to accurately export images from a video file?

Question

My goal; export a series of images from a video file using ffmpeg. I guess my issue is related to frames per second and bit-rate.

I tried the following command: ffmpeg -i myVideo.mp4 -r 1 images_%04d.jpg but somehow on a 45 second video, I end up with 47 images. It's quite important I get this accurate. I won't know ahead of time what the appropriate FPS or bit-rates are, if I need those values when running the command, I need to be able to pull them from ffmpeg somehow.

I also considered exporting ALL the images on the video with ffmpeg -i myVideo images_%04d.jpg and dividing the number of images by the total seconds in the video. This will give me something like 24.97, and I'll round to 25 and delete 24 of every 25 frames. My fear is that if the file is VBR and the high bit-rate parts are in the beginning of the video, the frames I finish with won't match exactly 1 per second. For instance, the 30th image might actually appear in the video closer to second 31.

 running ffmpeg -> ffmpeg -i "/videos/11.mp4" -s "352x264" "/images/image%06d.jpg"
FFmpeg version 0.6-4:0.6-2ubuntu6.1, Copyright (c) 2000-2010 the FFmpeg developers
  built on Mar 31 2011 18:43:47 with gcc 4.4.5
  configuration: --extra-version=4:0.6-2ubuntu6.1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf 
--enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex 
--enable-libtheora --enable-libvorbis --enable-vaapi --enable-pthreads --enable-zlib --enable-libvpx 
--disable-stripping --enable-runtime-cpudetect --enable-gpl --enable-postproc --enable-x11grab 
--enable-libdc1394 --enable-shared --disable-static
  libavutil     50.15. 1 / 50.15. 1
  libavcodec    52.72. 2 / 52.72. 2
  libavformat   52.64. 2 / 52.64. 2
  libavdevice   52. 2. 0 / 52. 2. 0
  libavfilter    1.19. 0 /  1.19. 0
  libswscale     0.11. 0 /  0.11. 0
  libpostproc   51. 2. 0 / 51. 2. 0

Seems stream 1 codec frame rate differs from container frame rate: 49938.00 (49938/1) -> 24.97 (24969/1000)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/videos/11.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isomavc1mp42
  Duration: 00:00:32.60, start: 0.000000, bitrate: 433 kb/s
    Stream #0.0(und): Audio: aac, 44100 Hz, stereo, s16, 127 kb/s
    Stream #0.1(und): Video: h264, yuv420p, 352x264 [PAR 1:1 DAR 4:3], 303 kb/s, 24.97 fps, 24.97 tbr, 24969 tbn, 49938 tbc
Output #0, image2, to '/images/image%06d.jpg':
  Metadata:
    encoder         : Lavf52.64.2
    Stream #0.0(und): Video: mjpeg, yuvj420p, 352x264 [PAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 90k tbn, 24.97 tbc
Stream mapping:
  Stream #0.1 -> #0.0
Press [q] to stop encodingframe=  176 fps=  0 q=24.8 size=      -0kB time=7.05 bitrate=  
-0.0kbits/s    ^Mframe=  312 fps=236 q=24.8 size=      -0kB time=12.50 bitrate=  -0.0kbits/s    
^Mframe=  316 fps=112 q=24.8 size=      -0kB time=12.66 bitrate=  -0.0kbits/s    ^Mframe=  322 
fps= 55 q=24.8 size=      -0kB time=12.90 bitrate=  -0.0kbits/s    ^Mframe=  327 fps= 39 q=24.8 
size=      -0kB time=13.10 bitrate=  -0.0kbits/s    ^Mframe=  331 fps= 33 q=24.8 size=      
-0kB time=13.26 bitrate=  -0.0kbits/s    ^Mframe=  336 fps= 31 q=24.8 size=      -0kB time=13.46 
bitrate=  -0.0kbits/s    ^Mframe=  339 fps= 27 q=24.8 size=      -0kB 
time=13.58 bitrate=  -0.0kbits/s    ^Mframe=  344 fps= 22 q=24.8 size=      -0kB 
time=13.78 bitrate=  -0.0kbits/s 

Does anyone have ideas of how to get time-accurate results exporting images from video with ffmpeg?? Thanks!

Answer 1

You need to manually specify a -vsync value. The discrepancy is most likely due to the difference in variable/constant frame rate and frame timestamps. If you have a VFR source, ffmpeg by default will try to target an estimated rate (the tbr value) and will drop or duplicate frames in an effort to "smooth" the motion. Exporting frames as images follows this behavior as well. Some info from the documentation:

-vsync parameter

Video sync method. For compatibility reasons old values can be specified as numbers. Newly added values will have to be specified as strings always.

0, passthrough

Each frame is passed with its timestamp from the demuxer to the muxer.

1, cfr

Frames will be duplicated and dropped to achieve exactly the requested constant frame rate.

2, vfr

Frames are passed through with their timestamp or dropped so as to prevent 2 frames from having the same timestamp.

drop

As passthrough but destroys all timestamps, making the muxer generate fresh timestamps based on frame-rate.

-1, auto

Chooses between 1 and 2 depending on muxer capabilities. This is the default method.

As it says here, ffmpeg will select -vsync -1 by default (for both video encoding and image exporting). What you want is a straight passthrough with no changes. Use -vsync 0 if you want to keep every frame as-is with no drops or dupes. You'll probably get messages about non-monotonous DTS but this can be largely ignored. Here's an example command for outputting all frames accurately from a video:

ffmpeg -i video.nut -f image2 -vsync 0 frame-%03d.tiff

Answer 2

Are you sure you're losing frames? Your Input video is h264 which support variable frame rate. to get details about each frame I would recommend using the following:

ffmpeg -i inputvideo -vf showinfo -acodec copy -vcodec mpeg2video temp.mp4

This way you will know how many frames your movie have.

You can avoid variable frame rate by converting your movie to a non variable frame rate video, using the -r float argument, and then exporting the photos (as you did in your command).

Answer 3

I have observed similar erratic behavior on extracting images with the -r argument.

I found a solution that worked for me by using the -vf fps=X option instead (with some important notes below). Believe it or not the behavior here seems more predictable than the behavior with -r.

1. To generate an image every X seconds from the input video, compute the floating point value of 1/X and run:

ffmpeg -i input.mp4 -vf fps=1.0/x image-%04d.jpg

for example, to get 2 images a second:

ffmpeg -i input.mp4 fps=0.5 image-%04d.jpg

This will do nearly the right thing, note that:

1. An extra first frame is generated, which should be ignored.
2. Frames are made every X seconds, where X is the argument to -vf fps=X, starting at X/2 seconds.

So, running:

ffmpeg -i input.mp4 fps=1 image-%04d.jpg

On a 5 second video will result in 6 files, the first one should be ignored, and the 2nd-6th will correspond to timestamps: 0.5, 1.5, 2.5, 3.5, and 4.5.

Here is a useful test video for debugging this kind of thing:

https://drive.google.com/file/d/0B56RokrDs3xabS1TS19wRjlBaVk/view?usp=sharing