1

I'm working on a video tracking experiment and got stuck with videos that are quite badly compressed with the MPEG4 DivX 5x/6x codec. I'm fairly new to image formats, codecs and compression, but I think I figured out that I'm stuck with this quality unless I violate the second law of thermodynamics.

Now, for tracking my insects (yeap, that's what I'm doing), I am only interested in I frames (frame rate is high enough), and I'm not interested in the color channels U and V, since they only have one value for every block and hence don't give me the resolution I want. It is the Y-channel that has all the info I'm interested in. I wrote my tracker myself and it cannot parse video, so it needs a folder with stills.

Now my question is: how can I extract all I-frames to gray scale (Y channel only) images WITHOUT any further quality loss? I'm working in ubuntu 14.04 and I'd preferentially use ffmpeg or imageJ, since they are already present in my pipeline. Where I am now is:

I think I figured out that every second frame is an I frame, but I'm not sure of that. I used:

ffprobe -show_frames movie.avi | grep -A2 "video" | grep "key_frame"

output: 
key_frame=1
key_frame=0
key_frame=1
key_frame=0
key_frame=1
key_frame=0
key_frame=1
key_frame=0
key_frame=1
key_frame=0

--
this goes on for exactly the number of frames, as this bit of code tells me: 
ffprobe -show_frames movie.avi | grep -A2 "video" | grep -c "key")
13369

Now, I thought I figured out how to extract every I frame:

ffmpeg -i movie.avi -vf '[in]select=eq(pict_type\,I)[out]' /picture%d.jpg         

But it seems to give me all frames back.

ls *jpg | wc -l
133370

What am I doing wrong? This is the output ffmpeg gives me:

ffmpeg version N-77455-g4707497 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04)
configuration: --extra-libs=-ldl --prefix=/opt/ffmpeg --mandir=/usr/share/man --enable-avresample --disable-debug --enable-nonfree --enable-gpl --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --disable-decoder=amrnb --disable-decoder=amrwb --enable-libpulse --enable-libdcadec --enable-libfreetype --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-libvorbis --enable-libmp3lame --enable-libopus --enable-libvpx --enable-libspeex --enable-libass --enable-avisynth --enable-libsoxr --enable-libxvid --enable-libvo-aacenc --enable-libvidstab
libavutil      55. 11.100 / 55. 11.100
libavcodec     57. 20.100 / 57. 20.100
libavformat    57. 20.100 / 57. 20.100
libavdevice    57.  0.100 / 57.  0.100
libavfilter     6. 21.101 /  6. 21.101
libavresample   3.  0.  0 /  3.  0.  0
libswscale      4.  0.100 /  4.  0.100
libswresample   2.  0.101 /  2.  0.101
libpostproc    54.  0.100 / 54.  0.100 
Guessed Channel Layout for  Input Stream #0.1 : stereo
Input #0, avi, from 'movie.avi':
Duration: 00:08:54.76, start: 0.000000, bitrate: 3006 kb/s
Stream #0:0: Video: mpeg4 (Simple Profile) (DX50 / 0x30355844), yuv420p, 720x576 [SAR 16:15 DAR 4:3], 1462 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, 2 channels, s16, 1536 kb/s
[swscaler @ 0x3c2e920] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to './picture%d.jpg':
Metadata:
encoder         : Lavf57.20.100
Stream #0:0: Video: mjpeg, yuvj420p(pc), 720x576 [SAR 16:15 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
Metadata:
  encoder         : Lavc57.20.100 mjpeg
Side data:
  unknown side data type 10 (24 bytes) 
Stream mapping:
Stream #0:0 -> #0:0 (mpeg4 (native) -> mjpeg (native))
Press [q] to stop, [?] for help

frame=13370 fps=506 q=24.8 Lsize=N/A time=00:08:54.80 bitrate=N/A dup=6685 drop=0 speed=20.2x    
video:157591kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

So, a couple of questions:

  • what am I doing wrong? Why does it give me all frames back?
  • Will jpeg cause further loss? Or is it the same compression as used intra-frame in mpeg4? Should I perhaps use tiff instead?
  • How do I only extract the y-channel?
  • Is it normal I get an I-frame every second frame? I have been reading into MPEG4 encoding a little bit and it seems that not entire frames, but rather blocks are used as reference? Am I then extracting all frames that contain such blocks? Is there a higher level with "real" entire reference frames?
  • I guess there is no way to recover more quality?

Many, many thanks for your help!

Best wishes,

Rik Verdonck

4

ffmpeg implicitly uses the frame rate of the source unless explicitly specified otherwise. If the number of frames supplied by the decoder/filter is different than that rate, then frames are duplicated or dropped to achieve it. This can be remedied by generating new timestamps for each selected frame, or specifying a framerate which matches the frequency of I-frames per 1 second of video. Safer to do the first.

You can use TIFF or PNG or BMP instead of JPEG to avoid further compression. Unsure of whether the prediction schemes are the same for JPEG and MPEG codecs.

An I-frame every other frame is unusual for a MPEG-4 codec, but you did say these were badly encoded. Someone set either a GOP i.e. keyframe interval of 2 or a very low scene change threshold, likely the former.

In summary, use

ffmpeg -i movie.avi -vf "select=eq(pict_type\,I),setpts=N/25/TB" -pix_fmt gray /picture%d.png 

Edited

For direct Y component extraction, use

ffmpeg -i movie.avi -vf "select=eq(pict_type\,I),setpts=N/25/TB,extractplanes=y" -pix_fmt gray /picture%d.png   
  • Looks good. Do you know though if -pix_fmt gray corresponds to the Y component of the input directly? – slhck Jan 30 '16 at 18:07
  • I thought so, but seems there's some filter applied - range expansion, looks like. Direct lift command added. – Gyan Jan 30 '16 at 19:15
  • Hi Mulvya, thank you for your help. It seems to work, but I get some output in my terminal that worries me a little. It looks like this: Past duration 0.999992 too large Last message repeated 30 times Past duration 0.999992 too largeN/A time=00:00:11.36 bitrate=N/A speed=22.7x Last message repeated 33 times Past duration 0.999992 too largeN/A time=00:00:23.48 bitrate=N/A speed=23.5x Repeated about 20 times. – Goat-Anti-Rabbit Jan 30 '16 at 21:54
  • Ignore it. Doesn't make a difference in the output. – Gyan Jan 31 '16 at 4:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.