3

I am trying to use ffmpeg to combine multiple video files at different start times using the overlay filter.

When I do so without any audio the generated result is correct.

When I add in audio filter elements with amix to include the audio from the various overlayed videos, ffmpeg starts issuing errors of this kind:

[Parsed_overlay_4 @ 0x305ec40] [framesync @ 0x305ed68] Buffer queue overflow, dropping.

Any help in avoiding this, or alternative ways of:

  • Combining multiple videos clips
  • Setting the start time of each video clip
  • Having the audio for each clip play in sync with its corresponding video

would be much appreciated.

I've already tried lots of combinations of setpts/asetpts/amerge/pan and can't get it working.

Here is my ffmpeg command line and output for the case that causes an error (the if clause in the overlay y= is to cause the overlayed clips to scroll across the video while they play):

ffmpeg -y \
 -i /wintmp/vid/butterfly_base.mp4 \
 -i /wintmp/vid/party_overlay.mp4 \
 -i /wintmp/vid/car_overlay.mp4 \
 -pix_fmt yuv420p -r 30000/1001 -filter_complex " [1:v] setpts=PTS-STARTPTS+2/TB [v1] ; [0:v] [v1] overlay=x=50:y='if( gte(t,2), H-(t-2)*128, NAN)' [o1] ; [2:v] setpts=PTS-STARTPTS+3/TB,scale=width=320:height=240 [v2] ; [o1] [v2]  overlay=x=100:y='if( gte(t,3), H-(t-3)*102.4, NAN)' [outv] ; [1:a] adelay=2000|2000 [a1] ; [2:a] adelay=3000|3000 [a2] ; [0:a] [a1] [a2] amix=inputs=3:duration=longest [outa] " -map "[outv]" -map "[outa]" /wintmp/vid/combined.mp4
ffmpeg version N-81978-g03d6d5f Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  configuration: --enable-gpl --enable-libx264 --enable-nonfree --enable-libfdk-aac
  libavutil      55. 32.100 / 55. 32.100
  libavcodec     57. 61.102 / 57. 61.102
  libavformat    57. 51.107 / 57. 51.107
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 63.100 /  6. 63.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  2.100 /  2.  2.100
  libpostproc    54.  0.100 / 54.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/wintmp/vid/butterfly_base.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.48.100
  Duration: 00:00:08.01, start: 0.000000, bitrate: 550 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 480x272 [SAR 1:1 DAR 30:17], 412 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from '/wintmp/vid/party_overlay.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Teenage Fun, 1949 Style - http://www.archive.org/details/CEP503
    date            : 1949
    encoder         : Lavf55.48.100
    comment         : license:  http://creativecommons.org/licenses/publicdomain/
  Duration: 00:00:04.02, start: 0.000000, bitrate: 587 kb/s
    Stream #1:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 476 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #1:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 101 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Input #2, mov,mp4,m4a,3gp,3g2,mj2, from '/wintmp/vid/car_overlay.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.48.100
  Duration: 00:00:05.09, start: 0.000000, bitrate: 951 kb/s
    Stream #2:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 94 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
    Stream #2:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 854 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
[libx264 @ 0x3066780] using SAR=1/1
[libx264 @ 0x3066780] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
[libx264 @ 0x3066780] profile High, level 2.1
[libx264 @ 0x3066780] 264 - core 142 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/wintmp/vid/combined.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.51.107
    Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 480x272 [SAR 1:1 DAR 30:17], q=-1--1, 29.97 fps, 30k tbn, 29.97 tbc (default)
    Metadata:
      encoder         : Lavc57.61.102 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1: Audio: aac (LC) ([64][0][0][0] / 0x0040), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      encoder         : Lavc57.61.102 aac
Stream mapping:
  Stream #0:0 (h264) -> overlay:main
  Stream #0:1 (aac) -> amix:input0
  Stream #1:0 (h264) -> setpts
  Stream #1:1 (aac) -> adelay
  Stream #2:0 (aac) -> adelay
  Stream #2:1 (h264) -> setpts
  overlay -> Stream #0:0 (libx264)
  amix -> Stream #0:1 (aac)
Press [q] to stop, [?] for help
[Parsed_overlay_4 @ 0x305ec40] [framesync @ 0x305ed68] Buffer queue overflow, dropping.
    Last message repeated 15 times
frame=  241 fps=0.0 q=-1.0 Lsize=     608kB time=00:00:08.10 bitrate= 614.6kbits/s speed=9.07x    
video:472kB audio:126kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.638017%
[libx264 @ 0x3066780] frame I:1     Avg QP:24.29  size: 24368
[libx264 @ 0x3066780] frame P:117   Avg QP:24.94  size:  3290
[libx264 @ 0x3066780] frame B:123   Avg QP:31.72  size:   599
[libx264 @ 0x3066780] consecutive B-frames:  6.2% 72.2% 14.9%  6.6%
[libx264 @ 0x3066780] mb I  I16..4:  0.0% 39.4% 60.6%
[libx264 @ 0x3066780] mb P  I16..4:  0.1%  1.4%  1.1%  P16..4: 32.1% 19.9% 14.0%  0.0%  0.0%    skip:31.4%
[libx264 @ 0x3066780] mb B  I16..4:  0.0%  0.3%  0.2%  B16..8: 27.4%  4.9%  1.2%  direct: 1.3%  skip:64.7%  L0:35.2% L1:57.1% BI: 7.8%
[libx264 @ 0x3066780] 8x8 transform intra:52.0% inter:52.6%
[libx264 @ 0x3066780] coded y,uvDC,uvAC intra: 89.9% 91.2% 74.9% inter: 15.6% 15.7% 3.1%
[libx264 @ 0x3066780] i16 v,h,dc,p: 29% 33% 11% 27%
[libx264 @ 0x3066780] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 12% 12%  7%  9% 12%  8% 12% 11%
[libx264 @ 0x3066780] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 15% 12%  7% 10% 12%  7% 10%  7%
[libx264 @ 0x3066780] i8c dc,h,v,p: 44% 22% 23% 11%
[libx264 @ 0x3066780] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x3066780] ref P L0: 73.2% 15.2%  7.9%  3.7%
[libx264 @ 0x3066780] ref B L0: 90.6%  9.0%  0.4%
[libx264 @ 0x3066780] ref B L1: 98.2%  1.8%
[libx264 @ 0x3066780] kb/s:480.51
[aac @ 0x3068d40] Qavg: 515.774

Here are the source input videos:

Here is the output video, where you can notice the dropped frames in the overlayed video of the car, where the car in the video stops rotating during playback:

If I remove all the audio entirely by running this:

ffmpeg -y \
-i /wintmp/vid/butterfly_base.mp4 \
-i /wintmp/vid/party_overlay.mp4 \
-i /wintmp/vid/car_overlay.mp4 \
-pix_fmt yuv420p -r 30000/1001 -an -filter_complex " [1:v] setpts=PTS-STARTPTS+2/TB [v1] ; [0:v] [v1] overlay=x=50:y='if( gte(t,2), H-(t-2)*128, NAN)' [o1] ; [2:v] setpts=PTS-STARTPTS+3/TB,scale=width=320:height=240 [v2] ; [o1] [v2]  overlay=x=100:y='if( gte(t,3), H-(t-3)*102.4, NAN)' [outv] " -map "[outv]" /wintmp/vid/combined_no_audio.mp4

This no-audio version does not cause any errors and renders the video correctly: https://drive.google.com/file/d/0B56RokrDs3xaTmh0YkpmaGt0aUk/view?usp=sharing

The problem also seems related to the number of overlays. If I just do one overlay the problem does not occur.

5

This defect has been reported here before, but is not solved yet.

It's a buffer-related problem and using fifo, afifo filters can help. Try:

ffmpeg -y -i butterfly_base.mp4 -i party_overlay.mp4 -i car_overlay.mp4 -pix_fmt yuv420p -r 30000/1001 -filter_complex " [1:v]fifo, setpts=PTS-STARTPTS+2/TB [v1] ; [0:v]fifo[v0buf];[v0buf][v1] overlay=x=50:y='if( gte(t,2), H-(t-2)*128, NAN)' [o1] ; [2:v]fifo, setpts=PTS-STARTPTS+3/TB,scale=width=320:height=240 [v2] ; [o1] [v2]  overlay=x=100:y='if( gte(t,3), H-(t-3)*102.4, NAN)' [outv] ; [1:a]afifo, adelay=2000|2000 [a1] ; [2:a]afifo, adelay=3000|3000 [a2] ; [0:a]afifo[a0buf];[a0buf] [a1] [a2] amix=inputs=3:duration=longest [outa] " -map "[outv]" -map "[outa]" combined.mp4

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.