1

So yeah, having a tricky ffmpeg issue here.

In my app I'm encoding a video based on image frames and add a WAV file for the sound from the same source but through a different web socket. It can happen that the WAV file doesn't always have exactly the same length as the video based on those images + fps (plus minus few seconds).

How can I tell ffmpeg to "stretch" the input WAV file to have it the same length as the video?

In short: images (i) + WAV (w) = final video (v)

length(w) != length (v) ... which is causing problems.

Here an example ffmpeg command I am currently using:

ffmpeg -f image2 -thread_queue_size 512 -framerate 21.5 -i /frames/%d.webp -i videomail_preview.wav -y -acodec aac -strict experimental -ac 1 -vcodec libx264 -crf 16 -preset fast -profile:v baseline -pix_fmt yuv420p -loglevel warning -movflags +faststart videomail_good.mp4 -acodec libvorbis -ac 1 -vcodec libvpx -crf 8 -deadline good -cpu-used 1 -pix_fmt yuv420p -loglevel warning -movflags +faststart videomail_good.webm

And here the ffmpeg version + its extensions

ffmpeg version 2.8.6-1ubuntu2 Copyright (c) 2000-2016 the FFmpeg developers built with gcc 5.3.1 (Ubuntu 5.3.1-11ubuntu1) 20160311 configuration: --prefix=/usr --extra-version=1ubuntu2 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv libavutil 54. 31.100 / 54. 31.100 libavcodec 56. 60.100 / 56. 60.100 libavformat 56. 40.101 / 56. 40.101 libavdevice 56. 4.100 / 56. 4.100 libavfilter 5. 40.101 / 5. 40.101 libavresample 2. 1. 0 / 2. 1. 0 libswscale 3. 1.101 / 3. 1.101 libswresample 1. 2.101 / 1. 2.101 libpostproc 53. 3.100 / 53. 3.100

Any suggestions/trick I am unaware of?

  • You should show the complete console output from your command: it will show if your ffmpeg supports librubberband. – llogan Oct 5 '16 at 23:01
  • oh, added the ffmpeg config output. dont have librubberband in it. would librubberband help solving my issue? – Michael Heuberger Oct 6 '16 at 1:10
  • I was hoping to see the complete console output so I could give you an exact answer with the correct filter values, but Behroozfar provided an answer that you can adapt. – llogan Oct 6 '16 at 18:26
3

rubberband will probably sound better than atempo; especially for larger ratio differences. If you want to use rubberband but don't have support for this filter in your ffmpeg (and don't want to compile ffmpeg), then you can use the standalone rubberband tool.

rubberband example

rubberband -D 12 input.wav output.wav
  • -D<X>, --duration <X> - Stretch or squash to make output file X seconds long

See rubberband -h or rubberband command line utility help text for more info.

Step by step process

Unfortunately this won't be a single command. Using this method you will have to:

  1. Encode the video.
  2. Get the video duration with ffprobe.
  3. Generate the audio with rubberband.
  4. Mux the video and audio:

    ffmpeg -i video -i audio -c copy -shortest output
    

Additional notes

  • -movflags +faststart Is suitable for MP4, but is ignored by the webm muxer.
  • FFmpeg version 2.8.6 is old. Users are recommended to use a build from the current git master branch. The easiest solution is to download a static build, but you can compile if you prefer. Compiling will allow you to use the rubberband filter in ffmpeg.
  • An old version? Well I installed it via apt-get from launchpad.net/ubuntu/+source/ffmpeg/7:2.8.6-1ubuntu2 ... there is no newer version I can install via apt-get at the moment – Michael Heuberger Oct 8 '16 at 1:59
  • i wonder if there is a newer apt-get package that also comes with the rubberband tool? – Michael Heuberger Oct 8 '16 at 2:24
  • @MichaelHeuberger rubberband-cli is available, but you don't have to use an old ffmpeg from a repo. Just download a standalone binary. – llogan Oct 8 '16 at 2:55
  • bumped to ubuntu 16.10 which comes with ffmpeg v3 shipped and that worked – Michael Heuberger Nov 3 '16 at 6:19
  • but i realise this is making my code a lot more complicated. isn't there a trick just to run this all within a single ffmpeg command? – Michael Heuberger Nov 3 '16 at 6:42
1

There are 2 option in FFMPEG Audio filters that may help you: in both scenario you must manually calculate the ratio of RATIO=(VIDEO_DURATION/AUDIO_DURATION) and put it in command.

  1. atempo : with this filter you can speed up or slow down your audio. for your goal ffmpeg -f image2 -thread_queue_size 512 -framerate 21.5 -i /frames/%d.webp -i videomail_preview.wav -y -acodec aac -strict experimental -ac 1 -vcodec libx264 -crf 16 -preset fast -profile:v baseline -pix_fmt yuv420p -loglevel warning -movflags +faststart -filter:a "atempo=RATIO" videomail_good.mp4 -acodec libvorbis -ac 1 -vcodec libvpx -crf 8 -deadline good -cpu-used 1 -pix_fmt yuv420p -loglevel warning -movflags +faststart -filter:a "atempo=RATIO" videomail_good.webm. IMPORTANT note : The atempo filter is limited to using values between 0.5 and 2.0. If you need to, you can get around this limitation by stringing multiple atempo filters together.

  2. rubberband : with this, you can have more control on stretching your audio in compare with atempo filter. one basic command is : ffmpeg -f image2 -thread_queue_size 512 -framerate 21.5 -i /frames/%d.webp -i videomail_preview.wav -y -acodec aac -strict experimental -ac 1 -vcodec libx264 -crf 16 -preset fast -profile:v baseline -pix_fmt yuv420p -loglevel warning -movflags +faststart -filter:a "rubberband=tempo=RATIO" videomail_good.mp4 -acodec libvorbis -ac 1 -vcodec libvpx -crf 8 -deadline good -cpu-used 1 -pix_fmt yuv420p -loglevel warning -movflags +faststart -filter:a "rubberband=tempo=RATIO" videomail_good.webm. this filter has more options such as pitch, transients, channels and more.

  • is there no way i can do this in one command that also includes ratio calculations? – Michael Heuberger Nov 3 '16 at 6:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.