How can I use ffmpeg to add music to a video being built from individual images with -filter_complex?

Question

I'm using -filter_complex to build a video out of several images, and fade them together. This is working fine. Now I'd like to add an audio track to the video, the command fails. How can I add audio to this video file I'm generating?

Here is what I'm doing that isn't working:

ffmpeg \
-i sunlight.mp3 \
-loop 1 -t 1 -i 1.jpg \
-loop 1 -t 1 -i 2.jpg \
-loop 1 -t 1 -i 3.jpg \
-loop 1 -t 1 -i 4.jpg \
-loop 1 -t 1 -i 5.jpg \
-filter_complex \
"[1:v][0:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b1v]; \
[2:v][1:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b2v]; \
[3:v][2:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b3v]; \
[4:v][3:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b4v]; \
[0:v][b1v][1:v][b2v][2:v][b3v][3:v][b4v][4:v]concat=n=9:v=1:a=0,format=yuv420p[v]" -map "[v]" out.mp4

Here is the ffmpeg output from this:

    ffmpeg version 3.1.3 Copyright (c) 2000-2016 the FFmpeg developers
        built with Apple LLVM version 7.3.0 (clang-703.0.31)
        configuration: --prefix=/usr/local/Cellar/ffmpeg/3.1.3 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libxvid --disable-lzma --enable-vda
        libavutil      55. 28.100 / 55. 28.100
        libavcodec     57. 48.101 / 57. 48.101
        libavformat    57. 41.100 / 57. 41.100
        libavdevice    57.  0.101 / 57.  0.101
        libavfilter     6. 47.100 /  6. 47.100
        libavresample   3.  0.  0 /  3.  0.  0
        libswscale      4.  1.100 /  4.  1.100
        libswresample   2.  1.100 /  2.  1.100
        libpostproc    54.  0.100 / 54.  0.100
    Input #0, mp3, from 'sunlight.mp3':
        Metadata:
            encoder         : Lavf57.41.100
        Duration: 00:02:05.68, start: 0.025057, bitrate: 128 kb/s
            Stream #0:0: Audio: mp3, 44100 Hz, stereo, s16p, 128 kb/s
            Metadata:
                encoder         : Lavc57.48
    Input #1, image2, from '1.jpg':
        Duration: 00:00:00.04, start: 0.000000, bitrate: 42613 kb/s
            Stream #1:0: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 1200x900 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 25 tbn, 25 tbc
    Input #2, image2, from '2.jpg':
        Duration: 00:00:00.04, start: 0.000000, bitrate: 44551 kb/s
            Stream #2:0: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 1200x900 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 25 tbn, 25 tbc
    Input #3, image2, from '3.jpg':
        Duration: 00:00:00.04, start: 0.000000, bitrate: 40433 kb/s
            Stream #3:0: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 1200x900 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 25 tbn, 25 tbc
    Input #4, image2, from '4.jpg':
        Duration: 00:00:00.04, start: 0.000000, bitrate: 57157 kb/s
            Stream #4:0: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 1200x900 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 25 tbn, 25 tbc
    Input #5, image2, from '5.jpg':
        Duration: 00:00:00.04, start: 0.000000, bitrate: 51745 kb/s
            Stream #5:0: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 1200x900 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 25 tbn, 25 tbc
    Stream specifier ':v' in filtergraph description [1:v][0:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b1v]; [2:v][1:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b2v]; [3:v][2:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b3v]; [4:v][3:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b4v]; [0:v][b1v][1:v][b2v][2:v][b3v][3:v][b4v][4:v]concat=n=9:v=1:a=0,format=yuv420p[v] matches no streams.

When I run the command without the -i sunlight.mp3 it works fine.

Answer 1

You have two methods to try:

Manually define output streams with -map

I usually recommend manually mapping your streams instead of relying on the default stream selection behavior so you don't get unexpected results, but this requires that you:

1. Label your filter outputs like you did with the [v]
2. Use -map

Example:

ffmpeg \
-loop 1 -t 1 -i 1.jpg \
-loop 1 -t 1 -i 2.jpg \
-loop 1 -t 1 -i 3.jpg \
-loop 1 -t 1 -i 4.jpg \
-loop 1 -t 1 -i 5.jpg \
-i sunlight.mp3 \
-filter_complex \
"[1:v][0:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b1v]; \
[2:v][1:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b2v]; \
[3:v][2:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b3v]; \
[4:v][3:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b4v]; \
[0:v][b1v][1:v][b2v][2:v][b3v][3:v][b4v][4:v]concat=n=9:v=1:a=0,format=yuv420p[v]" -map "[v]" -map 5:a -shortest out.mp4

The stream specifier values you use in your filtergraph and in your -map options depend on the order of your inputs.

Use the default stream selection behavior

Since your final output contains only one video stream and one audio stream you can omit -map and the output label ([v]) and just use the default stream selection behavior. This may be the easier method to use if you're making a script.

ffmpeg \
-loop 1 -t 1 -i 1.jpg \
-loop 1 -t 1 -i 2.jpg \
-loop 1 -t 1 -i 3.jpg \
-loop 1 -t 1 -i 4.jpg \
-loop 1 -t 1 -i 5.jpg \
-i sunlight.mp3 \
-filter_complex \
"[1:v][0:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b1v]; \
[2:v][1:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b2v]; \
[3:v][2:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b3v]; \
[4:v][3:v]blend=all_expr='A*(if(gte(T,0.5),1,T/0.5))+B*(1-(if(gte(T,0.5),1,T/0.5)))'[b4v]; \
[0:v][b1v][1:v][b2v][2:v][b3v][3:v][b4v][4:v]concat=n=9:v=1:a=0,format=yuv420p" -shortest out.mp4

Answer 2

Your script have two basic error:

1. in first filterchain (see first line of filter_complex) you specify 0:v as an input stream. but as ffmpeg output says: Stream specifier ':v' in filtergraph description ... matches no streams. there is no video stream in file 0. note that you send a mp3 file as your first input. put it as last input file and ...

2. Add a -map 5:a before output file name.You specified video stream [v] as your only output stream. When using -map option, you must define all output streams with that.