Map overlay audio with FFMpeg?

Question

Me and my friend each took a video of us playing online with each other, and now I want to make a single video comparing them side by side.

I’ve done this before, but I’ve never needed to deal with the audio. What I want to do is take the audio stream from one clip, and the audio stream from the other clip, and have them in separate streams in the new clip, similar to the way DVD videos have two separate streams for each language.

This is what I tried using ffmpeg:

ffmpeg -i "Fraps 2014-11-14 07-51-04-70_cut.mp4" -vf "movie='FFsplit-141106-222552_cut.mp4' [in1]; [in]fps=fps=30, setpts=PTS-STARTPTS, scale=1680:1040, pad=iw*2:ih:iw:0[in0]; [in0][in1] overlay=0:0 [out]" input_sidebyside.mp4

As far as video goes, everything works, the videos are in sync. But only the left audio stream is copied. The -map doesn’t seem to recognize the movie filter’s audio at all. I tried -map 1:a:0 but it tells me that stream doesn’t exist.

Is it possible map audio from the overlay filter in ffmpeg?

Answer 1

Indeed, overlay only works on video. You need to amix the two audio sources. -map 1 will not work since the second movie is loaded by a filter. You may want to use two -i inputs and amix their audio:

ffmpeg \
-i "Fraps 2014-11-14 07-51-04-70_cut.mp4" \
-i "FFsplit-141106-222552_cut.mp4" \
-filter_complex "[0:v]fps=fps=30, setpts=PTS-STARTPTS, scale=1680:1040, \
  pad=iw*2:ih:iw:0[in0]; [in0][1:v] overlay=0:0 [outv]; \
[0:a][1:a] amix [outa]" -map "[outv]" -map "[outa]" input_sidebyside.mp4

Something like the above. Don't forget to specify encoders and quality settings. For example, you may want to explicitly say -c:v libx264 -crf 22 to set the x264 quality.