2

This seems to be a common problem, but the answers I found don't seem to apply to my situation.

I'm merging 6 videos (one of them is a short background that loops) into 1 output video, and the audio and video are off by a little bit (maybe a quarter second). Here's my command line:

ffmpeg -y
-ss 0.2 -i "One.MOV"
-ss 3.2 -i "Two.MOV"
-i "Three.mp4"
-ss 1.9 -i "Four.mp4"
-ss 1.9 -i "Five.MOV"
-stream_loop 180 -i "Background.mp4"
-filter_complex "
   nullsrc=size=1920x1080 [back];
   [5:v] scale=1920x1080 [fred];
   [back][fred] overlay=shortest=1:x=0:y=0 [base];
   [0:v] crop=1312:1080:290:0,scale=576x474 [clip0];
   [0:a]volume=1.7[aud0];
   [1:v] crop=1080:1056:0:864,scale=497x486 [clip1];
   [1:a]volume=4.8,aresample=async=1:first_pts=0[aud1];
   [2:v] scale=576x324 [clip2];
   [2:a]volume=2.0,aresample=async=1:first_pts=0[aud2];
   [3:v] crop=673:720:322:0,scale=454x486 [clip3];
   [3:a]volume=6.9,aresample=async=1:first_pts=0[aud3];
   [4:v] crop=720:921:0:359,scale=380x486 [clip4];
   [4:a]volume=4.9,aresample=async=1:first_pts=0[aud4];
   [base][clip0] overlay=shortest=1:x=32:y=83 [tmp0];
   [tmp0][clip1] overlay=shortest=1:x=711:y=27 [tmp1];
   [tmp1][clip2] overlay=shortest=1:x=1312:y=158 [tmp2];
   [tmp2][clip3] overlay=shortest=1:x=93:y=567 [tmp3];
   [tmp3][clip4] overlay=shortest=1:x=770:y=567 [tmp4];
   [aud0][aud1][aud2][aud3][aud4]amix=inputs=5[a]
" -map "[tmp4]" -map "[a]" -c:v libx264 "Output.mp4"

I'm guessing it has to do with the aresample and first_pts but I've been reading the documentation and can't figure out what I need to do differently.

How can I get my audio and video to synchronize?

EDIT

Note: "Three.mp4" is the piano accompaniment; the other tracks are individual singers, adding their voices to the piano.

3
  • what if you add aresample to [0:a]? Oct 27, 2020 at 4:34
  • Alas, setting that filter to "[0:a]volume=1.7,aresample=async=1:first_pts=0[aud0]" didn't change the asynchronicity. Oct 27, 2020 at 13:48
  • - How can I get my audio and video to synchronize? - i.e. audacity?
    – Hastur
    Oct 31, 2020 at 11:02

2 Answers 2

1

I suggest making a script so that you can include files one by one. to find out which file is faulty.

Create files in new folder, structure is next: test1.mov test1.ss test1.filterv test1.filtera etc.

test1.ss with "0.2"

test1.filterv with "crop=1312:1080:290:0,scale=576x474"

test1.filtera with "volume=1.7"

etc...

#!/bin/bash
SUM="$(ls test*.mov test*.mp4 | wc -l)"
echo inputs = $SUM
BOX=1
while [ "$((BOX*BOX))" -lt "$SUM" ]; do ((BOX=BOX+1)); done
echo boxsize = "${BOX}x${BOX}"
WID=640
HEI=360
POX=0
POY=0
INP=()
FCT=-1
FSC=""
FLV=""
FLA=""
AUD=""
BAK="bg"

IFS=$'\n'
for f in $(ls test*.mov test*.mp4); do
  echo $f
  SSI="${f%.*}.ss"
  echo $SSI
  if [[ -f "$SSI" ]]; then
    INP+=("-ss")
    INP+=($(<"${SSI}"))
  fi
  INP+=("-i")
  INP+=("$f")
  ((FCT+=1))

  if [ "$POX" -ge "$BOX" ]; then
    POX=0
    ((POY+=1))
  fi
  ((x=POX*WID))
  ((y=POY*HEI))
  ((POX+=1))

  SSI="${f%.*}.filterv"
  echo $SSI
  if [[ -f "$SSI" ]]; then
    FIL=($(<"${SSI}"))
  else
    FIL=""
  fi

  if [ "$FIL" != "" ]; then
    FSC+="[${FCT}:v:0]${FIL}"
    SC1="${FCT}sca"
    FSC+="[${SC1}]; "
  else
    SC1="${FCT}:v:0"
  fi

  FLV+="[${BAK}][${SC1}]overlay=x=${x}:y=${y}"
  if [ "$FCT" -eq 0 ]; then
    FLV+=":eof_action=endall"
  fi
  BAK="${FCT}v0"
  FLV+="[${BAK}]; "

  SSI="${f%.*}.filtera"
  echo $SSI
  if [[ -f "$SSI" ]]; then
    FIL=$(<"${SSI}")","
  else
    FIL=""
  fi

  FLA+="[${FCT}:a:0]${FIL}aresample=async=1:first_pts=0[${FCT}a0]; "
  AUD+="[${FCT}a0]"
done
((FCT+=1))
((POX=BOX*WID))
((POY=POY*HEI+HEI))
ffmpeg "${INP[@]}" -filter_complex "color=size=${POX}x${POY}[bg]; ${FSC} ${FLV} ${FLA} ${AUD} amix=inputs=${FCT}[a]" -map "[${BAK}]" -map [a] -c:v h264_nvenc -cq 20 -c:a aac -q:a 4 -y output.mkv

Add files one by one.

3
  • I created a series of videos, each video is a single singer plus the piano; in every case, the piano is clearly off. It's much harder to tell with a singer whether the audio matches the video or not, especially if it's only a quarter of a second or so. So either it's the piano video that's somehow off (although it looks fine when played by itself) or it's my command line that's wrong. Oct 29, 2020 at 0:35
  • Your command line is normal. I check your cmdline with films with shooting. Up to 16 movies synced well. Oct 29, 2020 at 4:51
  • Hi guys, have you considered possible different driver and frame-per-second encoding issues? @BettyCrokker You should add the encoding parameters of MOV and mp4 files... maybe you can rule out this hypothesis by first trying to convert them all with the same audio / video encoding and next merging them together...
    – Hastur
    Oct 31, 2020 at 10:56
0

I suggest you to use some program with a cosy GUI.

This is because you do not seem to be completely sure when your output starts being dissynchronous. A program with a graphical interface, such as audacity, should help you solve the issue effectively.

A note on your specific problem: have you considered possible different driver and frame-per-second encoding issues? Seldom you can rule out the problem if this is the cause, by first trying to convert them each with the same audio/video encoding (included the frame per second), check for sync and next merging them together...

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .