Recently I did a re-encode using ffmpeg and encountered an inaudible but waveform visible audio desynchronization in the process.
Source information of MKV file and audio stream:
Mediainfo shows a 6ms audio delay relative to the video:
Audio ID : 2 Format : E-AC-3 Format/Info : Enhanced Audio Coding 3 Format settings, Endianness : Big Codec ID : A_EAC3 Duration : 1 h 0 min Bit rate mode : Constant Bit rate : 640 kb/s Channel(s) : 6 channels Channel positions : Front: L C R, Side: L R, LFE Sampling rate : 48.0 kHz Frame rate : 187.500 FPS (256 SPF) Compression mode : Lossy Delay relative to video : 6 ms Stream size : 276 MiB (7%) Language : English Service kind : Complete Main Default : Yes Forced : No
This can also be seen by looking at the pts and at the stream start time:
ffprobe -of compact -select_streams a:0 -show_frames -show_entries frame=pkt_pts,pkt_pts_time,pkt_dts,pkt_dts_time,best_effort_timestamp_time,pkt_duration,nb_samples %sourcemkv% frame|pkt_pts=6|pkt_pts_time=0.006000|pkt_dts=6|pkt_dts_time=0.006000|best_effort_timestamp_time=0.006000|pkt_duration=32|nb_samples=1536 frame|pkt_pts=38|pkt_pts_time=0.038000|pkt_dts=38|pkt_dts_time=0.038000|best_effort_timestamp_time=0.038000|pkt_duration=32|nb_samples=1536 frame|pkt_pts=70|pkt_pts_time=0.070000|pkt_dts=70|pkt_dts_time=0.070000|best_effort_timestamp_time=0.070000|pkt_duration=32|nb_samples=1536
Also, the delay is detected via stream
ffprobe -show_entries stream=codec_type,duration,start_time -of compact %sourcemkv% stream|codec_type=video|start_time=0.000000|duration=N/A stream|codec_type=audio|start_time=0.006000|duration=N/A stream|codec_type=subtitle|start_time=0.000000|duration=3617.638000
I opted for transcoding EAC3 to AAC and downmixing to stereo, using only these flags (no timestamp manipulation):
-acodec aac -b:a 160k -ac 2
One way I obtained the delayed result is this command to re-encode:
ffmpeg -i %sourcemkv% -vcodec libx264 -profile:v high -crf 23 -tune film -level 4.1 -acodec aac -b:a 160k -ac 2 re-encoded-with-aac.mkv
Detecting the delay
I detected a delay by inspecting the waveform corresponding to a video frame. The delay itself is not quite audible, as I found out the encoded version to be 15.333 ms late compared to the original, but according to inspection via the following AviSynth+ script in AvsPmod it is present:
FFmpegSource2("re-encoded-with-aac.mkv", vtrack=-1, atrack=-1, cache=True, cachefile="", fpsnum=-1, fpsden=1, threads=-1, timecodes="", seekmode=1, overwrite=False, width=-1, height=-1, resizer="BICUBIC", colorspace="", rffmode=0, adjustdelay=-1, utf8=False, varprefix="") ConvertToMono() # Test if DelayAudio will sync it! #AmplifydB(50.0) #DelayAudio(-0.015333) waveform( window=1, height=0.6)
I applied the same script to the original and compared the resulting waveform graphs for certain video frames: source mkv graph for frame 139 vs AAC re-encoded video's waveform view of frame 139
As can be seen, unless a
DelayAudio(-0.015333) is applied to the re-encoded AAC audio stream - they are not synchronized.
Thankfully, this delay is constant throughout the video!
Other information of re-encode
The remuxed audio stream has
start_time=0.000000 and first audio frame's pts is
0.000000. It is a bit strange that although the reported known delay of the original audio relative to the video is of 6ms, the resulting delay between original and re-encoded video with aac stream is of
15.333ms. I don't know from where this may come; maybe it has to do with the length of the output's audio frames, each having a duration of
- Is this a bug with ffmpeg when transcoding from EAC3 to AAC?
- Did I misuse ffmpeg or expect too much from it to correctly re-encode this?
- Is there a need to specify some extra ffmpeg parameters in order to have things synchronized?
- What would be the best solution to handle situations like this other than visual inspection of frame's waveform for determining the rather arbitrary delays which may occur?
- Does the 6ms delay of the original audio stream cause all these issues with the re-encode?
I also tried the following experiments:
%sourcemkv%in gMKVExtractGUI. The 6ms of delay in the EAC3 audio stream is detected. Extract this audio stream.
- Remux the re-encoded video stream in mkvtoolnix-gui with the original EAC3 audio, and specify the 6ms delay explicitely in the Delay field.
Result: AUDIO IS SYNCHRONIZED with the original, judging by the inspection method described above.
- convert the original audio to 2 channel AAC via FFmpeg
- Remux the resulting AAC audio stream with the video in mkvtoolnix-gui, also specifying a 6ms delay for it - just as the original has.
Result: Audio is not synchronized between the original and the result