I've been stuck with this problem for months. I have over 50 DV tapes (from and old Sony camcorder) to be converted to a more modern, usable format (most likely H264). I've started off with pulling the files to my PC (via firewire) using DVGRAB. There I had two options: pulling RAW data from the dv tape, resulting in a muxed file OR demuxing it and saving to a DVI file.

That's where the problems started. Saving it to a DVI file resulted in the audio being out of sync. I thought it's a problem with DVGRAB so I saved the RAW files (which are synced correctly) and wanted to process them with ffmpeg.

It turns out that no matter how I demux it the audio is always out of sync. BEFORE you say anything about the sampling frequency - the audio differences are of absolutely random length. An hour long tape can have between 0.1 and 4 seconds of audio lag at the end.

Here's an example file that I've split into separate audio and video files to check the differences.

# ffprobe -i ./video_conversion/13.dv 
ffprobe version 2.8.4 Copyright (c) 2007-2015 the FFmpeg developers
  built with gcc 5.3.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libdcadec --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-shared --enable-version3 --enable-x11grab
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
[dv @ 0x864f2a0] Detected timecode is invalid
[dv @ 0x864f2a0] Estimating duration from bitrate, this may be inaccurate
Input #0, dv, from './video_conversion/13.dv':
  Duration: 01:00:45.80, start: 0.000000, bitrate: 28800 kb/s
    Stream #0:0: Video: dvvideo, yuv420p, 720x576 [SAR 16:15 DAR 4:3], 28800 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
    Stream #0:1: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s

# ffprobe -i ./video_conversion/tmp/13.mp4
ffprobe version 2.8.4 Copyright (c) 2007-2015 the FFmpeg developers
  built with gcc 5.3.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libdcadec --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-shared --enable-version3 --enable-x11grab
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './video_conversion/tmp/13.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
  Duration: 01:00:45.80, start: 0.000000, bitrate: 5685 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 720x576 [SAR 16:15 DAR 4:3], 5683 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler

# ffprobe -i ./video_conversion/tmp/13.mp3
ffprobe version 2.8.4 Copyright (c) 2007-2015 the FFmpeg developers
  built with gcc 5.3.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libdcadec --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-shared --enable-version3 --enable-x11grab
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
[mp3 @ 0x954c2a0] Skipping 0 bytes of junk at 237.
Input #0, mp3, from './video_conversion/tmp/13.mp3':
  Metadata:
    encoder         : Lavf56.40.101
  Duration: 01:00:44.35, start: 0.023021, bitrate: 128 kb/s
    Stream #0:0: Audio: mp3, 48000 Hz, stereo, s16p, 128 kb/s
    Metadata:
      encoder         : Lavc56.60

This particular one differs by 1.448 seconds. As I said the differences vary greatly.

As for the solution. I could just stretch the audio and combine it with the video (I've tested that), but I can't be certain if the audio will be in sync somewhere in the middle of the recording.

I think I've pinpointed the source of this behaviour. Whenever I turn the camera on or off (as to start and stop recording) the video starts just a tiny bit faster then the audio. So the more "fragments" are on the tape, the more these differences add up.

How can I fix this? Is there a way to demux the audio and video with timestamps, so that after conversion they will add up correctly? Or is there anyway to fill these gaps in audio, so that both streams are the same size to begin with?

  • What's the command to demux the raw files? – Gyan Feb 19 '16 at 15:30
  • The raw .dv file is multiplexed by it's nature. FFMPEG is demuxing it by default when converting it to any container. – Wojciech Feb 19 '16 at 15:56
  • Ok, rather , what's your conversion command? I forgot you're transcoding. – Gyan Feb 19 '16 at 16:01
  • I've tried a dozen combinations. Nothing special though: avconv -f dv -i ./46raw.dv -f mp4 -acodec libvo_aacenc -b:a 256k -vcodec libx264 -b:v 4000k -y ./46raw.aac.mp4 – Wojciech Feb 19 '16 at 16:05
  • 1
    avconv != ffmpeg. If it's just an offset issue, you can use -af adelay=1000|1000 where 1000 is delay in ms. – Gyan Feb 19 '16 at 16:47

Here are three wildcard attempts at solving this issue:

Method 1a Use system time as timestamps

ffmpeg -use_wallclock_as_timestamps 1 -i input.dv \
       -c:v libx264 -b:v 4000k -c:a aac -b:a 128k -fflags +genpts method1.ts

Method 1b Use resampler with flag set to inject silence when input audio timestamps have gaps

ffmpeg -i input.dv -c:v libx264 -b:v 4000k \
       -af "aresample=async=1:first_pts=0" -c:a aac -b:a 128k -fflags +genpts method1.ts

Method 2 Merge with dummy audio

ffmpeg -i input.dv -f lavfi -i "aevalsrc=0:c=2:s=48000" \
       -filter_complex "[0:a][1:a]amerge[a]" -map 0:v -map "[a]" -c:v libx264 -b:v 4000k -c:a aac -b:a 128k -ac 2 -shortest method2.ts

Method 3 Combination of the above

ffmpeg -use_wallclock_as_timestamps 1 -i input.dv -f lavfi -use_wallclock_as_timestamps 1 -i "aevalsrc=0:c=2:s=48000" \
       -filter_complex "[0:a][1:a]amerge[a]" -map 0:v -map "[a]"  -c:v libx264 -b:v 4000k -c:a aac -b:a 128k -ac 2 -shortest method3.ts

You can test each of them for a short duration by inserting -t N e.g. -t 20 for a 20 second test.

If any of them work, we can then proceed to wrapping the output as MP4.

  • Option 2: Simple filtergraph 'amerge' was expected to have exactly 1 input and 1 output. However, it had >1 input(s) and 1 output(s). Please adjust, or use a complex filtergraph (-filter_complex) instead. Option 1. Gives a lot of errors: [aac @ 0x9160040] Queue input is backward in time [mp4 @ 0x915e1c0] Non-monotonous DTS in output stream 0:1; previous: 70000289337917, current: 70000289337250; changing to 70000289337918. This may result in incorrect timestamps in the output file. And stops after about 90MB of an unplayable output file. – Wojciech Mar 19 '16 at 16:46
  • Now, try the 3 commands. Also, test playback with ffplay i.e. ffplay method1.ts – Gyan Mar 19 '16 at 17:01
  • Options 1a and 3 produce 90MB and 20MB files respectively with little to no video. Options 1b and 2 produce the whole video, but do not help with regards to the delay :( – Wojciech Mar 22 '16 at 9:19
  • Doing this blindly is futile. Can you send a bit of the raw file, say, 20 seconds, or enough to observe loss of sync with your original command? – Gyan Mar 22 '16 at 11:10
up vote 0 down vote accepted

I've finally solved the issue - it's an overkill, but it works.

I've realized that if I copy the .dv to any other container, the audio and video is obviously out of sync. Then I wanted to cut that file to a 1 minute segment starting at the 51st minute (-ss 51:00 -t 60), it was obviously still out of sync.

However, when I used the same cut (-ss 51:00 -t 60) on the original .dv it was in sync! So what I ended up doing is I wrote a script that cut the .dv file into 1 second segment every second and saved that into separate files (yes over 3600 files per .dv). No encoding, just stream copy to a new container (avi). Then I used -f concat, to put the tiny files into one avi file, that was in sync now! Any gaps are inaudible! All that was left was encoding H264 and AAC into MP4.

I ran the script on my home server that was grinding the 50 .dv files for a couple of days, but now it's done!

THANK YOU ALL FOR YOU HELP! I've learned a lot about ffmpeg and a/v in general.

  • This is a good workaround but doesn't actually solve the sync issue since each DV to AVI wrapping is subject to the same error that you had when copying the whole .dv to .avi. What this workaround does is prevent the tiny discrepancies, if any, in each 1 second segment from cascading and accumulating since each second is a separate file. You'll still have a few of the AVIs where there's noticeable async, but those don't affect the remaining AVI segments. If you can, I'm still open to working on a short segment of the raw .dv to see if this can be accurately solved, and in one step. – Gyan May 11 '16 at 6:33
  • I am aware that the gaps are still there, but stretching the audio would be pretty munch the same kind of solution. This is good enough for me. About the sample - there is little sense is sending a small sample, because the error is at most 3s in 1h and that's less then 0.1%. I can't send you a whole file since these are my sister's family videos (she wouldn't approve). If I manage to get a blank tape I could make a fresh sample for you to work with (filming a movie on a TV would give you good sync reference). – Wojciech May 11 '16 at 8:38
  • My desired solution won't involve stretching audio. Raw DV doesn't have timestamps, but the audio is interleaved in sync, so my tinkering would be aimed at preserving that chronological relation.If you ever get the time, I'm ready to work with a sample. – Gyan May 11 '16 at 8:53

I have a similar setup with the same audio out of sync problem. I also managed to reproduce a clip with an out of sync audio. If anyone want's samples, please ask.

I may have found a solution for this problem. Kino is very old and no longer maintained software that has the capability to load a .dv from dvgrab (raw) and export again as a .dv or dv1/avi (or dv2/avi) file with "re-sampling" of the audio. Well, the output is a corrected file which will be well synchronized before and after an 'ffmpeg' transcode.

There are some downsides. Kino may stop work or even won't work at all as this is old. I just installed it from 'aur' (Arch linux) and I was able to use it straightforward. There is no command line interface. I couldn't find a way to automate this.

EDIT:

There might be another solution. I think the problem is that the stream's start and stop bits somehow get broken, and the timecode gets worse. I have some clips appearing to have a date from the year '2068'. Anyway, you can use the 'dvgrab' again for splitting the clips every time it thinks there is a new record stream:

dvgrab -I input -size 0 -a -format=raw -showstatus -srt -t output

'-a' does the auto split, '-srt' and '-t' helps tracking the files (builds a srt with the dates and appends the date to the files, respectively). This will create a new file for each new stream. As the beginning of each stream is in sync, you may 'ffmpeg' them individually. It appears each file contains the timecode of the original "session" (as dvgrab calls it) so, if your concat all the files directly with ffmpeg you still get the same out of sync.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.