I have several video recordings from Full HD IPTV channels that I stored using VLC. These stream dumps are stored as MPEG-TS files and contain AVC-encoded video with MPEG audio.
I would like to extract specific clips from these recordings, based on frame-accurate cutpoints. However, I'm having a difficult time determining the exact, millisecond timestamps I need to pass to FFMPEG's -ss and -to parameters to hit the cutpoints I want precisely. Here is what I have tried so far:
Attempt 1
My initial try was to load the .ts files into Avidemux since it allows comfortable frame-by-frame seeking, and displays the timestamp of the current frame in the necessary precision. However, if I put the timestamp of a frame I seeked to (in format HH:MM:SS.mmm) into the FFMPEG parameters, the cut will actually be off by several frames, sometimes over a second. The drift is different for each file, and can be positive or negative.
Then I noticed that for most of these recordings, for the first frame Avidemux actually shows a timestamp that isn't zero (e.g. 00:00:00.280 or even 00:00:00.216). I assumed that VLC starts the recording immediately, but Avidemux ignores everything until the first I-frame. I still wonder at a timestamp like 00:00:00.216, though, because these videos are at 25 fps and 216 is not even a multiple of 40 ms.
Attempt 2
I tried a two-pass process, in which I would encode the video once with starting the encode slightly earlier than the cut I want, and finishing it slightly after the end I want. I would then use Avidemux to count the superfluous frames in the beginning and end of the video, and move the cut points by nFrames * 40 ms inwards. However, the results were still not precise, it would lead me to sometimes shorten the video by too few frames, sometimes too many.
Losing trust in Avidemux's timestamp accuracy, or at least concluding that it does some calculations differently than FFMPEG, I thought the safest way would be to use the same tool for determining the cut points that I also use to actually cut the video.
Attempt 3
I tried to extract single frames around the desired cutting points using FFMPEG, by using the same -ss and -to parameters I use to cut the video, but selecting only a range closely around the desired cutting points, and writing the frames to picture files. I would also use a filter to burn in each frame's exact timestamp, as determined by FFMPEG, directly into the picture. That way I could just read off the desired cutting points and use them for the actual encode. My command line would look something like this:
ffmpeg.exe -i input.ts -ss 00:00:29.000 -to 00:00:31.000 -vf drawtext=fontfile=roboto.ttf:fontsize=40:text='%{pts\:hms}':[email protected]:x=10:y=10 image%03d.png
I would then find the exact PNG I want the cut to be at, look at the timestamp, and if it said 00:00:30.160, that would be the timestamp I'd use for the -ss of the actual cut.
However, this still didn't work and my cutting points are still several frames earlier or later than the extracted PNGs. So changing FFMPEG's output from video to images seems to affect how timestamps are calculated, because they don't match up!
So far I have not found a way to avoid having to do a lengthy binary search process to manually approach the exact timestamps the cuts need to be at, which is especially cumbersome because I can only get the timestamps I need by doing "full-decode seeks" (i.e. using -ss after the input parameter, not before) and it can take a long time to seek to a position several hours into a recording.
How can I find the timestamp I need to get a frame-exact cut from FFMPEG in an MPEG Transport Stream, without having to decode, cut, and save the video segments multiple times in order to manually inch my way towards them?
Here is the MediaInfo on one of the recordings, in case it contains any clues about something being weird with the streams themselves:
General
ID : 1 (0x1)
Complete name : C:\Users\…\vlc-record-2021-03-09-00h23m11s.ts
Format : MPEG-TS
File size : 2.31 GiB
Overall bit rate mode : Variable
Video
ID : 256 (0x100)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Codec ID : 27
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 FPS
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan type, store method : Separated fields
Scan order : Top Field First
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Audio
ID : 257 (0x101)
Menu ID : 1 (0x1)
Format : MPEG Audio
Format version : Version 1
Format profile : Layer 2
Codec ID : 3
Bit rate mode : Constant
Bit rate : 128 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Compression mode : Lossy
Delay relative to video : -248 ms
Menu
ID : 4096 (0x1000)
Menu ID : 1 (0x1)
List : 256 (0x100) (AVC) / 257 (0x101) (MPEG Audio)
Service name : Service01
Service provider : FFmpeg
Service type : digital television
Additional info: The timestamp drift is different for each file, but constant within each file. That is, if I need to do several cuts in a video, and for one cut determine that the timestamp given in the pictures extracted with FFMPEG is always 160ms lower than what I need to use for -ss, I can use that same offset for the rest of the cuts in the same file, and it will be precise.
-c:v copy, just that the latter will be garbled in the beginning. A typical command I use to encode the snippets would look like this:ffmpeg.exe -i in.ts -c:v libx265 -crf 25 -vf yadif -c:a copy -ss 00:15:00.000 -to 00:45:00.000 out.mkv