8

Problem

I have a MKV file which has 1 video and 1 subtitle stream, of which I want to hardcode the subtitle stream and output a MP4 file.

Example file, images and results can be found here.

mediainfo

General
Unique ID                                : 169861028457722525578038513971806461765 (0x7FCA0B4ABC222F3850F6CF659051DF45)
Format                                   : Matroska
Format version                           : Version 4
File size                                : 1.97 MiB
Duration                                 : 8 s 426 ms
Overall bit rate                         : 1 959 kb/s
Encoded date                             : UTC 2022-12-11 19:29:00
Writing application                      : mkvmerge v65.0.0 ('Too Much') 64-bit
Writing library                          : libebml v1.4.2 + libmatroska v1.6.4
Attachments                              : GENTIUMBASIC.TTF

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High 10@L5
Format settings                          : CABAC / 8 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 8 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 8 s 426 ms
Bit rate                                 : 1 694 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (23976/1000) FPS
Original frame rate                      : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.034
Stream size                              : 1.70 MiB (86%)
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Matrix coefficients                      : BT.709

Text
ID                                       : 2
Format                                   : ASS
Codec ID                                 : S_TEXT/ASS
Codec ID/Info                            : Advanced Sub Station Alpha
Duration                                 : 1 s 540 ms
Bit rate                                 : 4 815 b/s
Frame rate                               : 3.247 FPS
Count of elements                        : 5
Compression mode                         : Lossless
Stream size                              : 927 Bytes (0%)
Default                                  : Yes
Forced                                   : No

Normally I do the following when hardcoding the subtitles:

ffmpeg -i "original.mkv" -filter_complex "subtitles='original.mkv':si=0" -pix_fmt yuv420p -c:v libx264 -crf 18 "output.mp4"

But this time it gave me an undesired result:

undesired-result

What I would like to see, is the desired result, similar from what I'm seeing in Aegisub:

desired-result

At this point I'm guessing it possibly has something to do with different colourspaces or pixel formats and/or not using it correctly with ffmpeg. As I'm not really familiar with this, I'm asking for help.

Additional notes

I've hardcoded it both with FFmpeg as well as Handbrake, both giving different results while hardcoding with similar settings. As Handbrake uses FFmpeg under the hood, I did not expect a different result regarding the subtitles, but apparently Handbrake gave the desired result.

Handbrake

All filters off.

Log
HandBrake 1.5.1 (2022011000)
OS: Microsoft Windows NT 10.0.19044.0
CPU: AMD Ryzen 9 3900X 12-Core Processor            
Ram: 32693 MB, 
GPU Information:
  NVIDIA GeForce RTX 3060 Ti - 31.0.15.2225
Screen: 1920x1080
Temp Dir: C:\Users\AppData\Local\Temp\
Install Dir: C:\Program Files\HandBrake\
Data Dir: C:\Users\AppData\Roaming\HandBrake

-------------------------------------------

 # Starting Encode ...

[20:39:10] base preset: Fast 1080p30 (Modified)
[20:39:10] Remote Process started with Process ID: 31324 using port: 8037. Max Allowed Instances: 1
[20:39:10] Worker: Starting HandBrake Engine ...
[20:39:10] Worker: Starting Web Server on port 8037 ...
[20:39:11] Worker: Disconnected worker monitoring enabled!
[20:39:11] Compile-time hardening features are enabled
[20:39:11] hb_init: starting libhb thread
[20:39:11] Starting work at: Sun Dec 11 20:39:11 2022
[20:39:11] 1 job(s) to process
[20:39:11] json job:
{
  "Audio": {
    "AudioList": [],
    "CopyMask": [
      "copy:aac",
      "copy:ac3",
      "copy:dtshd",
      "copy:dts",
      "copy:eac3",
      "copy:flac",
      "copy:mp3",
      "copy:truehd",
      "copy:mp2"
    ],
    "FallbackEncoder": "ac3"
  },
  "Destination": {
    "ChapterList": [],
    "ChapterMarkers": false,
    "AlignAVStart": false,
    "File": "C:\\Users\\Desktop\\handbrake-result.mp4",
    "Mp4Options": {
      "IpodAtom": false,
      "Mp4Optimize": false
    },
    "Mux": "av_mp4"
  },
  "Filters": {
    "FilterList": [
      {
        "ID": 13,
        "Settings": {
          "crop-bottom": "0",
          "crop-left": "0",
          "crop-right": "0",
          "crop-top": "0",
          "height": "1080",
          "width": "1920"
        }
      },
      {
        "ID": 6,
        "Settings": {
          "mode": "1"
        }
      }
    ]
  },
  "PAR": {
    "Num": 1,
    "Den": 1
  },
  "Metadata": {},
  "SequenceID": 0,
  "Source": {
    "Angle": 1,
    "Range": {
      "Type": "chapter",
      "Start": 1,
      "End": 1
    },
    "Title": 1,
    "Path": "C:\\Users\\Desktop\\original.mkv"
  },
  "Subtitle": {
    "Search": {
      "Burn": false,
      "Default": false,
      "Enable": false,
      "Forced": false
    },
    "SubtitleList": [
      {
        "Burn": true,
        "Default": false,
        "Forced": false,
        "ID": 1,
        "Offset": 0,
        "Track": 0
      }
    ]
  },
  "Video": {
    "Encoder": "x264",
    "Level": "5.0",
    "TwoPass": false,
    "Turbo": false,
    "ColorMatrixCode": 0,
    "Options": "",
    "Preset": "fast",
    "Profile": "high",
    "Quality": 22,
    "QSV": {
      "Decode": false
    }
  }
}
[20:39:11] CPU:
[20:39:11]  - logical processor count: 24
[20:39:11] Intel Quick Sync Video support: no
[20:39:11] hb_scan: path=C:\Users\Desktop\original.mkv, title_index=1
udfread ERROR: ECMA 167 Volume Recognition failed
src/libbluray/disc/disc.c:333: failed opening UDF image C:\Users\Desktop\original.mkv
src/libbluray/disc/disc.c:437: error opening file BDMV\index.bdmv
src/libbluray/disc/disc.c:437: error opening file BDMV\BACKUP\index.bdmv
src/libbluray/bluray.c:2646: nav_get_title_list(C:\Users\Desktop\original.mkv\) failed
[20:39:11] bd: not a bd - trying as a stream/file instead
libdvdread: Encrypted DVD support unavailable.
libdvdread: Canlibdvdread: DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
't open file VIDEO_TS.IFO.
libdvdnav: vm: vm: failed to read VIDEO_TS.IFO
[20:39:11] dvd: not a dvd - trying as a stream/file instead
Input #0, matroska,webm, from 'C:\Users\Desktop\original.mkv':
  Metadata:
    encoder         : libebml v1.4.2 + libmatroska v1.6.4
    creation_time   : 2022-12-11T19:29:00.000000Z
  Duration: 00:00:08.43, start: 0.000000, bitrate: 1958 kb/s
  Stream #0:0: Video: h264 (High 10), yuv420p10le(tv, bt709/unknown/unknown, progressive), 1920x1080, SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
    Metadata:
      BPS             : 1693698
      DURATION        : 00:00:08.426000000
      NUMBER_OF_FRAMES: 202
      NUMBER_OF_BYTES : 1783888
      _STATISTICS_WRITING_APP: mkvmerge v65.0.0 ('Too Much') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2022-12-11 19:29:00
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:1: Subtitle: ass (default)
    Metadata:
      BPS             : 4815
      DURATION        : 00:00:01.540000000
      NUMBER_OF_FRAMES: 5
      NUMBER_OF_BYTES : 927
      _STATISTICS_WRITING_APP: mkvmerge v65.0.0 ('Too Much') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2022-12-11 19:29:00
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:2: Attachment: none
    Metadata:
      filename        : GENTIUMBASIC.TTF
      mimetype        : font/ttf
[20:39:11] scan: decoding previews for title 1
[20:39:11] scan: 10 previews, 1920x1080, 23.976 fps, autocrop = 0/0/0/0, aspect 16:9, PAR 1:1, color profile: 1-1-1, chroma location: left
[20:39:11] libhb: scan thread found 1 valid title(s)
[20:39:11] Starting Task: Encoding Pass
[20:39:11] Skipping crop/scale filter
[20:39:11] [ass] libass API version: 0x1502000
[20:39:11] [ass] libass source: tarball: 0.15.2
[20:39:11] [ass] Shaper: FriBidi 1.0.11 (SIMPLE) HarfBuzz-ng 3.1.2 (COMPLEX)
[20:39:11] [ass] Using font provider directwrite (with GDI)
[20:39:11] job configuration:
[20:39:11]  * source
[20:39:11]    + C:\Users\Desktop\original.mkv
[20:39:11]    + title 1, chapter(s) 1 to 1
[20:39:11]    + container: matroska,webm
[20:39:11]    + data rate: 1958 kbps
[20:39:11]  * destination
[20:39:11]    + C:\Users\Desktop\handbrake-result.mp4
[20:39:11]    + container: MPEG-4 (libavformat)
[20:39:11]  * video track
[20:39:11]    + decoder: h264 10-bit (yuv420p10le)
[20:39:11]    + filters
[20:39:11]      + Framerate Shaper (mode=1)
[20:39:11]        + frame rate: 23.976 fps -> constant 23.976 fps
[20:39:11]      + Subtitle renderer ()
[20:39:11]      + Format (format=yuv420p)
[20:39:11]    + Output geometry
[20:39:11]      + storage dimensions: 1920 x 1080
[20:39:11]      + pixel aspect ratio: 1 : 1
[20:39:11]      + display dimensions: 1920 x 1080
[20:39:11]    + encoder: H.264 (libx264)
[20:39:11]      + preset:  fast
[20:39:11]      + profile: high
[20:39:11]      + level:   5.0
[20:39:11]      + quality: 22.00 (RF)
[20:39:11]      + color profile: 1-1-1
[20:39:11]      + chroma location: left
[20:39:11]  * subtitle track 1, Unknown [SSA] (track 0, id 0x1, Text) -> Render/Burn-in
[20:39:11] sync: expecting 202 video frames
[20:39:11] encx264: min-keyint: 24, keyint: 240
[20:39:11] encx264: encoding at constant RF 22.000000
[20:39:11] encx264: unparsed options: level=5.0:ref=2:weightp=1:subme=6:vbv-bufsize=168750:vbv-maxrate=168750:rc-lookahead=30
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x264 [info]: profile High, level 5.0, 4:2:0, 8-bit
[20:39:11] sync: first pts video is 0
[20:39:11] sync: Chapter 1 at frame 1 time 0
[20:39:11] sync: first pts subtitle 0x1 is 110700
[20:39:12] [ass] fontselect: (Arial, 400, 0) -> ArialMT, 0, ArialMT
[20:39:12] [ass] fontselect: (Gentium Basic, 400, 0) -> GentiumBasic, 0, GentiumBasic
[20:39:12] reader: done. 1 scr changes
[20:39:13] work: average encoding speed for job is 0.000000 fps
[20:39:13] vfr: 202 frames output, 0 dropped and 0 duped for CFR/PFR
[20:39:13] vfr: lost time: 0 (0 frames)
[20:39:13] vfr: gained time: 0 (0 frames) (0 not accounted for)
[20:39:13] h264-decoder done: 202 frames, 0 decoder errors
[20:39:13] sync: got 202 frames, 202 expected
[20:39:13] sync: framerate min 23.976 fps, max 23.976 fps, avg 23.976 fps
x264 [info]: frame I:6     Avg QP:13.01  size:143206
x264 [info]: frame P:51    Avg QP:15.88  size:  2936
x264 [info]: frame B:145   Avg QP:18.37  size:   443
x264 [info]: consecutive B-frames:  3.5%  1.0%  4.5% 91.1%
x264 [info]: mb I  I16..4: 18.3% 66.7% 15.0%
x264 [info]: mb P  I16..4:  0.2%  0.7%  0.1%  P16..4:  3.2%  1.0%  0.6%  0.0%  0.0%    skip:94.0%
x264 [info]: mb B  I16..4:  0.0%  0.4%  0.0%  B16..8:  0.6%  0.1%  0.0%  direct: 0.6%  skip:98.3%  L0:51.4% L1:42.6% BI: 6.0%
x264 [info]: 8x8 transform intra:68.7% inter:78.3%
x264 [info]: coded y,uvDC,uvAC intra: 64.1% 64.8% 55.3% inter: 0.5% 0.8% 0.0%
x264 [info]: i16 v,h,dc,p: 50% 32%  8% 10%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 35% 23%  3%  1%  1%  2%  2%  4%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 34% 28%  5%  6%  5%  5%  6%  5%  5%
x264 [info]: i8c dc,h,v,p: 55% 22% 20%  4%
x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x264 [info]: ref P L0: 81.4% 18.6%
x264 [info]: ref B L0: 84.8% 15.2%
x264 [info]: ref B L1: 96.0%  4.0%
x264 [info]: kb/s:1019.07
[20:39:13] mux: track 0, 202 frames, 1072961 bytes, 1018.83 kbps, fifo 256
[20:39:13] Finished work at: Sun Dec 11 20:39:13 2022
[20:39:13] libhb: work result = 0

 # Job Completed!
Result mediainfo
General
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (mp42/iso2/avc1/mp41)
File size                                : 1.03 MiB
Duration                                 : 8 s 426 ms
Overall bit rate                         : 1 023 kb/s
Encoded date                             : UTC 2022-12-11 19:39:11
Tagged date                              : UTC 2022-12-11 19:39:11
Writing application                      : HandBrake 1.5.1 2022011000

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 8 s 426 ms
Bit rate                                 : 1 019 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 23.976 (24000/1001) FPS
Minimum frame rate                       : 23.974 FPS
Maximum frame rate                       : 23.981 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.020
Stream size                              : 1.02 MiB (100%)
Writing library                          : x264 core 164 r3065 ae03d92
Encoding settings                        : cabac=1 / ref=2 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=6 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=34 / lookahead_threads=5 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=240 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=30 / rc=crf / mbtree=1 / crf=22.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=168750 / vbv_bufsize=168750 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00
Encoded date                             : UTC 2022-12-11 19:39:11
Tagged date                              : UTC 2022-12-11 19:39:11
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
mdhd_Duration                            : 8425
Codec configuration box                  : avcC
Visual

handbrake-result

FFmpeg

Log
ffmpeg version 5.0.1-essentials_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 11.2.0 (Rev7, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
[matroska,webm @ 00000203f1dbb980] Could not find codec parameters for stream 2 (Attachment: none): unknown codec
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, matroska,webm, from 'original.mkv':
  Metadata:
    encoder         : libebml v1.4.2 + libmatroska v1.6.4
    creation_time   : 2022-12-11T19:29:00.000000Z
  Duration: 00:00:08.43, start: 0.000000, bitrate: 1958 kb/s
  Stream #0:0: Video: h264 (High 10), yuv420p10le(tv, bt709/unknown/unknown, progressive), 1920x1080, SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 1k tbn (default)
    Metadata:
      BPS             : 1693698
      DURATION        : 00:00:08.426000000
      NUMBER_OF_FRAMES: 202
      NUMBER_OF_BYTES : 1783888
      _STATISTICS_WRITING_APP: mkvmerge v65.0.0 ('Too Much') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2022-12-11 19:29:00
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:1: Subtitle: ass (default)
    Metadata:
      BPS             : 4815
      DURATION        : 00:00:01.540000000
      NUMBER_OF_FRAMES: 5
      NUMBER_OF_BYTES : 927
      _STATISTICS_WRITING_APP: mkvmerge v65.0.0 ('Too Much') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2022-12-11 19:29:00
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:2: Attachment: none
    Metadata:
      filename        : GENTIUMBASIC.TTF
      mimetype        : font/ttf
[Parsed_subtitles_0 @ 00000203f1e54b00] libass API version: 0x1502002
[Parsed_subtitles_0 @ 00000203f1e54b00] libass source: commit: 0.15.2-69-gf664ced049394e2a5d4300ba526e206df73ec729
[Parsed_subtitles_0 @ 00000203f1e54b00] Shaper: FriBidi 1.0.11 (SIMPLE) HarfBuzz-ng 4.2.0 (COMPLEX)
[matroska,webm @ 00000203f2510c80] Could not find codec parameters for stream 2 (Attachment: none): unknown codec
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[Parsed_subtitles_0 @ 00000203f1e54b00] Using font provider directwrite (with GDI)
Stream mapping:
  Stream #0:0 (h264) -> subtitles:default
  subtitles:default -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
[Parsed_subtitles_0 @ 00000203f2088b80] libass API version: 0x1502002
[Parsed_subtitles_0 @ 00000203f2088b80] libass source: commit: 0.15.2-69-gf664ced049394e2a5d4300ba526e206df73ec729
[Parsed_subtitles_0 @ 00000203f2088b80] Shaper: FriBidi 1.0.11 (SIMPLE) HarfBuzz-ng 4.2.0 (COMPLEX)
[matroska,webm @ 00000203f1fdeac0] Could not find codec parameters for stream 2 (Attachment: none): unknown codec
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[Parsed_subtitles_0 @ 00000203f2088b80] Using font provider directwrite (with GDI)
[libx264 @ 00000203f2973040] using SAR=1/1
[libx264 @ 00000203f2973040] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 00000203f2973040] profile High, level 4.0, 4:2:0, 8-bit
[libx264 @ 00000203f2973040] 264 - core 164 r3094 bfc87b7 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=34 lookahead_threads=5 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=23 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=18.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'ffmpeg-result.mp4':
  Metadata:
    encoder         : Lavf59.16.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 24k tbn
    Metadata:
      encoder         : Lavc59.18.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[Parsed_subtitles_0 @ 00000203f2088b80] fontselect: (Arial, 400, 0) -> ArialMT, 0, ArialMT
[Parsed_subtitles_0 @ 00000203f2088b80] fontselect: (Gentium Basic, 400, 0) -> GentiumBasic, 0, GentiumBasic
frame=  202 fps=158 q=-1.0 Lsize=    1748kB time=00:00:08.30 bitrate=1724.9kbits/s speed=6.47x
video:1744kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.185189%
[libx264 @ 00000203f2973040] frame I:6     Avg QP: 8.48  size:245788
[libx264 @ 00000203f2973040] frame P:51    Avg QP:12.66  size:  4628
[libx264 @ 00000203f2973040] frame B:145   Avg QP:17.59  size:   516
[libx264 @ 00000203f2973040] consecutive B-frames:  3.5%  1.0%  4.5% 91.1%
[libx264 @ 00000203f2973040] mb I  I16..4: 25.7% 58.7% 15.7%
[libx264 @ 00000203f2973040] mb P  I16..4:  0.1%  0.9%  0.2%  P16..4:  3.0%  1.2%  0.8%  0.0%  0.0%    skip:93.9%
[libx264 @ 00000203f2973040] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  2.6%  0.2%  0.0%  direct: 0.0%  skip:97.2%  L0:42.9% L1:54.4% BI: 2.7%
[libx264 @ 00000203f2973040] 8x8 transform intra:60.1% inter:73.3%
[libx264 @ 00000203f2973040] coded y,uvDC,uvAC intra: 78.9% 79.0% 76.4% inter: 0.6% 0.6% 0.1%
[libx264 @ 00000203f2973040] i16 v,h,dc,p: 45% 29% 18%  8%
[libx264 @ 00000203f2973040] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 35% 21%  3%  2%  1%  2%  2%  5%
[libx264 @ 00000203f2973040] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 32% 27%  6%  6%  6%  5%  6%  6%  6%
[libx264 @ 00000203f2973040] i8c dc,h,v,p: 50% 24% 22%  4%
[libx264 @ 00000203f2973040] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 00000203f2973040] ref P L0: 80.0%  3.9% 11.0%  5.1%
[libx264 @ 00000203f2973040] ref B L0: 83.7% 14.4%  1.9%
[libx264 @ 00000203f2973040] ref B L1: 92.4%  7.6%
[libx264 @ 00000203f2973040] kb/s:1695.50
Result mediainfo
General
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 1.71 MiB
Duration                                 : 8 s 426 ms
Overall bit rate                         : 1 699 kb/s
Writing application                      : Lavf59.16.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 8 s 426 ms
Bit rate                                 : 1 696 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.034
Stream size                              : 1.70 MiB (100%)
Writing library                          : x264 core 164 r3094 bfc87b7
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=34 / lookahead_threads=5 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=18.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Color range                              : Limited
Matrix coefficients                      : BT.709
Codec configuration box                  : avcC
Visual

ffmpeg-result

Subtitle

The embedded ASS subtitle contains the entry YCbCr Matrix: TV.709, so I reckon that corresponds to the video's BT.709 color space.


I've tried different values for pix_fmt, or adding -filter:s with colorspaces filters, but no luck so far.

As there seems to be no seemingly significant differences in mediainfo, I'm a bit at a loss on what's causing the incorrect colouring. Any bit of help is appreciated.

Tested FFmpeg versions
5.0.1 on Windows 10
5.1.2 on Ubuntu 22.10

2 Answers 2

8

For fixing the issue we may convert the video to BT.601 color standard, add the subtitle and convert the result to BT.709 color standard:

ffmpeg -y -i "original.mkv" -filter_complex "scale=in_color_matrix=bt709:out_color_matrix=bt601,subtitles='original.mkv':si=0,scale=in_color_matrix=bt601:out_color_matrix=bt709" -pix_fmt yuv420p -c:v libx264 -crf 18 "output.mp4"


The issue may be related to the fact that the original video is encoded with "High 10@L5" profile, with bit depth of 10 bits.

I don't know if subtitles has a defined color standard (as BT.709 limited range), but it seems like FFmpeg uses BT.601 limited range color standard for the subtitles.

It also seems like subtitles filter has no option for selecting BT.709 color standard for the drawn subtitles.

My suggested solution is:

  • Convert each video frame from BT.709 to BT.601 color standard:

     scale=in_color_matrix=bt709:out_color_matrix=bt601
    
  • Add (draw) the subtitles over the "BT.601 colored" frame:

     subtitles='original.mkv'
    
  • Convert the color standard back from BT.601 to BT.709 standard before re-encoding:

     scale=in_color_matrix=bt601:out_color_matrix=bt709
    

Please note that we are losing some color information when converting from 10 bits depth to 8 bits depth, but I couldn't find a solution for that.

Sample output:
enter image description here

9
  • Thank you for your answer! I had no idea scale could do such a thing. After bit of playing around with it, I've noticed 2 things: the initial scaling has quite a performance impact (x12 rtime (-benchmark) compared to my initial command); Also still noticed very slight color box. Then I saw in the ffmpeg log the video stream stating yuv420p10le, so I used that and looks almost identical to the handbrake result
    – YTZ
    Commented Dec 11, 2022 at 23:38
  • 1
    Do you also have a reference for your statement ...it seems like FFmpeg uses BT.601 limited range color standard for the subtitles. ? As I could not find this myself, I would very much appreciate the source of this (or derivation of it).
    – YTZ
    Commented Dec 11, 2022 at 23:42
  • I wrote "it seems like..." because I am not really sure about it. I tried different color standards using the scale filter, and saw that with "BT.601 limited range" the colors are visually match. I concluded that the subtitles are treated as BT.601. I may be wrong... Maybe the ASS codec defines "BT.601", maybe it allows defining different color standard (or maybe the definition is using the same standard as the video, and there is a bug in FFmpeg). We may have to check the ASS codec definitions.
    – Rotem
    Commented Dec 12, 2022 at 6:39
  • From performance perspective, it might be better to restructure the filter to convert subtitles from BT.601 to BT.709 before burn-in - it'll be much more performant (subtitles have less pixels) and avoids loss of color due to conversion to a smaller color space in video picture.
    – Mavrik
    Commented Dec 12, 2022 at 19:41
  • @Mavrik converting the subtitles to BT.709 is better for sure. Do you know how to convert the color space (color standard) of the subtitles? I admit that my solution is inefficient, and there must be a better solution.
    – Rotem
    Commented Dec 12, 2022 at 20:26
0

After @Rotem' solution I've started thinking of another solution, yielding the same result without the need for scaling with color matrix.

However, this requires additional steps of modifying the subtitle files with Aegisub and remuxing/replacing the new subtitle file in the existing MKV file.


Aegisub

I've denoted that the subtitle file had a YCbCr Matrix: TV.709 entry set. While this matches the video's color space, I've tried changing it anyway.

  1. In File > Properties, change YCbCr Matrix from TV.709 to TV.601.
  2. In Subtitle > Resample resolution
    • Source Resolution use YCbCr Matrix: TV.709
    • Destination Resolution use YCbCr Matrix: TV.601
  3. Click on Ok and save the new subtitle file.

MKVToolnix

Remux the new subtitle file into the existing MKV file.

This will look kind of strange when seeing it in VLC/MPC-HC initially (screenshot with windows snipping tool):

vlc-snipping-tool-screenshot

But using the build-in screenshot feature of VLC shows a more promising prospect:

vlc-build-in-screenshot

FFmpeg

Run the original command to hardcode the new file.

ffmpeg -i "original-v2.mkv" -filter_complex "subtitles='original-v2.mkv':si=0" -pix_fmt yuv420p -c:v libx264 -crf 18 "output-v2.mp4"

output-v2

This looks almost identical to Rotem's result with scaling of the color matrix. I say almost as visually the rescaled one looks a tiny bit more dark (red).

Mediainfo of both results below for comparison, almost identical except for bitrate/filesize.

ffmpeg scale

General
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 1.55 MiB
Duration                                 : 8 s 426 ms
Overall bit rate                         : 1 548 kb/s
Writing application                      : Lavf59.16.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 8 s 426 ms
Bit rate                                 : 1 545 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.031
Stream size                              : 1.55 MiB (100%)
Writing library                          : x264 core 164 r3094 bfc87b7
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=34 / lookahead_threads=5 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=18.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Color range                              : Limited
Matrix coefficients                      : BT.709
Codec configuration box                  : avcC

aegisub-mkvtoolnix-ffmpeg

General
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 1.70 MiB
Duration                                 : 8 s 426 ms
Overall bit rate                         : 1 696 kb/s
Writing application                      : Lavf59.16.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 8 s 426 ms
Bit rate                                 : 1 693 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.034
Stream size                              : 1.70 MiB (100%)
Writing library                          : x264 core 164 r3094 bfc87b7
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=34 / lookahead_threads=5 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=18.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Color range                              : Limited
Matrix coefficients                      : BT.709
Codec configuration box                  : avcC

All results can be found here.


My conclusion is that this way there's no need for the scaling with ffmpeg, so better performance there, but it comes at the cost of manually changing the subtitle files properties (and resampling) & remuxing, therefore still being time-consuming.

While both solutions seem to work, I'm not sure if either one is an actual a viable solution appliciable for multiple use cases as they both are intensive operations to perform.

I would love to have a general solution that does not require this scaling and would "magically" figure this out for itself, but I just cannot figure out if this is 1) a bug in FFmpeg, 2) related to the subtitle renderer itself, 3) never been implemented, 4) something completely different.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .