0

Just out of curiosity, I checked two versions of a video file that were NOT re-encoded, just re-muxed. I was expecting ssim scores to be 1.0. Instead, I'm seeing scores of <1.0.

How to repeat:

First, re-mux a video:

MP4Box -cat video.mp4 -new test-1.mp4

-or-

mkvmerge -o test-2.mkv video.mp4

Now, I'd expect that all three of those files to be 100% identical in terms of quality. But when I use ffmpeg to check any two of them with ssim, I get something like this:

[Parsed_ssim_0 @ 0x7413a0] SSIM Y:0.899826 U:0.994930 V:0.994679 All:0.931486 (11.642182)

So, what's going on? Why are these three files not getting ssim scores of 1.0, when tested against each other? How is this affecting the ssim scores I'm getting with videos that actually are re-encoded?

ffmpeg: ffmpeg version 2.8.14-0ubuntu0.16.04.1

//// Edit: Adding actual terminal commands and output:

exiftool video.mp4
ExifTool Version Number         : 10.10
File Name                       : video.mp4
Directory                       : .
File Size                       : 10 MB
File Modification Date/Time     : 2018:07:02 02:08:41+12:00
File Access Date/Time           : 2018:07:02 02:09:09+12:00
File Inode Change Date/Time     : 2018:07:02 02:09:04+12:00
File Permissions                : rw-r--r--
File Type                       : MP4
File Type Extension             : mp4
MIME Type                       : video/mp4
Major Brand                     : MP4  Base Media v1 [IS0 14496-12:2003]
Minor Version                   : 0.2.0
Compatible Brands               : isom, iso2, mp41
Movie Data Size                 : 10427101
Movie Data Offset               : 44
Movie Header Version            : 0
Create Date                     : 0000:00:00 00:00:00
Modify Date                     : 0000:00:00 00:00:00
Time Scale                      : 1000
Duration                        : 0:01:01
Preferred Rate                  : 1
Preferred Volume                : 100.00%
Preview Time                    : 0 s
Preview Duration                : 0 s
Poster Time                     : 0 s
Selection Time                  : 0 s
Selection Duration              : 0 s
Current Time                    : 0 s
Next Track ID                   : 3
Track Header Version            : 0
Track Create Date               : 0000:00:00 00:00:00
Track Modify Date               : 0000:00:00 00:00:00
Track ID                        : 1
Track Duration                  : 0:01:01
Track Layer                     : 0
Track Volume                    : 0.00%
Image Width                     : 1920
Image Height                    : 1100
Graphics Mode                   : srcCopy
Op Color                        : 0 0 0
Compressor ID                   : hev1
Source Image Width              : 1920
Source Image Height             : 1100
X Resolution                    : 72
Y Resolution                    : 72
Bit Depth                       : 24
Video Frame Rate                : 30
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Media Header Version            : 0
Media Create Date               : 0000:00:00 00:00:00
Media Modify Date               : 0000:00:00 00:00:00
Media Time Scale                : 44100
Media Duration                  : 0:01:01
Media Language Code             : und
Handler Description             : SoundHandler
Balance                         : 0
Audio Format                    : mp4a
Audio Channels                  : 2
Audio Bits Per Sample           : 16
Audio Sample Rate               : 44100
Handler Type                    : Metadata
Handler Vendor ID               : Apple
Encoder                         : Lavf56.40.101
Avg Bitrate                     : 1.36 Mbps
Image Size                      : 1920x1100
Megapixels                      : 2.1
Rotation                        : 0

re-mux the file:

MP4Box -cat video.mp4 -new test-1.mp4
Appending file video.mp4                                                                                No suitable destination track found - creating new one (type vide)
No suitable destination track found - creating new one (type soun)
Saving test-1.mp4: 0.500 secs Interleaving

Compare the two files with ffmpeg/ssim:

ffmpeg -hide_banner -i video.mp4 -i test-1.mp4   -filter_complex "ssim" -f null -
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    title           : Intro
    encoder         : Lavf56.40.101
  Duration: 00:01:01.14, start: 0.000000, bitrate: 1376 kb/s
    Stream #0:0(und): Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv), 1920x1100, 1236 kb/s, 30 fps, 30 tbr, 16k tbn, 30 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x24117e0] multiple edit list entries, a/v desync might occur, patch welcome
    Last message repeated 1 times
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'test-1.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 1
    compatible_brands: isom
    creation_time   : 2018-07-01 14:09:09
  Duration: 00:01:01.14, start: 0.000000, bitrate: 1372 kb/s
    Stream #1:0(und): Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv), 1920x1100, 1236 kb/s, 30 fps, 30 tbr, 44100 tbn, 30 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #1:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    title           : Intro
    encoder         : Lavf56.40.101
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1100, q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc (default)
    Metadata:
      encoder         : Lavc56.60.100 rawvideo
    Stream #0:1(und): Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc56.60.100 pcm_s16le
Stream mapping:
  Stream #0:0 (hevc) -> ssim:main (graph 0)
  Stream #1:0 (hevc) -> ssim:reference (graph 0)
  ssim (graph 0) -> Stream #0:0 (rawvideo)
  Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
[null @ 0x24d47c0] Encoder did not produce proper pts, making some up.
frame= 1834 fps= 54 q=-0.0 Lsize=N/A time=00:01:01.13 bitrate=N/A
video:172kB audio:10532kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[Parsed_ssim_0 @ 0x23f90c0] SSIM Y:0.893099 U:0.994654 V:0.994361 All:0.926902 (11.360919)

This is the part I'm not expecting: SSIM Y:0.893099 U:0.994654 V:0.994361 All:0.926902

  • ffmpeg command & full console output missing. – Gyan Jul 1 '18 at 5:22
  • ffmpeg -hide_banner -i video.mp4 -i test-1.mp4 -filter_complex "ssim" -f null - Or, to just do a quick test on the first 60 seconds of video: ffmpeg -hide_banner -i video.mp4 -i test-1.mp4 -t 60 -filter_complex "ssim" -f null - – atom Jul 1 '18 at 9:33
  • Full console output missing – Gyan Jul 1 '18 at 9:45
  • OP edited to include more I/O from CLI. – atom Jul 1 '18 at 14:38
1

Here's the cause: multiple edit list entries, a/v desync might occur, patch welcome

I suggest using

ffmpeg -hide_banner -ignore_editlist 1 -i video.mp4 -ignore_editlist 1 -i test-1.mp4
       -filter_complex "[0]setpts=N/30/TB[a];[1]setpts=N/30/TB[b];[a][b]ssim" -f null -
  • Thanks. For these, just adding -ignore_editlist 1 before each -i file.mp4 does the trick and seems to solve the problem. – atom Jul 1 '18 at 20:43

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.