Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Suppose I have same video material encoded in two (or more) files. I'd like to run some utility on them which groundly pointed out which file is "best" in quality. "Groundly" means that I'd like to get report which compares different aspects (e.g. video resolution, video bitrate, audio sampling rate, audio bitrate, etc., etc.) one by one, and then some integral score which accounts for all of them.

That's about the functionality, but for that utility to be actually usable, it should be open-source and command-line.

share|improve this question
    
To start collecting some related info (not really a solution per criteria above), there's repo.or.cz/w/mplayer.git/blob/HEAD:/TOOLS/psnr-video.sh Here's "like a pro" stuff: compression.ru/video/quality_measure/… . But it's not open-source, and compares "original" and "copy", not just 2 unbiased files. –  pfalcon Sep 22 '11 at 17:45
    
Related question: stackoverflow.com/questions/3518417/… –  pfalcon Jan 12 '12 at 18:36

3 Answers 3

I work in video quality research (yes, there's something like this), and the thing you're looking for is somewhat everywhere but nowhere to be found. There are plenty of institutes who write their own video quality software, but most of them are either not available or not flexible enough.

As for your requirements: Just comparing video resolution won't tell anything about the quality. The same goes for bitrate (unfortunately), as different encoders can actually deliver better quality at less bitrate. You would also have to define how this "integral score" is composed of the individual quality factors. This heavily depends on:

  • the type of videos you are comparing (cartoons, movies, news, etc.)
  • their length
  • their viewing audience
  • their original frame size
  • et cetera

We're not even talking about how humans would perceive the videos.

I'll try to give you some tips later on, but first, I'm afraid I have never seen the thing that you're looking for. Let me give you a list of what I think of is most commonly used for basic evaluation of video quality today.

You are basically looking for a no-reference metric, i.e. a metric that does not depend on an original video to estimate the quality. Such a metric will take one video and output one quality score. If you do a web search (i.e. on Google Scholar) for "no-reference video quality", you'll find a lot of resources, as this is the holy grail the video quality research community is looking for at the moment. Might be that someone also posted their no-reference metric as open source code.

The following are mostly full-reference metrics. Their estimation of "quality" depends on an original video. For example, you could take a DVD movie, then create two rips from it, and use a full-reference metric to estimate the quality loss.

PSNR

For starters, PSNR (Peak Signal-to-Noise Ratio) is a very simple-to-use but somewhat poor method of assessing video quality. It's been around forever is used more often than it should be (even in research). It works relatively well though for most applications, and the PSNR script you found should do the trick for videos of the same frame size.

The basic idea is to compare the reference video (i.e. the original) against the processed video, and then use the PSNR value calculated between them, frame-by-frame.

Per your criteria though, this won't really help much. You could compare PSNR differences between each video, but that would be a combinatoric nightmare to analyze.

SSIM

Structural Similarity (SSIM) is as easy to calculate as PSNR, and it delivers more accurate results, but still on a frame-by-frame basis. You will find some implementations under the Wikipedia link. The results should be similar to PSNR. Again, you need to compare a reference to a processed video for this to work, and both videos should be of the same size.

I'm afraid, this is also not what you want.

VQM

The Video Quality Metric is standardized and patented, developed by the Institute for Telecommunication Sciences (ITS). It was heavily tested in the Video Quality Experts Group (VQEG). They're actually the main research group in this subject today (similar to the Motion Picture Experts Group for video).

You can download the VQM for free. There's a quite long list of individual software components that you will probably find useful. You'll need to register, but that shouldn't be the issue. You can then use it in MATLAB or as a standalone executable under Windows.

Still, the problem is: You need a reference video (as in the DVD case mentioned above). You can't just use one video and get a quality score.

MSU Video Quality Tool

Finally, the MSU Video Quality Measurement Tool is the Windows application to facilitate tasks like the one you're talking about.

[The MSU VQMT] is a program for objective video quality assessment. It provides functionality for both full-reference (two videos are examined) and single-reference (one video is analyzed) comparisons.

There's a free version, but it's not scriptable. The full version – which has batch processing capabilities – is rather expensive for personal use though. It's also not open-source, but I would recommend it if you have to do these tasks over and over again. We occasionally use it, and it seems a bit buggy, but it does its job.

The interesting part is that it has a few no-reference metrics, including a blocking and blurring metric. This would be something you could look for, since blocking is very common in low-quality videos:


If you just seek to compare simple objectively measurable criteria like:

  • Frame size
  • Bit rate
  • Frames per second
  • Video resolution

… a simple call to ffmpeg -i should give you all the details you need at the beginning. Also have a look at the -vstats option. You could then summarize this in a spreadsheet. Note that when you encode videos, x264 for example will log stuff like PSNR straight to a file if you need to, so you can use these values later.

As for how to weigh these criteria, you should probably emphasize the bit rate – but only if you know that the codec is the same. Comparing different codecs according to their bit rate is not possible unless you know more about the content and the individual encoding settings. Frame rate is a very subjective thing too and should be counted into your measurements if it is well below 25 Hz.

To summarize, heavily emphasize the bitrate if it's the only thing you have. Don't forget to use your eyes, too :)

share|improve this answer
1  
+1 @slhck - Awesome answer. –  jftuga Sep 22 '11 at 19:10
    
First of all, as the original author of the question, I'm sorry for not commenting before. The post is indeed awesome and well appreciated. Unfortunately, I cannot accept it as the answer to my original question. The reason is: I deliberately posted the question here and not on StackOverflow, because I wanted suggestion of existing, ready-to-use tool. Be it a question of how to write such tool, your answer would be absolute winner. But sorry, I cannot write everything I need from scratch, so let user in me asking questions and expecting answers (maybe not yesterday or today, maybe in future;-) –  pfalcon Jan 12 '12 at 16:57
    
@pfalcon No worries. I totally understand your situation. It's just that I've never heard of something like this.. maybe only stuff some research departments wrote themselves. –  slhck Jan 12 '12 at 17:08

I'm unaware of any tool which will give you a final recommendation or score, but using FFmpeg, you can output all the details you listed in the question.

On the command line, ffmpeg -i will list the information from the video. From there, you can write a script to parse the information and weight it as you see appropriate.

share|improve this answer
    
Thanks, yep, that's what I was doing for couple of hours before deciding to ask if some better, "magic" tool exists ;-) –  pfalcon Jan 12 '12 at 16:59

http://mediaarea.net/en/MediaInfo

Media Info will give you detailed informaton about your video and audio files

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.