I work in video quality research (yes, there's something like this), and the thing you're looking for is somewhat everywhere but nowhere to be found. There are plenty of institutes who write their own video quality software, but most of them are either not available or not flexible enough.
As for your requirements: Just comparing video resolution won't tell anything about the quality. The same goes for bitrate (unfortunately), as different encoders can actually deliver better quality at less bitrate. You would also have to define how this "integral score" is composed of the individual quality factors. This heavily depends on:
- the type of videos you are comparing (cartoons, movies, news, etc.)
- their length
- their viewing audience
- their original frame size
- et cetera
We're not even talking about how humans would perceive the videos.
I'll try to give you some tips later on, but first, I'm afraid I have never seen the thing that you're looking for. Let me give you a list of what I think of is most commonly used for basic evaluation of video quality today.
You are basically looking for a no-reference metric, i.e. a metric that does not depend on an original video to estimate the quality. Such a metric will take one video and output one quality score. If you do a web search (i.e. on Google Scholar) for "no-reference video quality", you'll find a lot of resources, as this is the holy grail the video quality research community is looking for at the moment. Might be that someone also posted their no-reference metric as open source code.
The following are mostly full-reference metrics. Their estimation of "quality" depends on an original video. For example, you could take a DVD movie, then create two rips from it, and use a full-reference metric to estimate the quality loss.
For starters, PSNR (Peak Signal-to-Noise Ratio) is a very simple-to-use but somewhat poor method of assessing video quality. It's been around forever is used more often than it should be (even in research). It works relatively well though for most applications, and the PSNR script you found should do the trick for videos of the same frame size.
The basic idea is to compare the reference video (i.e. the original) against the processed video, and then use the PSNR value calculated between them, frame-by-frame.
Per your criteria though, this won't really help much. You could compare PSNR differences between each video, but that would be a combinatoric nightmare to analyze.
Structural Similarity (SSIM) is as easy to calculate as PSNR, and it delivers more accurate results, but still on a frame-by-frame basis. You will find some implementations under the Wikipedia link. The results should be similar to PSNR. Again, you need to compare a reference to a processed video for this to work, and both videos should be of the same size.
I'm afraid, this is also not what you want.
The Video Quality Metric is standardized and patented, developed by the Institute for Telecommunication Sciences (ITS). It was heavily tested in the Video Quality Experts Group (VQEG). They're actually the main research group in this subject today (similar to the Motion Picture Experts Group for video).
You can download the VQM for free. There's a quite long list of individual software components that you will probably find useful. You'll need to register, but that shouldn't be the issue. You can then use it in MATLAB or as a standalone executable under Windows.
Still, the problem is: You need a reference video (as in the DVD case mentioned above). You can't just use one video and get a quality score.
MSU Video Quality Tool
Finally, the MSU Video Quality Measurement Tool is the Windows application to facilitate tasks like the one you're talking about.
[The MSU VQMT] is a program for objective video quality assessment. It provides functionality for both full-reference (two videos are examined) and single-reference (one video is analyzed) comparisons.
There's a free version, but it's not scriptable. The full version – which has batch processing capabilities – is rather expensive for personal use though. It's also not open-source, but I would recommend it if you have to do these tasks over and over again. We occasionally use it, and it seems a bit buggy, but it does its job.
The interesting part is that it has a few no-reference metrics, including a blocking and blurring metric. This would be something you could look for, since blocking is very common in low-quality videos:
If you just seek to compare simple objectively measurable criteria like:
- Frame size
- Bit rate
- Frames per second
- Video resolution
… a simple call to
ffmpeg -i should give you all the details you need at the beginning. Also have a look at the
-vstats option. You could then summarize this in a spreadsheet. Note that when you encode videos,
x264 for example will log stuff like PSNR straight to a file if you need to, so you can use these values later.
As for how to weigh these criteria, you should probably emphasize the bit rate – but only if you know that the codec is the same. Comparing different codecs according to their bit rate is not possible unless you know more about the content and the individual encoding settings. Frame rate is a very subjective thing too and should be counted into your measurements if it is well below 25 Hz.
To summarize, heavily emphasize the bitrate if it's the only thing you have. Don't forget to use your eyes, too :)