What is the most efficient compression format for ISO files created from DVD movies?
I wouldn't mind compressing them one at a time or a whole group of them, whatever would save me the most space on my storage device.
Don't bother. First of all, the movies are already compressed. (What do you think MPEG-2, h.264, etc. are?) They're not going to compress very much at all. Second of all, even if the data were compressible, the result of them being put into an ISO filesystem makes them much less compressible than they would normally be. That's not to say that ISOs don't compress very well, just normally not as well as the raw data would.
You're talking about tons of overhead to save, maybe, a few MB per ISO.
If you really want to save space, convert the movies to a more space-efficient format, like XviD (MPEG-4 ASP) or preferably h.264 (MPEG-4 AVC).
VOB files, which are the bulk of the data on a DVD are already compressed, you will gain very little space by compressing the ISO file.
I did a sample compression of just one VOB file I had,
Roughly a 3% size reduction.
I'm 100% sure you could, in theory, recompress DVDs in a lossless manner with non-trivial ratio of compression, because MPEG-2 is not really optimal (read: atrocious) and was thought more for speed of decoding than for efficiency, but I can't find anything out there that does it (it would actually be very complicated and I can't see it being neither easy to sell nor terribly fun to code).
Actually, read this reply in a forum from one of the lead x264 devs.
As for practical solutions, you can re-compress it using a lossy format. This assumes you only care about the movie and not about menus or specials (you'd have to encode each extra clip as an extra file). I actually like this and don't care too much for the menus, so this is good enough for me, and you can fit an entire DVD into the size of a CD with unnoticeable difference in quality.
There are many tutorials online about how to do this, and it would be boring to just say the same things, besides they're so easy to find. My tools of choice are MeGUI, Avisynth, x264, Nero's AAC codec and MKVMerge GUI, but anything that you like is good. These tools I mentioned also let you deal with noisy sources, de-interlacing (a whole topic of its own), and various artifacts (dot-crawl, logo removal, blurry images, cartoon or anime restoration, grainy film, etc).
First, lossless movie formats don't really exist.
Attempting to capture 640x480 pixels at 24-bit color 30-times every second would result in ~27MB/second of raw video. It is very unrealistic to store that on most forms of portable media (DVD/portable players/etc.), especially when you stop to consider video at hidef resolutions like 1080p.
It only gets exponentially worse. Typically, for most DVD formats you are using an MPEG format (MPEG 2) which has a pretty good quality to loss ratio. It uses similar compression schemes to the JPEG image format.
A lot of new streaming media makes use of other compression schemes (they're still lossy compression) like h.264. Such formats are not typically readable by your home DVD player. They're still not "lossless" compression.
Additionally, moving from one compression scheme to another, will typically result in additional loss in quality.