I need to reduce the size of my photo library so I naturally want to compress them. Many of them are not quite identical, but still very similar (subsequent shots of the same scene). Is there any compression algorithm that takes advantage of this fact to effectively compress these images? 7zip (LZMA) is useless.
You might try Paq 8 (fp8_v2.zip). I just tried it myself on 1440 similar PNG images and then again on 111 similar JPG images. Here are the results.
- 1440 PNG Files, 28,631,615 bytes => 2,058,653 bytes compressed
- 111 JPG Files, 15,003,820 bytes => 489,096 bytes compressed
Compression of the PNG files took about 8 minutes and 550 MB of memory when using:
fp8_v2.exe -7 images *.png
Compression of the JPG files took about 5 minutes and 125 MB of memory when using:
fp8_v2.exe -5 images image12*.jpg
See also: jpg lossless image compression test
97% compression for the JPGs seems unrealistic. The numbers in the comparison test you link to say the compression is about 20% May 28, 2015 at 5:30
@OneSolitaryNoob Did you forget that the JPEGs are similar? 110/111 = 99.1% expected compression. The comparison test that he linked is about compressing a single JPEG.– NavinApr 22, 2018 at 19:51
@Navin it's really unlikely. Even if they look nearly identical most of the pixels will be slightly different. Apr 22, 2018 at 19:55
@OneSolitaryNoob So what? 1 second of video (30 frames) can be compressed to about the same size as a single frame. As long as the slight difference makes up a small fraction of the data, you're all good.– NavinApr 23, 2018 at 1:57
@Navin thats lossy compression, many details are gone but most people won't notice. Paq* is lossless compression Apr 23, 2018 at 1:59
Here's a simple solution which doesn't work for photos but may work if one has several images with large pixel-by-pixel identical areas: save the images in an unpacked format like BMP (not PNG or GIF) and then TAR them and compress with a decent compressor like XZ, e. g. on Linux with something like
tar -c myDirectory | xz -9 >myDirectory.tar.xz
Instead of TAR and XZ, one may use 7-Zip with the “solid archive” option to get roughly the same performance. This way I could compress 16 similar screenshots, that took about 900 KB each when saved as separate PNG files, into a 2 MB archive. The benefit of this solution is that it uses common file formats, so it works without installing new software. (Unfortunately the older and even more common programs GZIP and BZIP2 didn't do a good job for me — maybe because the block size of BZIP2 cannot be configured to be larger than 900 KB.)
I tried this on 719 consecutive screenshots of my desktop taken every 2 minutes, with resolution 5920x2560. The total size of source PNG files was 1,797,639,471 bytes (1.7 GiB), when converted to BMP, 40.5 GiB. The resulting xz file was only 1,387,249,076 bytes (1.3 GiB). So this technique can be useful! Sep 18, 2022 at 5:40
I would imagine that that the burrows-wheeler transform with an arithmetic coder would be ideal for this given a large enough window. What happens if you configure BZIP2 to use a block size equal to a small run of photos? It'll be slower and take more memory but the compression ratio should skyrocket. And have you tried LZMA with larger block sizes yet?
Not that I've seen. Probably the closest thing would be taking several similar JPEGs and putting them into an MJPEG movie. You could also use APNG or animated GIFs for a similar purpose.
I'm not sure how well that would work though, and it sounds like you're already talking about movie screencaps, so repacking them into a movie file sounds... counterproductive.
Maybe a better way, if you still have the clips that the screens came from, would be to simply find a command line tool that can extract the exact frame for you, copy that unique identifier into a text file someplace, and then you can always easily re-extract the frame when you need it.
GIF is not suitable for compressing photos, and animated GIF is even less. Even compressing photos to a MJPEG video does not seem useful in my eyes, because restoring single images would be difficult.– MartinApr 29, 2012 at 17:11
I didn't say it was a good idea... :-) The ultimate idea was to put the photos into an animated variant of their original format.– afrazierApr 29, 2012 at 20:25