9

I need to both create and split multipage TIFF images, ranging from 2 to almost 100 pages (A4, 300 dpi, 2500×3500 px). The job is performed periodically by a script on an x64 Linux server. Currently I'm using Imagemagick. The smaller cases do not pose any problems, but the larger ones do.

I need to radically reduce amount of memory used during the operation.

For example, this:

convert *.jpg -compress lzw output.tif

(70 jpeg files) consumes about 4.6 GB of RAM, even though each input is less than 2MB the resulting file is less than 250MB.

The reverse operation:

convert input.tif output-%04d.png

has similar issues.

From what I have read, this happens because Imagemagick first loads and decodes all the input images and only after that it starts encoding them into the output file.

How can I create and split multipage TIFF images without such huge memory footprint? I don't have to necessarily use ImageMagick, any other free tool will be fine.

3
  • To put some perspective on it: EACH 2500×3500 pixel image will take up 2500×3500×3 bytes at least as it resides in memory.That is 26250000 bytes per image, 1837500000 bytes total for 70 images. Then you create a DUPLICATE of that in the TIF, total 3675000000. Then you request to save it using lzw compression; some buffers is probably required for that. Maybe add buffers for writing... Handling 70-100 page files isn't easy, especially if the pages are nothing but bitmaps.
    – Hannu
    Aug 6, 2014 at 12:59
  • @Hannu Not easy for who? The real world says that there's a concept of streaming transforms, and unpacking a huge image stack in memory simultaneously is lame and fugly. Mar 25, 2016 at 0:13
  • The first example convert above creates a single PAGED tiff. Depending on how Imagemagick works internally, you MIGHT indeed have a "huge image stack in memory".
    – Hannu
    Mar 28, 2016 at 8:52

4 Answers 4

7

I had the same problem today while trying to split a 1700 image, 1G tif file. 16G of memory wasn't enough, then tried having it cache on disk, but that was slow and it easily exhausted more than 100G on the hard drive without accomplishing anything (this was probably a bug).

But apparently ImageMagick can extract a specific tif from the original file without loading it completely, so was able to split the bigger file with a simple bash script:

subfiles=$(identify -quiet -format '%n\n' largefile.tif | head -n1)
for (( i = 0; i < subfiles; i++ )); do
    convert largefile.tif[$i] -scene 1 split/smallerfile_$i.tif
done

No idea though how to create a big file without running out of memory, so maybe this is half an answer?

3

I find @tarikki's answer one of the best, because it really doesn't hang the server nor does it eat RAM and disk space, and it's fast.

Some improvements that helped me:

  1. replace END=2000 by END=$(identify -format "%n" largefile.tif)
  2. The TIF index is 0-based, so the looping should start at 0 and use < instead of <= : for((i=0;i<END;i++))
3
  • This returns e.g. 666666, so a 6 for every of the six pages, on my system, so it's not directly usable. Any idea why? I opened a new question. Jul 26, 2019 at 23:07
  • 2
    Yes, 666666 is correct output, as I also found out after posting here. What you see here is the total number of pages, listed for each page. To get only the first entry, I had to modify the code into : zzcount=$(identify -quiet -format "%n\n " "$1" | head -n1); Where $1 is the TIF file.
    – MrMacvos
    Jul 27, 2019 at 0:32
  • Your answer probably should have been an edit, rather than a separate one. This is done now.
    – Walf
    May 24, 2021 at 7:27
1

You can do it with PowerShell for an entire folder of TIFFs using the latest ImageMagick for Windows and the following script:

$Files = Get-ChildItem
mkdir split

$Files | ForEach-Object {
    $PageCount = (magick identify -format '%n' $_.Name).length
    for ($i = 0; $i -lt $PageCount; $i++) {
        $Name = $_.Name + "[" + $i + "]"
        $OutputName = "split\" + $_.Name.Split(".")[-2] + "-" + $i + ".tif"
        magick convert $Name -scene 1 $OutputName
    }
}
0

tiffcp can be used to create a multipage tiff, like this:

tiffcp *.tif out.tif

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .