Remove blank pages from PDF from command line

Question

I have a PDF that has some blank pages inserted. These pages are the background colour (grey in this case). I would like to remove these pages using a bash script.

It has been suggested that we can to scan for text using e.g. pdftotext, but in my case this does not find text even on the non-blank pages.

Answer 1

Thank you for the code, gmatht. I have modified it to check for page coverage using GhostScript and delete pages with coverage less than a threshold (0.1%).

#!/bin/sh
IN="$1"
filename=$(basename "${IN}")
filename="${filename%.*}"
PAGES=$(pdfinfo "$IN" | grep ^Pages: | tr -dc '0-9')

non_blank() {
    for i in $(seq 1 $PAGES)
    do
        PERCENT=$(gs -o -  -dFirstPage=${i} -dLastPage=${i} -sDEVICE=inkcov "$IN" | grep CMYK | nawk 'BEGIN { sum=0; } {sum += $1 + $2 + $3 + $4;} END { printf "%.5f\n", sum } ')
        if [ $(echo "$PERCENT > 0.001" | bc) -eq 1 ]
        then
            echo $i
            #echo $i 1>&2
        fi
        echo -n . 1>&2
    done | tee "$filename.tmp"
    echo 1>&2
}

set +x
pdftk "${IN}" cat $(non_blank) output "${filename}.pdf"

Answer 2

There does not appear to be a utility to remove blank pages from PDFs, but we can create a histogram of colours using the convert command from imagemagick. Blank slides will only have one entry which can be detected with wc. Once we have a list of non-blank pages we can feed this into pdftk.

Note that imagemagick numbers pages starting from 0, so we need to adjust for this. We can use a low value in the -density flag improve performance (though too low seems to result in imagemagick segfaulting).

If we call the following script pdf_rm_blank.sh, running pdf_rm_blank.sh A will create A.rm.pdf from A.pdf

#!/bin/sh
IN="$1"
PAGES=$(pdfinfo $IN.pdf | grep ^Pages: | tr -dc '0-9')

non_blank() {
    for i in $(seq 1 $PAGES)
    do
        if [ $(convert -density 35 "$IN.pdf[$((i-1))]" -define histogram:unique-colors=true -format %c histogram:info:- | wc -l) -ne 1 ]
        then
            echo $i
            #echo $i 1>&2
        fi
        echo -n . 1>&2
    done | tee out.tmp
    echo 1>&2
}

set +x
pdftk $IN.pdf cat $(non_blank) output $IN.rm.pdf

Answer 3

If you can assume that an page is empty if there is no text, you can go with the following code. If you have pdfs with pages with only charts, images, ... this won't work I think.

First use xpdf/pdf2text to extract the txt of the pdf. Detect "Page break/new pages" with char 0x0C. To remove empty pages just use pdftk and cat all non-empty pages to a new pdf.

  /** page break constant in pdf */
  private static final String PAGEBREAK = new String(new byte[] { 0x0C });
  /** dummy string for an empty page */
  private static final String EMPTY_PAGE = "EMPTY_PAGE";

  /**
   * @param pdfIn
   * @param pdfOut
   * @param txt --> contains pdf2txt output of tool xpdf/pdftotext.exe
   * @return
   * @throws Exception
   */
  private static byte[] removeEmptyPages(File pdfIn, File pdfOut, String txt) throws Exception {
    // replace "page break" with some dummytext+"page break"
    txt = txt.replace(PAGEBREAK, EMPTY_PAGE + PAGEBREAK);
    StringTokenizer tokenizer = new StringTokenizer(txt, PAGEBREAK);
    int pageCounter = 0;
    String pagesWithContent = "";
    boolean foundEmptyPage = false;
    String currentPage = null;
    while (tokenizer.hasMoreTokens()) {
      currentPage = tokenizer.nextToken();
      pageCounter++;
      if (currentPage.equals(EMPTY_PAGE)) {
        foundEmptyPage = true;
      } else {
        pagesWithContent += (pageCounter + " ");
      }
    }

    if (foundEmptyPage) {
      String pdfShellCmd = "..\\tools\\pdftk\\bin\\pdftk.exe \"$IN\" cat $PAGES output \"$OUT\"";
      String cmd = pdfShellCmd.replace("$IN", pdfIn.toString());
      cmd = cmd.replace("$OUT", pdfOut.toString());
      cmd = cmd.replace("$PAGES", pagesWithContent);

      int resultCode = executeShellCmd(cmd);
      if (0 == resultCode) {
        return FileTools.readFile(pdfOut).array();
      } else {
        throw new Exception("Result code for " + cmd + " was " + resultCode);
      }
    } else {
      // if no empty pages, return input file
      copyFile(pdfIn, pdfOut);
      return read(pdfOut);
    }
  }

Answer 4

My intention is to first slice the original pdf into small pdfs then removed the blanks from them and again merge back.

######++++++start blankRemover.sh ++++++++++

#!/bin/sh
IN="${1}"  #Input PDF File Name
OUT="${2}"   #Output PDF File Name
filename=$(basename "${IN}")
filename="${filename%.*}"
echo "filename: ${filename}"
PAGES=$(pdfinfo "$IN" | grep ^Pages: | tr -dc '0-9')

ranges="myout ; " 
for i in $(seq 1 $PAGES)
do
        pdftk "${1}" cat $i output "${1}-temp.pdf"
        pdftotext "${1}-temp.pdf";
        cmp -s blank.txt  "${1}-temp.txt";if test $? -ne 0;then ranges="${ranges}$i ";fi;
        rm -f "${1}-temp.pdf" "${1}-temp.txt"
done
#echo "$ranges"
ranges=`echo ${ranges%?} | cut -d ';' -f 2`
pdftk "${1}" cat $ranges output "${2}"

#######++++++++ end blankRemover.sh +++++++++++++++++++++++++++

#######++++++++ start blankRemover-main.sh ++++++++++++++++++++

#!/bin/sh
IN="${1}"   #Input PDF File Name
OUT="${2}"  #Output PDF File Name 
sizeDefine=$(expr $3)    #slice the pdf
filename=$(basename "${IN}")
filename="${filename%.*}"
echo "filename: ${filename}"
PAGES=$(pdfinfo "$IN" | grep ^Pages: | tr -dc '0-9')

actualSize=${PAGES}
start=1
currentMin=$start
currentMax=$sizeDefine
tempNumPdf=1
echo $actualSize
echo $currentMin
echo $currentMax
echo $tempNumPdf
while test  $currentMax -le $actualSize
do
        pdftk "${IN}" cat $(expr $currentMin)"-"$(expr $currentMax) output "temp-$tempNumPdf.pdf"
        currentMin=$(expr $currentMax + 1 )
        currentMax=$(expr $currentMax + $sizeDefine )
        tempNumPdf=$(expr $tempNumPdf + 1 )
        echo "actualSize: $actualSize"
        echo "currentMin: $currentMin"
        echo "currentMax: $currentMax"
        echo "tempNumPdf: $tempNumPdf"
done
echo "while done"

if test $currentMin -lt $actualSize
then
        pdftk "${IN}" cat $currentMin-$actualSize  output "temp-$tempNumPdf.pdf"
else
        tempNumPdf=$(expr $tempNumPdf - 1 )
fi

for i in $(seq 1 $tempNumPdf)
do
        ./blankRemover.sh "temp-$i.pdf" "temp_update_$i.pdf"
done
echo "for done"
rm -f temp-*.pdf 
pdftk temp_update_*.pdf cat output "${2}"
rm -f temp_update_*.pdf

#######++++++++ end blankRemover-main.sh ++++++++++++++++++++++

Answer 5

In Linux I did this (with an old dictionary of more than 1500 pages but with more than one hundred blanks):

Extract all pages as images or pdf, look at the very small size of the blank ones, and remove them all. Then, create a new pdf out of the remaining files.

To extract pages there are many ways, for example as pdf:

pdftk FILE burst

as png:

pdftoppm INPUT OUTPUT -png

pdfimages INPUT OUTPUT -png

To join pages:

pdftk INPUT cat output OUTPUT.pdf

Answer 6

As most of the answers show, there doesn't seem to be one tool to get this done. I had to patch together some existing tools, taking inspiration from the use of gs in @Antony's answer.

I found I couldn't automate this from top-to-bottom, it needed some fine tuning along the way. The scripts below take a directory and batch operate on all PDF files. I ended up with 3 distinct steps:

1. Gather some per-page metrics about each page in the PDFs using Ghostscript's ink_cov output device (I found ink_cov's mean percent values much more useful than the "percent of pixels with non-zero channels" values returned by inkcov):

   #!/usr/bin/env bash
   
   device="ink_cov"
   out="/tmp/pdf_trim/analysis.txt"
   
   [ "$#" -eq 1 ] || { echo "Target directory required as argument"; exit 1; }
   in="$(realpath "$1")"
   [ -f "$out" ] && rm "$out" || mkdir -p "$(dirname "$out")"
   pushd "$in"
   find . -name '*.pdf' | while read p; do
     gs -o - -sDEVICE="$device" "$p" | grep CMYK | grep -n '' | \
       sed 's/:/ /; s|^|'$in' '$(echo "$p" | sed 's|^\./||')' |' | \
       tee -a "$out"
   done
   

   Example usage:

   ./script_1 ./target_folder
   

2. Fiddle around with different "blank-page" criteria from the metrics reported by GhostScript, until you find a "good one" for your PDFs. A good criteria should sort your pages from blank to not blank.

   #!/usr/bin/env bash
   
   criteria='$4+$5+$6+$7'
   
   in="/tmp/pdf_trim/analysis.txt"
   out="/tmp/pdf_trim/criteria"
   
   tmp_over="/tmp/pdf_trim/over.pdf"
   tmp_under="/tmp/pdf_trim/under.pdf"
   
   # Apply the criteria to each line, and then sort
   with_criteia="$(cat "$in" | awk '{ print $0, '$criteria' }' | \
     sort -n -k 10 | tee "$out.txt")"
   
   # Create an overlay pdf with the criteria values printed
   echo "$with_criteia" | awk '{printf "%s: %s p%s\n", $10, $2, $3 }' | \
     enscript --no-header --font Courier-Bold18 --lines-per-page 1 -o - | \
     ps2pdf - "$tmp_over"
   
   # Create an underlay pdf with the sorted pages by generating PDFtk handle lists
   handles="$(paste -d ' ' \
     <(echo "$with_criteia" | grep -n '' | sed 's/:.*//' | tr '0-9' 'A-Z') \
     <(echo "$with_criteia"))"
   pushd "$1"
   pdftk $(echo "$handles" | awk '{ printf "%s=%s/%s ", $1, $2, $3 }') \
     cat $(echo "$handles" | awk '{ printf "%s%s ", $1, $4}') \
     output "$tmp_under"
   
   # Merge them into the final result & remove temporary files
   pdftk "$tmp_over" multibackground "$tmp_under" output "$out.pdf"
   rm "$tmp_over" "$tmp_under"
   

   Example usage (this will create a criteria.pdf file with pages sorted by your chosen criteria):

   ./script_2
   

3. Batch re-generate each of the PDFs in a new directory, minus the blank pages:

   #!/usr/bin/env bash
   
   threshold=1.59
   
   input="/tmp/pdf_trim/criteria.txt"
   
   [ "$#" -eq 1 ] || { echo "Output directory required as argument"; exit 1; }
   out="$(realpath "$1")"
   
   in_list="$(cat $input)"
   out_list="$(cat "$input" | awk '$10 >'$threshold' {print}' | \
     sort -k 2,2 -k 3,3n)"
   
   in_files="$(echo "$in_list" | cut -d ' ' -f 1,2 | sort -u )"
   out_files="$(echo "$out_list" | cut -d ' ' -f 1,2 | sort -u)"
   
   echo "$out_files" | while read f; do
     dest="$(echo "$f" | sed 's|[^ ]* |'$out'/|; s/\.pdf$/_trimmed\.pdf/')"
     echo "$dest"
     mkdir -p "$(dirname "$dest")"
     pdftk "$(echo "$f" | sed 's| |/|')" \
       cat $(echo "$out_list" | grep "$f" | cut -d ' ' -f 3 | tr '\n' ' ' | \
         sed 's/ $//') \
       output "$dest"
   done
   
   printf "\nTrimmed %s pages with criteria value below %s\n" \
     "$(($(echo "$in_list" | wc -l) - $(echo "$out_list" | wc -l)))" "$threshold"
   printf "All pages were skipped from the following files:\n%s\n" \
     "$(comm -23 <(echo "$in_files") <(echo "$out_files") | sed 's/^/\t/; s| |/|')"
   

   Example usage:

   ./script_3 ./output_directory
   

I've written a detailed post about this, with more information and explanations about each of the steps.