We're a small group that is promoting the spread of Unicode in India (here legacy encodings are deeply entrenched). But I have a problem when I convert a document in unicode text in any Indic language to PDF format. The text displays as intended, but on copy pasting the content partially turns gibberish.

I am using inDesign CC for typesetting on a Win 7. I can export to epub format just fine. But the exported PDF has this problem. I also tried printing to Adobe PDF printer and PrimoPDF, it only got worse. On checking out PDF's on the internet, turns out this problem exists in all such unicode encoded Indic PDF (and probably all East Asian complex scripts). Is that a problem in the PDF specs?

Check out the PDF here http://www.rajbhasha.nic.in/pdf/dolebook-4.pdf

Copy any text and match with the original, you'll see characters are replaced by other characters, unnecessary white space has crept in.

Now we're promoting unicode on grounds that it'll make copy-pasting and searching/indexing easier. This problem totally destroys that. Any ideas?

  • I can confirm that copy/pasting your document on a Mac also alters the characters. I can't read it, but there are a few noticeable differences. That might suggest the source conversion is at fault. Maybe have a look at Calibre (freeware) to do the conversion instead. It might at least tell you where the issue starts.
    – Tetsujin
    Sep 15, 2016 at 10:18
  • Can confirm copy/pasting problem on Linux with xpdf. I looked at the PDF with mutool, it uses special fonts that don't use unicode encoding. You need some other program to do your typesetting (instead of inDesign CC), and one that produces PDFs with unicode encoding (no, I don't know any option for Windows 7). This question has technical details about unicode in PDF, it seems possible, but not easy to do.
    – dirkt
    Sep 15, 2016 at 11:35
  • Correction: the example PDF actually uses /ToUnicode mappings, but they don't seem to work for some reason. Don't know yet what goes wrong.
    – dirkt
    Sep 15, 2016 at 11:51
  • See also: stackoverflow.com/questions/12703387/… Sep 15, 2016 at 13:35
  • @Tetsujin: I did try to convert the resulting epub to pdf using various tools, including calibre. The problem actually worsened. Sep 17, 2016 at 17:27

1 Answer 1


I decompressed the pdf with mutool clean and had a look at. The problem seems to be that as described as in this stackoverflow question, it's difficult to use unicode encoding for the fonts. For this reason, the fonts that the PDF contains use a different encoding. However, it also contains /ToUnicode objects for each font with a complicated mapping from the font glyphs to the unicode characters.

Now many PDF viewers (like e.g. xpdf on Linux) don't seem to pay attention to this complicated mapping (or at least not to a mapping with such a complexity, though they may work on more simple mappings), which is why you get garbage when trying to copy and paste. However, with other PDF viewers (like mupdf) it works, as I've confirmed.

So the problem is located in the PDF viewer, not in the document. Also, PDFs and unicode don't go together that well, as you can see from the complicated means necessary to do the translation.

Possible solutions: (1) pressure the developers of PDF viewers to fully support \ToUnicode mappings. Maybe fix them yourself for open source ones. (2) Promote the usage of a particular PDF viewer that works with the mappings. (3) Try to use fonts inside the PDF where the glyph encoding matches the unicode encoding. This seems possible with 16-bit unicode codepoints (and the Indian characters seem to be 16-bit as far as I can tell), but I don't know how well this will work, or which application you should use to produce such PDFs.

  • This seems to be correct, because with different PDF readers I get different outputs on copying which is only possible if each one implemented the ToUnicode table interpretation differently. As for your proposed solutions, 1 and 2 are not feasible because people wont shift their OS and pdf reader just for this. And all pdf readers i've tested have problems (Adobe Acrobat, PDF X, Foxit, Google pdf viewer). As for 3, I didn't get it. This document uses standard Unicode Hindi font Mangal supplied with Win Vista and above. Sep 17, 2016 at 17:41
  • Mupdf also works on Windows, so you can try that too. qpdfview on Linux also works. I'll try to make an example file for (3) to see if it works even without ToUnicode tables, but that might take some time.
    – dirkt
    Sep 17, 2016 at 19:31
  • It looks like xpdf just ignores any "complex" characters besides ASCII for cut and paste, while mupdf produces an UTF8 encoded paste. That means I can't properly test here on Linux. I've created a decompressed PDF file with xetex that is not as complex as your example. You can inspect it with a text editor. Glyphs are in the 0200-0400 range, corresponding unicode is 09xx. Test your viewers with it, if you can paste unicode chars in the 0200-0400 range, creating a special font should work with that viewer.
    – dirkt
    Sep 19, 2016 at 9:25
  • I was away from home, just returned and tested mupdf on Win 7 and Ubuntu 14.04. The same problem persists on copying. I am not sure, what I am doing wrong. I tried your PDF on all my viewers as well. No luck. Sep 19, 2016 at 18:54
  • Huh. I'm on Debian, which is very close to Ubuntu, and mupdf works fine (shift-right button to select). Where were you pasting it into? Can you do a xclip -o | hexdump -C from the commandline on the selection and post the results? (Packages xclip, bsdmainutils if not installed). Also, can you post what exactly are the results for my PDF with the various viewers? A tool like inside clipboard helps, IIRC it also shows hex.
    – dirkt
    Sep 20, 2016 at 6:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .