124

Unicode contains various characters that look like typographically stylised variants of characters of the basic Latin alphabet and that allow one to write texts in the corresponding typographic styles without resorting to mark-up or similar. For example, one can simulate:

  • Small caps:

    ʙᴇʜᴏʟᴅ ᴛʜɪꜱ ꜰᴀɴᴄɪʟy ᴇɴᴄᴏᴅᴇᴅ ᴛᴇxᴛ.

  • Script:

    𝓑𝓮𝓱𝓸𝓵𝓭 𝓽𝓱𝓲𝓼 𝓯𝓪𝓷𝓬𝓲𝓵𝔂 𝓮𝓷𝓬𝓸𝓭𝓮𝓭 𝓽𝓮𝔁𝓽.

  • Blackletter:

    𝕭𝖊𝖍𝖔𝖑𝖉 𝖙𝖍𝖎𝖘 𝖋𝖆𝖓𝖈𝖎𝖑𝖞 𝖊𝖓𝖈𝖔𝖉𝖊𝖉 𝖙𝖊𝖝𝖙.

This met interest on Stack Exchange (e.g., here, here, and here) and criticism of such techniques was made. But what can go wrong when I use them?

  • 223
    I'm reading this from my phone and I can't see the last two fancy texts. – Scimonster Dec 26 '16 at 13:00
  • 22
    Because it is unreadable on some devices: i.stack.imgur.com/kM73J.png – Chris Kent Dec 26 '16 at 18:28
  • 15
    Because some of us want to see web pages in what WE consider to be readable fonts (and sizes, colors, &c), so we use e.g. user CSS stylesheets to override author styles. You might note that even though your three examples display on my device, apparently just as you intend them to appear, to me they are only borderline readable. Why would you place your artistic cravings above your readers' ease of reading? – jamesqf Dec 26 '16 at 19:28
  • 38
    Here's an interesting observation: Edge cannot find text in the latter two samples, and Chrome cannot find text in the first one. (Try Ctrl+F'ing for BEHOLD in both browsers.) Haven't checked Firefox. – Schism Dec 26 '16 at 20:16
  • 22
    @Schism Firefox finds none of them. Looks like Chrome probably uses NFKC/NFKD normalisation before searching, which decomposes the script and blackletter text to Basic Latin. Firefox doesn't seem to do so. Edge... is doing something weird. – Bob Dec 27 '16 at 11:19
219

General

Those characters are not intended for regular Latin-alphabet text but for phonetics, Cyrillic-alphabet text, for use as mathematical symbols (representing variables), or similar. The only Unicode-compliant way to encode text in the basic Latin alphabet is to use the characters predominantly used for this purpose (i.e., from the Basic Latin Unicode block).

As with many other standards, you should think twice about violating Unicode. Moreover, Unicode comprises so many writing systems, use cases, and stuff that just exists for backwards compatibility with other standards1 that fully understanding all its motivations is a science of its own. Long story short, unless you really really know what you are doing, it is extremely likely that something breaks that you haven’t even remotely thought of.

Specific examples

Accessibility

Encoded text does not only exist to be rendered in some font. It can also be interpreted, e.g., by screen readers. And a screen reader should not need to guess whether

𝓽𝓱𝓮

is meant to be the definite article or the mathematical product2 of the variables 𝓽, 𝓱, and 𝓮 – which is what those characters are made for. The best behaviour will therefore be that it spells out these characters, e.g. saying literally the following:

bold script small t, bold script small h, bold script small e

It should not just say “the” instead because then it would not properly read mathematical texts whose symbols happen to form a pronouncable word.3

Portability

If your text is nicely rendered on your machine, this does not mean that it will also be on the reader’s one. The most obvious example is that the reader does not have any font supporting these characters or the text is rendered by a software not supporting fallback fonts. Admittedly, this is becoming increasingly less common. Keep in mind though that some people like dyslexics need special fonts which are less likely to support these characters.

But even if the reader’s machine only uses a different font, this may make the text considerably less readable. For a first example, this is 𝓉𝒽ℯ rendered with two different fonts:

𝓉𝒽ℯ rendered with FreeSerif and STIX

Free Serif renders the text as you would probably want it to be rendered when using special characters to simulate text, namely simulating handwriting with a continuous stroke. However, these characters are made for use as mathematical symbols, connecting which makes no sense. Hence the rendering by STIX, which is specifically designed for mathematical purposes, is more in line with how these characters are intended to be used.

In a second example, suppose you or the reader italicise “сᴜт мy вᴀʀ” for some reason. With a good font, you will get4:

сᴜт мy вᴀʀ rendered in upright and italics. italics reads as "cum my bar"

The reason for this is that the small caps were (partially) simulated with Cyrillic letters, and Cyrillic italics sometimes look very different from their upright counterparts. So again, this is the proper behaviour.

Searchability

As a first example, consider what you would want a reasonable search to do with the character 𝒲 (mathematical script W). Assume that the search has two modes, the default mode and the exact mode (usually called case-sensitive). This character should be:

  • found when searching for w or W in default mode – for those who do not want to bother entering or copy-pasting the special character into the search field;

  • found when searching for 𝒲 in exact mode – for those who want to search where the corresponding variable is mentioned in a mathematical document³;

  • not found when searching for 𝓌, w or W in exact mode on account of breaking a search similar to the above.

However if you use this character to simulate regular text, it should be found when searching for W or 𝒲 in exact mode, which is in conflict with the above.

As a second example consider that Cyrillic characters should never be found when searching Latin characters and vice versa, as they completely different things. However if use Cyrillic characters to simulate Latin small caps, you need this to happen, if you do not want searchability to be broken. This would lead to people find a lot of useless stuff if they search for a rare Latin-alphabet word that just so happens to correspond the faux small caps of some popular Cyrillic-alphabet word (and vice versa).

An exact search option cannot solve this problem, as this is reserved for other purposes in those alphabets.

In general, it is impossible to build a search (without an insane amount of options) that is not broken by using special characters to simulate styled Latin text.


1 You know that XKCD about the inevitable failure of unifying standards? Well, Unicode succeeded.
2 or whatever the empty operator is in the pertinent convention
3 I am aware that very few mathematical texts nowadays support this encoding or something compatible to it but the point is that some day they hopefully do. Your Unicode-abusing text may still be around and read then.
4 Unless you are localising for Macedonian or Serbian, in which you will get different but still undesirable result.

  • Comments are not for extended discussion; this conversation has been moved to chat. – Journeyman Geek Dec 30 '16 at 3:00
  • 1
    What about the third example? Why does blackletter text even exist in Unicode versus just some font? – posfan12 Dec 30 '16 at 23:20
  • 2
    @posfan12: It’s used as a distinct mathematical alphabet – like the script characters. (Note that the example numbering here has nothing to do with the bullet points in the question.) – Wrzlprmft Dec 30 '16 at 23:55
  • 3
    Your small-cpas example suggests a puzzle/competitoin: Find a valid sentence that becomes a different sentence when italicized ... (both interpreted by human latin-based readers, say) – Hagen von Eitzen Dec 31 '16 at 12:50
66

What can go wrong? Well, I see this:

enter image description here

in Firefox 50.1.0 on Windows 7.

The problem of missing glyphs, in this case on a mobile device, is further illustrated in an image given by user Chris Kent in a comment, which I have cropped and resized from the original:

enter image description here

And user oals kindly contributed another example:

enter image description here

  • 9
    Me? i.stack.imgur.com/lWRAa.png – oals Dec 27 '16 at 16:49
  • 7
    @Lilienthal I'm impressed by how many years you must've gone without checking for or installing updates. I mean, I'm still using something based off of Firefox 3.5 on some devices, but I don't exactly have a choice in the matter (device too good, but no newer usable browser is available) – mtraceur Dec 29 '16 at 6:38
  • 15
    @Lilienthal Some time in 2020, Firefox major version numbers may require 64-bit numbers. Shortly after that, the version number will take more space than the actual program. In an attempt to regain market share, Chrome version numbers will need entire planets to store them. – Andrew Morton Dec 29 '16 at 17:15
  • 2
    You folks have any idea how many known and active security vulnerabilities you're exposing yourself to by using such ancient browsers? – Zach Lipton Dec 31 '16 at 17:43
  • 3
    @Zach Lipton: Do you realize that's way down on the list of my priorities? #1 is having something I can actually use. Though maybe the Firefox people (and many others) have embraced a new paradigm: security through unusability. – jamesqf Dec 31 '16 at 18:36
28

I'm having an XY problem with this.

Y and X appear smaller than the rest of the text

Here, we see that Y and X appear smaller than the rest of the text. At certain zoom levels they appear to be the same size, but this appears to have exposed a problem with these particular glyphs in this particular font.

  • I can offer three possible explanations for what you are seeing: 1) Bad hinting. 2) A fallback font is used for everything but x and y. 3) Bad font design: Small caps were designed to be slightly larger than lowercase letters (which is a valid choice) and then used for phonetic and Cyrillic Unicode characters – which is not a good choice since, e.g., in Phonetic use, these characters have to harmonise with basic latin characters. — Either way: Points 1) and 3) are not due to Unicode abuse. Point 2) is already addressed in the other answers. – Wrzlprmft Dec 26 '16 at 19:57
  • 12
    @Wrzlprmft: By definition they are unicode abuse because unicode does not have small caps. What unicode does have are letters that look like small caps in the Phonetic alphabets blocks and the Latin Extended-D block. Specifically, neither the two Phonetic blocks nor the Latin Extended-D block contain the letter that looks like small caps X, therefore my guess is that the X comes from somewhere else, probably the Cyrillic block. The difference is caused by neither 1, 2 nor 3. It's caused by the letter belonging to another alphabet. – slebetman Dec 27 '16 at 4:08
  • @slebetman: The small-caps x is just a regular lowercase x (you can just copy the character from the question and inspect it yourself). And a lowercase x has to harmonise with the phonetic characters, because it is also used as a phonetic character. It also should also have the same height as Cyrillic lowercase characters, as you do not want single Cyrillic words to stand out from Latin text (and vice versa). – Wrzlprmft Dec 27 '16 at 8:39
  • 19
    Upvoted for the lol at "XY problem" :) – Andrew Morton Dec 27 '16 at 18:41
13

Using non-Latin characters that kinda-sorta look like Latin ones puts you in the company of spammers, pornmongers, and who-knows-what-they're-up-to dissemblers that want their text unsearchable, unindexable, and repudiatable. ("I never said it was safe!! I said it was sigma-alpha-integral-sign-epislon!!! Can't sue me!!!")

If you're comfy in that club, then go for it.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.