7

How can I grep for Unicode Character 'ZERO WIDTH SPACE' (U+200B) in Linux?

$ grep '%U200B' filename?
1
  • Which distro/shell/printf/grep combination are you using?
    – l0b0
    Sep 8 '14 at 14:11
8

First let's print one:

$ printf %b '\u200b' | uniname
character  byte       UTF-32   encoded as     glyph   name
        0          0  00200B   E2 80 8B               ZERO WIDTH SPACE

Now we should be able to use the same format to search for it (using Bash):

$ printf %b '\u200b' | grep -q "$(printf %b '\u200b')"
$ echo $?
0

The trick here is that printf %b treats the arguments as encoded characters, so you can use \x to print single-byte characters and \u* to print multi-byte characters.

To find it in a file, simply do this:

grep "$(printf %b '\u200b')" filename

* The POSIX specification isn't actually clear on how %b works. The printf page says "The %b conversion specification [...] has been added here as a portable way to process -escapes expanded in string operands as provided by the echo utility", and the echo page shows a single undocumented example of its use.

Test:

$ printf %b '\u200b' > test.txt
$ grep -q "$(printf %b '\u200b')" test.txt
$ echo $?
0
2
  • Thank you. I think I have to elaborate my question. I have a file in linux which has <200e> in one of the lines. NOw I need to catch it using grep in linux shell. i.e. without opening the file
    – Dinesh
    Sep 8 '14 at 11:58
  • grep "$(printf %b '\u200b')" filename is not working
    – Dinesh
    Sep 8 '14 at 12:10
4

The following, works fine. I created the file with BabelMap(google) and used it's save option.

Created file w/line nums 1-5 and on line 4 added the zero len space:

> hexdump testout.txt -C                 
00000000  31 0a 32 0a 32 0a 33 0a  34 20 e2 80 8b 0a 35 0a  |1.2.2.3.4 ....5.|
00000010

Note the utf8 encoding of the char 'e2808b' in the file.

This simple grep finds the correct line:

> grep $'\u200b' testout.txt  
4 ​
> grep $'\u200b' testout.txt|hexdump -C
00000000  34 20 e2 80 8b 0a                                 |4 ....|
00000006 

FWIW, my GREP_OPTIONS are set: "--color=auto -I -D skip -d skip", but I don't think any of them are relevant.

2
  • $'...' is not portable. It was going to be my answer before I saw that OP had just tagged the question "shell".
    – l0b0
    Sep 9 '14 at 7:03
  • Not just shell -- linux shell. I don't know of any shell associate with linux as much as bash.
    – Astara
    Sep 9 '14 at 12:44
1

You can also use Perl regex with GNU grep

grep --perl-regexp '\x{200B}' filename

In macos it is trickier, as the BSD grep that comes with it doesn't support multibyte. However GNU grep can be installed via Homebrew where it is made available as ggrep.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.