Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have some files that are corrupted with this symbol:

^@

It's not part of the string; it's not searchable. How do I substitute this symbol with nothing, or how do I delete this symbol?

Here is an example line from one file:

^@F^@i^@l^@e^@n^@a^@m^@e^@ ^@ ^@ ^@ ^@ ^@ ^@ ^@ ^@ ^@ ^@:^@ ^@^M^@
share|improve this question
add comment

7 Answers

up vote 15 down vote accepted

You could try:

  • %s/<CTRL-2>//g (on PC)

  • %s/<CTRL-SHIFT-2>//g (on Mac)

where <CTRL-2> means first press down the CTRL on PC, keeping it as pressed down, hit 2, release CTRL.

and <CTRL-SHIFT-2> means first press down the control on Mac, keeping it as pressed down, press down shift on Mac, keeping it as pressed down, hit 2, release control and shift.

Finally, both of the two commands should result in %s/^@//g on screen. ^@ means a single character, not ^ followed by @, so you can't just type ^ and @ in a row in the above command.

This command removes all the ^@.

share|improve this answer
1  
this removes the nullbytes, thanks –  mrt181 Nov 26 '09 at 11:15
1  
Just stumbled upon this question/answer through a related link: This is actually a bad advice and will only work properly in very few cases. It's better to actually change the encoding rather than removing null bytes. If you remove the null bytes, you might still have other multibyte characters that show up as garbage. –  Mario Mar 7 at 9:57
    
@Mario could you tell us more about the encoding change? Is it something related to jrb's answer below? –  congliu Mar 8 at 13:11
    
See rpyzh's answer further down below. Shows loading the file using the proper encoding as well as saving it with a different one (although the answer could need some more explanation). Jrb's last note is enough if you just want to read it, but not if you want to have it saved without the null bytes using another encoding. –  Mario Mar 8 at 13:37
add comment

I don't think your files are corrupted. Your example line looks like it contains regular text with null bytes between each character. This suggests it's a text file that's been encoded in UTF-16 but the byte-order mark is missing from the start of the file. See http://en.wikipedia.org/wiki/Byte-order%5Fmark

Suppose I open Notepad, type the word 'filename', and save as Unicode Big-endian. A hex dump of this file looks like this:

fe ff 00 66 00 69 00 6c 00 65 00 6e 00 61 00 6d 00 65

If I open this file in Vim it looks fine - the 'fe ff' bytes tell Vim how the file is encoded. Now suppose I create a file containing the exact same sequence of bytes, but without the leading 'fe ff'. Vim inserts ^@ (or <00>, depending on your config), in place of the null bytes; Notepad inserts spaces.

So rather than remove the nulls, you should really be looking to get Vim to interpret the file correctly. You can get Vim to reload the file with the correct encoding with the command:

:e ++enc=utf16

share|improve this answer
    
Yes, the last command made vim interpret the file correctly but does not remove the nullbytes. –  mrt181 Nov 26 '09 at 11:14
5  
To remove them, choose another encoding and save the file again: :set fenc=utf-8 –  Scytale Aug 12 '10 at 9:19
add comment

This actually worked for me within vim:

:%s/\%x00//g
share|improve this answer
1  
this works with substitute(), but Ctl-VCtl-Shift-2 does not. –  dsummersl Jan 18 '13 at 15:26
    
Same problem for me, I couldn't get <Ctrl-V><Ctrl-2> (as well as the one with <Ctrl-Shift-2>) to work either, but this worked. –  Jeff Bridgman Jul 31 '13 at 16:12
add comment

That 'symbol' represents a NULL character, with ASCII value 000.

It's difficult to remove with vim, try

tr -d '\000' < file1 > file2
share|improve this answer
add comment

The accepted solution did not work for me. I made vim pipe the file through tr instead:

:%!tr -d '\000'

This would also work well with visual mode (just type :!tr -d '\000') or on a range of lines:

# Remove nulls from current line:
:.!tr -d '\000'

# Remove nulls from lines 3-5:
:3,5!tr -d '\000'
share|improve this answer
add comment

FWIW, it my case I had to use vim on cygwin to edit a text file created on a mac. The accepted solution didn't work for me, but was close. According to Vim wiki page about working with Unicode, there is a difference between Big Endian and Little Endian versions of the BOM byte. So, I had to explicitly tell vim to use a Little Endian version of BOM encoding.

Only after picking the right encoding I converted the file format (line endings) to dos so I could edit the file in Windows editor. Trying to set reset the file format before specifying the encoding gave me grief. Here is the full list of commands I used:

:e ++enc=utf16le
:w!
:e ++ff=mac
:setlocal ff=dos
:wq
share|improve this answer
    
Precious info. In my case it was the endianness of the BOM byte. –  Andre Albuquerque 14 hours ago
add comment

In addition to @jrb's answer, in Vim, the character encoding of the file is detected based on the fileencodings option. (note the 's' at end of fileencodings)

I.e. on Windows, the default value for the fileencodings option is ucs-bom, which means:

check if BOM exists at the beginning of the file.

If BOM exists, then 'read the character encoding of the file out of BOM'.

If BOM doesn't exist (and in this case that would also mean that all character encodings specified in the fileencodings option failed to match), then read the file with the character encoding specified in the encoding option. The default character encoding for the encoding option is: latin1. Now, because latin1 is the one byte length character encoding, all bytes in the file are valid latin1 characters (even the Nul character ^@ that you're seeing*).

*- actually, ^@ is the newline character in the Vim's buffer text, not the Nul character.

The proper way to read the file is to specify the character encoding manually as UTF-16 (as it looks like UTF-16 is the proper char encoding in this case).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.