Tell me more ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.
up vote 0 down vote favorite

I want to remove the weird / special characters from the bunch of text files. Here i used the following command to remove the weird characters but it gives me the error.

Command :

1) grep -lri -e ââ¬Å temp.txt | xargs sed -i ' s_ââ¬Å_"_g'

2) grep -lri -e ââ¬Å temp.txt | xargs sed -i ' s/ââ¬Å/"/g'

3) grep -lri -e ââ¬Å temp.txt | xargs perl -piew ' s_ââ¬Å_"_g'

4) grep -lri -e ââ¬Å temp.txt | xargs perl -piew ' s/ââ¬Å/"/g'

Here I want to replace ââ¬Å this character with " quote.

These are the some Weird characters that I want to remove from the text files :

– , — , ; , : , ! , ¡ , ¿ , · , ‚ , ‹ , › , » , @ , / , [ , ]\ , \^ , + , \ , \$ , \s , & , # , % , †, ‡ , ` , ´ , ¯ , ˘ , ¨ , § , ¶ , © , ® , â„ , ° , º , ∂ , ∆ , ∠, ∑ , ± , = , ≠, ¬ , \ , , ~ , ∫ , € , ª , à , â , Ã… , ä , Æ , Ç , è , ë , ï¬ , fl , Æ’ , í , ì , î , ï , ó , ò , ô , ö , õ , ø , Å“ , ß and so on.

Please help me on this & Also suggest the command to remove the bunch of weird characters.

share|improve this question
1  
What error are you getting? – slhck♦ Feb 12 at 13:39
2  
Please don't cross-post. Same question on stackoverflow. – Thor Feb 12 at 13:43
Here I want to replace ââ¬Å this character with " quote. likewise 1) ú with u (lower case letter u) 2) õ with o (lower case letter o) 3) í‘ with N (upper case letter N) 4) í© with e (lower case letter e) 5) íº with u (lower case letter u) – – Amit Durge Feb 12 at 13:44
@Thor I've already flagged this for closing on Stack Overflow as it's not really a… programming problem. Thanks for bringing it to our attention. – slhck♦ Feb 12 at 14:29
1  
Your question is really unclear. Please post the actual file you want to work on. As @slhck suggested, this is likely an encoding problem and all you need to do is set the right encoding. – terdon Feb 12 at 15:16

1 Answer

up vote 1 down vote

The perl command can easily operate on a set of files. There's no need to grep the files to search them for a certain string, then pass those file names via a pipe to xargs and then to perl. This will definitely break even in the simple case that any of the files contain a spaces in their name.

Why not do the following, for example to remove invalid characters?

perl -pi -e 's/[—Ç…]//g' *.txt

Or, to remove non-ASCII characters:

perl -pi -e 's/[^[:ascii:]]//g' *.txt

Or, to replace a certain character with another:

perl -pi -e 's/ú/u/g' *.txt
share|improve this answer
Here I want to replace ââ¬Å this character with " quote. likewise 1) ú with u (lower case letter u) 2) õ with o (lower case letter o) 3) í‘ with N (upper case letter N) 4) í© with e (lower case letter e) 5) íº with u (lower case letter u) So what is the command? – Amit Durge Feb 12 at 13:46
1  
No need to copy-paste the same comment. To replace ââ¬Å with a quote, use s/ââ¬Å/"/g, et cetera. See: Perl Text Patterns for Search and Replace — but actually, I'm pretty sure your problem is just that the file you have is improperly encoded, right? Wouldn't it be better to fix the overall character encoding? Where did you get the file from? – slhck♦ Feb 12 at 13:47
perl -pi -e 's/ú|‚/u/g' *.txt as like this i want to add multiple weird chjaracters to replace with some text – Amit Durge Feb 12 at 13:51
You can put multiple expression in the Perl command: perl -pi -e 's/foo/bar/g; s/yak/yaz/g; s/…/…/g; …' *.txt. No way around that, really, unless you want to write a script where you could put the replacment strings into an associative array of some sort. But that'd probably be overkill here. – slhck♦ Feb 12 at 14:02
ok Also How can I use the regular expression? Please give me the example. – Amit Durge Feb 12 at 14:05
show 3 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.