92

I use Linux. There is a pesky ^M (Windows cariage return) somewhere hidden in thousands of configuration files, and I have to find it, because it makes the server fail.

How do I find ^M among a directories hierarchy full of configuration files?

I think I can not enter ^M on the bash command line. But I have it in a text file that I called m.txt

3
  • Related: Remove carriage return in Unix. Commented May 15, 2014 at 21:17
  • windows would be ^M^J
    – barlop
    Commented Sep 20, 2015 at 1:43
  • 3
    "I can not enter ^M on the bash command line". Yes you can. Try control-V Control-M
    – Hennes
    Commented Aug 23, 2016 at 11:06

9 Answers 9

117
grep -r $'\r' *

Use -r for recursive search and $'' for c-style escape in Bash.

Moreover, if you are sure it's a text file, then it should be safe to run

tr -d $'\r' < filename

to remove all \r in a file.

If you are using GNU sed, -i will perform an in-place edit, so you won't need to write the file back:

sed $'s/\r//' -i filename
12
  • 11
    @Nicolas: You can enter a ^M at the command line by pressing ^V^M, but it's better to use $'\r'. Commented Oct 1, 2010 at 5:54
  • Great, it works! Thanks for the ^V^M trick too :-) Commented Oct 1, 2010 at 5:56
  • 5
    Under Cygwin, -U is needed to make this work. And -n will tell you the line number: grep -r -U -n -e $'\r' Commented Jan 3, 2013 at 16:16
  • 5
    Add an -l to the grep command to just view the filenames. Else you might be bombarded with matching lines.
    – SineSwiper
    Commented Mar 25, 2014 at 20:28
  • 1
    @uprego not sure if you understand them now, but fyi and other's, search $' read the first hit in manpage bash(1), basically, you can see it as if you were writing C literal string. As for the command < filename, the use of < or > is called redirection, this is first time I have seen anyone called it greater expression. Search REDIRECTION in bash(1).
    – livibetter
    Commented Sep 23, 2015 at 2:03
15

When I tried, I could tell it was sort-of working, but the lines were printing blank. Add in the option:

--color=never

If you get this issue, I think it's the escape characters for color highlighting interfering with the \r character.

1
  • Thanks! I did not know why I was getting blank lines.
    – KolaB
    Commented Sep 7, 2020 at 16:07
4

If your server does not have a bash shell, an alternative is to use the -f option on grep, in combination with a prepared file containing \r.

To create the file:

$ echo -ne '\r' > /tmp/cr                    --or--                   $ printf '\r' > /tmp/cr

$ od -c /tmp/cr
0000000  \r
0000001

To actually do the search

$ grep -f /tmp/cr *.html *.php *.asp *.whatever

or you can be a little lazy and just type *,

$ grep -f /tmp/cr *

The -f filename option on grep is used to specify a file that contains patterns to match, one per line. In this case there's only one pattern.

1
  • even works on mingw/Windows
    – jifb
    Commented Jun 4 at 14:06
3

If I understand your question correctly, what you really want is to normalize all line-endings to the Unix LF (\x0a) standard. That is not the same as just blindly removing CRs (\x0d).

If you happen to have some Mac files around which use just CR for newlines, you will destroy those files. (Yes, Macs are supposed to use LF since almost 20 years, but there are still (in 2019) many Mac apps which use just CR).

You could use Perl's \R linebreak escape to replace any sort of newline with \n.

perl -i.bak -pe 's/\R/\n/g' $your_file

This would replace in-place any sort of linebreak with \n in $your_file, keeping a backup of the original file in ${your_file}.bak.

3
  • dos2unix is the proper command.
    – pbies
    Commented Apr 28, 2022 at 9:16
  • @pbies: Unfortunately, no. dos2unix doesn't seem to work in cases of mixed line endings (just re-tested on files with mixed CRLF and CR-only). That was the original reason why I started to use Perl instead. Besides, dos2unix needs to be installed, while Perl is already available on any decent Unix system.
    – mivk
    Commented Apr 29, 2022 at 15:12
  • Have you tried binary mode?
    – pbies
    Commented Apr 29, 2022 at 19:11
1

To use grep on end-of-line characters, I guess you have to tell grep the file is binary.

  • -l (letter L) is for printing only the filename
  • -P is for perl regexp (so \x0d is transformed to \r or ^M)
grep -l --binary -P '\x0d' *
0

If you are on a Mac and use homebrew, you can do:

brew install tofrodos
fromdos file.txt

to remove all the Windows carriage returns from file.txt

To switch back to Windows carriage returns,

todos file.txt
1
  • to search in a folder and clean all files coming from dos, run this command : find . -type f -name "*.java" | xargs fromdos
    – Taiko
    Commented May 19, 2014 at 3:38
0

In regular expression style, various newlines:

Windows (CR LF)
\r\n

Unix (LF)
\n

Since the \r\n sequence is fairly unique, I think you should be able to search for it that way?

To make things worse Macs used to have just '\r' in place of newline. I cannot verify this, but I don't think MacOSX generations does that any more.

Older Macs (CR)
\r

2
  • In the directory that contains m.txt, grep "\r\n" * gives no result. No result either for egrep -e "\r\n" * nor grep -E "\r\n" * Commented Oct 1, 2010 at 5:54
  • @nicolas ah, I misunderstood.. you meant CR only \r my bad. A full windows newline is indeed \r\n or CRLF Commented Oct 1, 2010 at 5:58
0

Following up on previous answers, the tr method is good:

533$ if [[ -n "`tr -cd "\r" <~/.bashrc`" ]]; then echo "DOS"; else echo "UNIX"; fi
UNIX

534$ if [[ -n "`tr -cd "\r" <dosfile.txt`" ]]; then echo "DOS"; else echo "UNIX"; fi
DOS
0

Other answers require Bash, this one should not:

grep -a -r "$(printf '\r')"

Explanation

  • printf '\r' prints a literal carriage return character
  • The wrapping "$(..)" puts the CR into an argument to the grep command.
  • -a tells grep to act on binary files, too, so that it actually prints matching lines even if the file is considered binary.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .