grep to find files that contain ^M (Windows carriage return)

Question

I use Linux. There is a pesky ^M (Windows cariage return) somewhere hidden in thousands of configuration files, and I have to find it, because it makes the server fail.

How do I find ^M among a directories hierarchy full of configuration files?

I think I can not enter ^M on the bash command line. But I have it in a text file that I called m.txt

Answer 1

grep -r $'\r' *

Use -r for recursive search and $'' for c-style escape in Bash.

Moreover, if you are sure it's a text file, then it should be safe to run

tr -d $'\r' < filename

to remove all \r in a file.

If you are using GNU sed, -i will perform an in-place edit, so you won't need to write the file back:

sed $'s/\r//' -i filename

Answer 2

When I tried, I could tell it was sort-of working, but the lines were printing blank. Add in the option:

--color=never

If you get this issue, I think it's the escape characters for color highlighting interfering with the \r character.

Answer 3

If your server does not have a bash shell, an alternative is to use the -f option on grep, in combination with a prepared file containing \r.

To create the file:

$ echo -ne '\r' > /tmp/cr                    --or--                   $ printf '\r' > /tmp/cr

$ od -c /tmp/cr
0000000  \r
0000001

To actually do the search

$ grep -f /tmp/cr *.html *.php *.asp *.whatever

or you can be a little lazy and just type *,

$ grep -f /tmp/cr *

The -f filename option on grep is used to specify a file that contains patterns to match, one per line. In this case there's only one pattern.

Answer 4

If I understand your question correctly, what you really want is to normalize all line-endings to the Unix LF (\x0a) standard. That is not the same as just blindly removing CRs (\x0d).

If you happen to have some Mac files around which use just CR for newlines, you will destroy those files. (Yes, Macs are supposed to use LF since almost 20 years, but there are still (in 2019) many Mac apps which use just CR).

You could use Perl's \R linebreak escape to replace any sort of newline with \n.

perl -i.bak -pe 's/\R/\n/g' $your_file

This would replace in-place any sort of linebreak with \n in $your_file, keeping a backup of the original file in ${your_file}.bak.

Answer 5

To use grep on end-of-line characters, I guess you have to tell grep the file is binary.

• -l (letter L) is for printing only the filename
• -P is for perl regexp (so \x0d is transformed to \r or ^M)

grep -l --binary -P '\x0d' *

Answer 6

If you are on a Mac and use homebrew, you can do:

brew install tofrodos
fromdos file.txt

to remove all the Windows carriage returns from file.txt

To switch back to Windows carriage returns,

todos file.txt

Answer 7

In regular expression style, various newlines:

Windows (CR LF)
\r\n

Unix (LF)
\n

Since the \r\n sequence is fairly unique, I think you should be able to search for it that way?

To make things worse Macs used to have just '\r' in place of newline. I cannot verify this, but I don't think MacOSX generations does that any more.

Older Macs (CR)
\r

Answer 8

Following up on previous answers, the tr method is good:

533$ if [[ -n "`tr -cd "\r" <~/.bashrc`" ]]; then echo "DOS"; else echo "UNIX"; fi
UNIX

534$ if [[ -n "`tr -cd "\r" <dosfile.txt`" ]]; then echo "DOS"; else echo "UNIX"; fi
DOS

Answer 9

Other answers require Bash, this one should not:

grep -a -r "$(printf '\r')"

Explanation

• printf '\r' prints a literal carriage return character
• The wrapping "$(..)" puts the CR into an argument to the grep command.
• -a tells grep to act on binary files, too, so that it actually prints matching lines even if the file is considered binary.