40

What's the best way to convert CRLF's to line feeds in files on Linux?

I've seen sed commands, but is there anything simpler?

2
  • 4
    Dupe: superuser.com/questions/38744/…. The link provided in the accepted answer covers the dos2unix, perl and vi options among others.
    – user4358
    Oct 7, 2009 at 9:13
  • 4
    This already has better answers though (so if one of these is to be closed, it should probably be that one)
    – Jonik
    Oct 7, 2009 at 12:02

13 Answers 13

47

Use this command:

fromdos yourtextfile

The other way around:

todos yourtextfile

These commands are found in the tofrodos package (on most recent distributions), which also provides the two wrappers unix2dos and dos2unix that mimic the old unix tools of the same name.

8
  • 3
    +1 Much more useful than the currently top-voted "Use dos2unix" answer.
    – Jonik
    Oct 7, 2009 at 10:00
  • 1
    Yeah, even I'm voting this one up. Mine was more of a drive-by suggestion. Oct 8, 2009 at 7:26
  • 2
    @SorinSbarnea: something like find . -name '*.txt' -print0 | xargs -null fromdos
    – bstpierre
    Jul 25, 2012 at 18:49
  • 2
    @Jonik what makes it "Much more useful"? Serious question Jan 31, 2015 at 22:43
  • 1
    This helped me, the other one didn't. But I really don't remember what my use case was 5+ years ago. :P Also see the original version of the other answer.
    – Jonik
    Feb 2, 2015 at 14:49
26

Use dos2unix.

dos2unix - DOS/MAC to UNIX text file format converter

dos2unix  [options] [-c convmode] [-o file ...] [-n infile outfile ...]

Options:
          [-hkqV] [--help] [--keepdate] [--quiet] [--version]
3
  • 2
    and unix2dos for the other way 'round. Oct 7, 2009 at 5:41
  • 1
    Consider elaborating on how to get this utility for your Linux system. At least on Ubuntu it's not installed by default (but by installing tofrodos package you get something very similar: packages.ubuntu.com/jaunty/tofrodos).
    – Jonik
    Oct 7, 2009 at 9:33
  • Nowadays it looks like unix2dos should be preferred over tofrodos, as tofrodos seems to be abandoned since 2013 and unix2dos is still maintained. Also unix2dos has very detailed man page, which is a plus, and reading of it leaves a feeling of a well-thought tool.
    – Yoory N.
    Sep 10, 2022 at 12:03
21

I prefer perl:

perl -lne 's/\r//g; print' winfile.txt > unixfile.txt

But that's well-suited to my uses, and it's very easy for me to remember. Not all systems have a dos2unix command, but most that I work on have a perl interpreter.

Another is recode, a powerful replacement for dos2unix and iconv; it's available in the "recode" package in Debian repositories:

recode ibmpc..lat1 winfile.txt   # dos2unix
recode lat1..ibmpc unixfile.txt  # unix2dos

For awk fans:

awk '{ sub("\r$", ""); print }' winfile.txt > unixfile.txt

...and sed:

sed 's/\r$//' winfile.txt > unixfile.txt

And now, only slightly-less-convoluted than deleting the CR's by hand in a hex editor, straight from one of our stackoverflow.com friends, useable with the beef interpreter (located on your friendly neighborhood Debian repository),

dos2unix in brainfuck!

,[[->+>+<<]>>>,[<-------------[+++++++++++++.>>>]<[>>----------[>+++++++++++++.-------------]<++++++++++>]<<<<[-]>>>[-<<<+>>>]]<[-]<[-]<]++++++++++.

big thanks to jk for wasting an hour of his life to write this!

10
  • 1
    (useless use of cat and) perl is as complicated as sed... thus you are not really answering the question but rather collecting reputation :)
    – akira
    Oct 7, 2009 at 6:28
  • 3
    "best way" is subjective. this works best for me (i'm tons more comfortable with perl than sed). i didn't promise it would work best for you. Oct 7, 2009 at 6:32
  • @akira: a question can have multiple valid answers. I use this method as well, occasionally, mostly in combination with other changes, so it is definitely a valid answer; but "use dos2unix" is definitely the more practical answer in most situations. So I think the ratings are fine. Oct 7, 2009 at 9:47
  • @akira: if you find it simpler, please post it as an answer and enlighten the rest of us. Oct 7, 2009 at 10:31
  • @~quack: that is the point: it is not simpler. thats the same for your perl answer. u2d or fromdos/todos are the right answers because they are simpler than any stuff expressed in any other programming language.
    – akira
    Oct 7, 2009 at 11:22
13

I think you can use tr, as well (though I have no funny format files on which to try):

tr -d '\r' < file1 > file2
3
  • 1
    Preferred: tr should be available on any Linux system, whereas most of the other answers (fromdos, dos2unix etc.) are not necessarily present. So unless you have sudo, they're less useful. (Also preferred to Voigt and JustJeff's answer because this one avoids the superfluous cat--not that I have anything against the one sitting on my lap.) Jan 4, 2022 at 23:42
  • @MikeMaxwell - some people are opposed to the uuoc, but it is rarely actually "useless". And it's never wrong. Sometimes it's more performant. I almost always use it for readability - which trumps any couple microseconds that might be saved by not including it :)
    – warren
    Jan 5, 2022 at 14:46
  • 1
    I don't disagree with you. It also has an advantage if I'm liable to forget the file I want to read from (assuming I'm typing in from the command line) by the time I write the other part (in this case, by the time I write "tr -d '\r'"). And at my age, forgetting is all too common... But that was the only thing I had to choose between this answer and Voight/JustJeff's, which were otherwise identical. Jan 6, 2022 at 22:57
10

I do this on Bash:

cat cr_stuffed.file | tr -d \r > no_more_crs.file
1
  • nice. i saw another mention of tr earlier today. it's not a program that gets mentioned very often is it? Oct 7, 2009 at 23:46
5

I prefer Vim and :set fileformat=unix. While not the fastest, it does give me a preview. It is especially useful in the case of a file with mixed endings.

1
  • 1
    Awesome! Worth mentioning, that :set fileformat=dos helps to convert to CRLF.
    – RAM237
    Jun 22, 2022 at 13:33
4

In vi or Vim:

:%s/^V^M//g
1
4

I found a very easy way… Open file with nano: ## nano file.txt

press Ctrl+O to save, but before pressing Enter press: Alt+D to toggle betwen DOS and Unix/Linux line-endings, or: Alt+M to toggle betwen Mac and Unix/Linux line-endings then press Enter to save and Ctrl+X to quit.

3
  • 1
    Could you edit your answer to clarify which toggle settings will replicate the behaviour requested by the OP?
    – Burgi
    May 1, 2016 at 1:52
  • 1
    The OP wants to toggle off DOS line endings, so Alt+d. Sometimes alt gets intercepted by the terminal program, so you can use esc+d instead. Aug 25, 2016 at 14:56
  • 1
    Lots of nano shortcuts also work with Shift pressed, which often prevents terminal interception, so 'Alt-Shift-D' works too.
    – mwfearnley
    Mar 2, 2017 at 10:45
2

If you want a GUI method, try the Kate text editor (other advanced text editors may be able to handle this too). Open the find / Replace dialog (Ctrl+R), and replace \r\n with \n. (NB: you'll need to choose "Regular expression" from the drop down and deselect "Selection only" from the options.)

EDIT: Or, if you simply want to convert to Unix format, then use the menu option Tools > End of Line > Unix.

2
  • There are text editors, such as jEdit, that can do these transformations automatically - you just tell it if you want Unix, Windows or Mac line separators.
    – Jonik
    Oct 7, 2009 at 10:24
  • 1
    Actually, KATE can do that too through the Tools > End of Line menu. Maybe I should have thought more laterally than answering the question exactly as it was worded - but if you know you specifically want to convert \r\n to \n then using search/replace is easier than remembering which OS uses which line ending. ;) Oct 10, 2009 at 23:22
2

CR LF to LF using awk:

awk -v RS='\r?\n' 1
command | awk -v RS='\r?\n' 1
awk -v RS='\r?\n' 1 filename

Usage example:

echo -e 'foo\nbar\r\nbaz' | awk -v RS='\r?\n' 1 | hexdump -C

Explanation:

-v RS='\r?\n' sets variable RS (input record separator) to \r?\n, meaning input is read line by line separated by LF (\n) which may (?) be preceded by CR (\r).

1 is the script awk executes. A script consists of condition { action }. In this case, 1 is the condition which evaluates to true. The action is omitted, so the default action is executed, which means print the current line (which could also be written as {print $0} or simply {print}).


LF to CR LF: You can set the variable ORS (output record separator) to modify the line ends of the output. Example:

echo -e 'foo\nbar\r\nbaz' | awk -v RS='\r?\n' -v ORS='\r\n' 1 | hexdump -C
1

Paste this into dos2unix.py Python script.

#!/usr/bin/env python
"""\
convert dos linefeeds (crlf) to unix (lf)
usage: dos2unix.py <input> <output>
"""
import sys

if len(sys.argv[1:]) != 2:
  sys.exit(__doc__)

content = ''
outsize = 0
with open(sys.argv[1], 'rb') as infile:
  content = infile.read()
with open(sys.argv[2], 'wb') as output:
  for line in content.splitlines():
    outsize += len(line) + 1
    output.write(line + '\n')

print("Done. Saved %s bytes." % (len(content)-outsize))

Should work on any platform with Python installed. Public domain.

1

Use Perl's generic \R in a regex. That way, you can convert files with any of CR, CRLF or already LF or a mix of them (yes, there are files which mix 2 different newline conventions!).

perl -i.bak -pe 's/\R/\n/g' $yourfile

(-i.bak tells perl to convert the file in-place, saving the original as ${yourfile}.bak)

More info on \R in this answer

0

I used this script for files I needed to emergency transfer files from a windows system to a unix system.

 find . -type f | xargs file | grep CRLF | cut -d: -f1 | xargs dos2unix

find . -type f

Finds all the files, recursively in the directory you're running the command from

xargs file

Pass it to the file program to get an analysis of the file.

grep CRLF

We only want the output of file that shows CRLF.

cut -d: -f1

Get the output up to to the color. discard the rest. We should only have a filename now

xargs dos2unix

Pass the filename to the program dos2unix using xargs.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .