Have a txt file keep it's line breaks after being send over FTP via Curl

Question

I'm using Curl to send a file over FTP. It is just a generated text file from a screen hardcopy command. When I cat the file, it displays correctly, it looks like this:

System up for    14844
 1, 0000, R D S, 0, 0, x, 0.00
 2, 0000, R D S, 0, 0, x, 0.00
 3, 0000, R D S, 0, 0, x, 0.00
 4, 0000, R D S, 0, 0, x, 0.00

However when I look at the transferred file, it is all on 1 line. Any way to fix this? Is there a command line option? This is my curl command:

curl -T ~/hardlog.log ftp://10.0.0.7 --user user:secret

Thanks

Answer 1

As it was already mentioned this is a classical symptom of incompatible line endings. In this case the source system is Unix-like (Linux) and the destination is Windows-like. In Windows the Unix newlines are not recognized so the text file appears as a single long line. In such cases you should do a newline conversion.

Though the FTP protocol has the newline conversion built-in Curl does not support it even if forcing ASCII transfer using -Q '+TYPE A' option.

The most convenient option would be to do the conversion on Linux before the transfer using a script:

#!/bin/sh
SRCFILE=~/hardlog.log
DSTLOGIN=user:secret
DSTURL=ftp://10.0.0.7/              # the URL must end by a slash

unix2dos <"$SRCFILE" | curl -T- "$DSTURL$(basename "$SRCFILE")" -u"$DSTLOGIN"

If the unix2dos utility is not available on your system, you can use sed 's/$/\r/' or awk '{printf("%s\r\n",$0)}' instead. See Alternatives to unix2dos conversion.

Another option is to use a client supporting ASCII transfers like LFTP. Example:

lftp -e "put -a $HOME/hardlog.log ; exit" -u user,secret 10.0.0.7

Note: On Windows there are text editors capable of working with Unix newlines like Wordpad or some third-party ones: Notepad++, Notepad2. The last two editors can be also used for newline conversions.

Answer 2

Here's a crude but effective little loop that adds a newline to the end of each line of text in a file.

    today=`date +%m%d` # file name filter
    for file in *${today}*
    do
      > ${file}.tmp
      while read readline
      do
        echo -e "$readline\n">> ${file}.tmp
      done <${file}
      cp ${file}.tmp ${file}
      rm ${file}.tmp
    done