There is no option to ignore line endings for
As you have found out,
rsync considers files with differing line endings to be different. This is inconvenient in your situation because the files are visually/semantically the same.
rsync decides what to sync in a file based on checksums on blocks. There is a good overview on this:
(T)he old version of the file is split into blocks of, e.g. 1024 or 2048 bytes, and a checksum is calculated for each block.
The new file is then searched byte for byte for blocks with checksums matching those in the old version. Here is a diagram illustrating this process:
Repeating these operations on the new version of the file you will iterate through the file byte for byte. During this iteration you will find two types of data in the file:
- Blocks of data that matches blocks in the old file.
- Sequences of bytes that is not part of a matching block.
From RSync - Detecting File Differences by Jakob Jenkov.
If you are interested, the next section is on the checksums used. However, the main point of the checksum is that it works on bytes, and your files have different bytes because of the line endings. As such,
rsync is correctly detecting that they are different and so is transferring them.
Avoid transferring files with differing line endings by sanitising pre-transfer
The best say to do this is to make sure all your files have consistent line endings, or sanitise them, as Kamil suggested in the comments.
How you do this is up to you. You might decide that you will make the change when the files are generated, edited or updated. Or you could do it as a pre-transfer step.
Apply the sanitisation to only the files needed
If you do sanitise, make sure you do not apply it blindly, as Kamil further warns:
One shouldn't use any conversion tool blindly on all files. Even if the tool tries to guess if a file is text or binary, it's only heuristics. CRLF may appear inside a binary file; blocks that look like text may appear as well. Modifying a binary file by removing some bytes will most probably corrupt it.
For example, if you have files in the two directories that you know are text files which need to be sanitised, apply the sanitisation step to that subset only.
Which tool should I use to change the endings?
A full solution is beyond the scope of this answer. There are several suggestions over on this SO QA, including
You can use
tr to convert from DOS to Unix; however, you can only do this safely if CR appears in your file only as the first byte of a CRLF byte pair. This is usually the case. You then use:
tr -d '\015' <DOS-file >UNIX-file
However, if you're going to have to do this very often (more than once, roughly speaking), it is far more sensible to install the conversion programs (e.g.
unix2dos, or perhaps
utod) and use them.
from Jonathan Leffler's answer.
Do bear in mind the above warning when using any of those tools, however.