Rsync is an open source command line tool for Unix and Windows systems that provides fast incremental file transfer to copy files and synchronize directories locally or between remote hosts. It is widely used for mirroring and backup.

learn more… | top users | synonyms

5
votes
2answers
3k views

Properly escaping forward slash in bash script for usage with sed

I'm trying to determine the size of the files that would be newly copied when syncing two folders by running rsync in dry mode and then summing up the sizes of the files listed in the output of rsync. ...
0
votes
0answers
1k views

Can't git push to Bitbucket.org — with ssh keys

I'm running doku-wiki on may (Ubuntu) laptop for some note taking. So, to keep backup, I was using Bitbucket and periodically pushed my wiki files there and everything worked smoothly. (a) Some time ...
0
votes
1answer
47 views

rsync per-site configuration file?

I know how to configure a per-site entry for ssh, but is there any kind of a client configuration for rsync that allows per-site configuration options and aliases or similar shortcuts like the ...
2
votes
1answer
558 views

rsync to windows (cygwin)

We have a windows file storage (don't ask) and now I want to rsync with the machine from Windows, Mac and Linux. So I installed freeSSHd (login shell is set to C:/cygwin64/bin/sh.exe), set up ...
0
votes
0answers
57 views

rsync stops at client version

I try to get rsync working between a freeSSHd Windows Server and Windows, Mac and Linux Clients. Current if I execute on Linux: rsync --progress -vvvvzrlt -e ssh myusername@10.254.0.108:/c/mypath ...
1
vote
2answers
152 views

Could server's support of RSync speed up backup process?

I have Synology 210j NAS, which supports rsync. Currently this feature is disabled and I use the following code to backup data there from my Mac: mkdir /Volumes/BackupTemp mount -t smbfs ...
2
votes
1answer
440 views

Why does rsync copy files after copying files manually?

I copied many files in many folders (~54GB) from an ext4 location to another with cp ~/1 ~/2 -d -r -v -i in bash. I then wanted to check that all files were copied correctly, so I ran rsync --delete ...
1
vote
0answers
227 views

File transfer tools for linux with support of zero-copy transfer

Are there small and simple file transfer tools for Linux, which can use "zero-copy" style of transferring files from HDD (SSD) to tcp socket, and back? I want to often send huge amount of ...
1
vote
1answer
83 views

Setting up Debian server for remote backups

I am running a Debian Wheezy server at home, and am offering an organization to let them do rsync backups to their own folder on my system. I created the unique user for them, and got a strong ...
2
votes
1answer
848 views

Rsync out of memory

I'm using a VPS (running Debian) with 512mb RAM. I'm trying to copy some file locally on the server with rsync, but it fails fast with this message: rsync: writefd_unbuffered failed to write 5 bytes ...
17
votes
4answers
6k views

Rsync friendly gzip

I must not be the only one - I'm rsyncing .tar.gz files and notice that every time the full file gets rsynced rather than the differences. Reading into it it seems back in 1999 someone created an ...
4
votes
4answers
8k views

Does rsync --inplace write to the entire file, or just to the parts that need to be updated? (for btrfs+rsync backups)

I was reading several guides how combine btrfs snapshots with rsync to make an efficient backup solution with history. However it all depends on if rsync --inplace modifies only those portions of ...
1
vote
1answer
719 views

How to speed up data transfer between Portable HD and NAS (QNAP)?

I am trying to transfer data from a Western Digital USB2 Drive to a QNAP TS-212E. I am transfering the data between the portable HD and the NAS using rsync (see following command). rsync -v -r -h ...
0
votes
1answer
344 views

ntfs disk dying or..? rsync and mkdir failing, strange behavior

System root@host:~# uname -a Linux host 3.2.0-4-686-pae #1 SMP Debian 3.2.57-3+deb7u1 i686 GNU/Linux Mounts root@host:~# mount | tail -2 /dev/sda1 on /mnt/sda1 type ntfs ...
0
votes
0answers
95 views

Win7 -> Linux cannot seem to copy large file without errors

I have a 70 GB file I am trying to copy from a Windows 7 machine to a Linux Mint 16 machine over a network, and I don't seem to be able to get the MD5 sums to match afterwards. I've so far tried two ...
0
votes
1answer
25 views

rsync exclude config.js but include lib/project/config.js

I have a folder like this / config.js /lib /project config.js In the root directory I execute rsync rsync -azvr --exclude config.js --include lib/project/config.js . dest The ...
1
vote
0answers
482 views

rsync failed: No Such file or directory

I'm using rsync in Windows ( through cygwin64 ) to backup some data to an samba share. I've made this bat file that would take care of everything: net use S: \\10.1.1.23\marc password /user:marc ...
0
votes
1answer
228 views

Make rsync update existing files first, create new ones later?

I have a source directory which I'm periodically syncing to a target using rsync: rsync --bwlimit=1600 --recursive --times --omit-dir-times --copy-links --modify-window=30 --inplace ...
3
votes
1answer
163 views

How to control permissions on external USB hard disk

I'm using RHEL 6.5, andhave an auto-mounted external USB drive. Any and all files created in the mounted filesystem are given permission 700 for dirs and 777 for files. The chmod command does not ...
0
votes
0answers
12 views

Using an SD card for regular backups? [duplicate]

I recently updated to a 128 GB solid state drive, and I've gotta say, I love the boot times and app load times, however I'm not keen on the increased possibility of spontaneous failure. I figure I've ...
0
votes
1answer
460 views

How to make rsync completely ignore permission from source

I'm trying to use rsync to copy from an Android source where the sdcard mount has rather nonsensical permissions (---rwxr-x) and rsync fails because it creates the destination directories with the ...
3
votes
3answers
1k views

Is there a way to use plink with rsync on Windows?

I use rsync to sync files between a local directory and a remote host. I also use plink to automate my SSH passphase entry. What I can't figure out is how to use rsync with plink. This is what I ...
3
votes
2answers
941 views

How do I compare two folders recursively and generate a list of files and folders that are different?

tl;dr and an example I'm looking for a way to compare two folders recursively and output the relative paths all files (and folders) that are different (by size or by timestamp, à la rsync). For ...
0
votes
1answer
28 views

Below what network speed should I be using compression for an rsync transfer?

I'm transferring some files from a remote server. The speeds for the transfer are abysmal (~1 MBps), and have got me thinking about optimizing the transfer for speed. At what network speed does it ...
0
votes
2answers
37 views

rsync from two servers to a common location

I have two servers and I want to rsync files from both servers to a single location. Server 1 has files A and B Server 2 has files A and B After rsync server 3 has file A and B When a user ...
0
votes
1answer
35 views

without ssh key setup, can have write a non-interactive script to rsync/scp/etc withut me having to input the password everytime?

Is there some switch/option for ssh or rsync (something like -u username -p passowrd) that would allow me to pass in the password?
0
votes
2answers
44 views

Rsync folders automatization

I've got a folder on the remote server, to which I'm connecting via SSH. When I need to synchronise folders I'm using manually access via scp. (I'm synchronising my folder with the folder on the ...
1
vote
1answer
69 views

What does the output of rsync look like when it completes successfully?

I used rsync to back up my main drive to an external drive last night. At some point in the night the external drive was unplugged. This is the tail of the output of rsync -avRP ... ...
1
vote
1answer
62 views

How to keep two directories on two unconnected machines in sync via emailing of delta-archives

My situation is that I have two directories on two different computers (one Windows machine, OSX machine) and I need to keep the two directories on the two machines in sync (= mirrored), so that they ...
1
vote
2answers
5k views

What is more efficient / faster, rsync compression or ssh compression?

I currently rsync files: (i) locally and (ii) remotely over WAN (connection c. 8mbps down / 1.5mbps up). I am considering adding to rsync: -z to compress files. Or alternatively adding to ...
2
votes
1answer
1k views

Rsync very slow

I have just setup a Debian server for backup of Windows PCs. I am currently testing various methods including: rsync (using Deltacopy on Windows), SMB via Samba and scp. Currently rsync (without SSH ...
0
votes
1answer
532 views

Exact copy of files from NTFS to another NTFS using Linux

I try to copy all files from an NTFS to an external drive and want to preserve all permissions and attributes. The external drive has already an NTFS and contains other files. I've considered using ...
16
votes
3answers
1k views

Moving 2TB (10 mil files + dirs), what's my bottleneck?

Background I ran out of space on /home/data and need to transfer /home/data/repo to /home/data2. /home/data/repo contains 1M dirs, each of which contain 11 dirs and 10 files. It totals 2TB. ...
1
vote
1answer
188 views

How to Find the Bottleneck when Transferring Files over WAN

This question may be a little too broad, but I'm looking for some methodology to identify a transfer speed bottleneck. I'm using rsync to transfer files between 2 NAS devices (DNS-323 to be specific) ...
0
votes
0answers
34 views

web interface for server-to-server file transfer [duplicate]

I'm looking for a way to transfer files between two servers. I currently ssh into one server and use rsync inside a screen session to initiate a transfer from the other host. This works, but I'm ...
0
votes
1answer
216 views

Transparent failover for NFS home directory (with LDAP users)

Background: In our work environment, we have LDAP users together with NFS homes. So you sign on LDAP server, autofs mounts the homes, etc. Pretty neat so far. However, from time to time, the NFS or ...
1
vote
0answers
109 views

ssh proxycommand equivalent on server

Lets say we have three machines me, proxy, server I would like to have an ssh connection to server, so normally I would do me$ ssh proxy proxy$ ssh server server$ # and here we are ssh proxy ...
0
votes
2answers
2k views

Can't rsync locally on synology NAS

I'm logged in to my Synology NAS and although I can rsync from my servers to the NAS and vice versa, I can't use rsync where the source and destination are both on the DiskStation. midnight my_user ...
1
vote
3answers
3k views

scp certain files only

I am trying to copy changed files to godaddy via ssh. Godaddy is terrible and so they of course do not allow rsync. I can use scp, but I really only want to copy changed files. I don't think this is ...
2
votes
3answers
537 views

How can this backup strategy work?

I am trying to script a very simple backup strategy. Here is the general idea. Daily - Backup the entire filesystem using rsync, overwriting the previous day's backup. Weekly - Once a week copy the ...
0
votes
2answers
2k views

rsync slow copy from one disk to another

I got two drives of the same model which are connected using SATA. When benchmarking read speeds using hdparm -t /dev/sdX, speeds of about 160MB/s are returned. When benchmarking write-speed using dd ...
3
votes
2answers
289 views

restrict rsync (or bash script) to a directory

After a bad accident with rsync deleting wrong files due to wrong destination parameter, I want to restrict rsync (or the bash script running it) to be able to delete/modify files only under a certain ...
9
votes
2answers
2k views

How to copy Time Machine volume to another disk using rsync while preserving ACLs?

Recently I had to change my Mac's filesystem from case-preserving to being fully case-sensitive. I did that by using CCC and copying data to another disk, then returning back to newly formatted main ...
0
votes
1answer
56 views

rsync don t work as expected

I have a script which download and compress a folder of a remote server (example: /home). I m trying to make the "restore" script, but I have a strange behavior. restore.sh: ($1 is IP adress, $2 is ...
0
votes
1answer
158 views

Error message using scp over ssh

I'm not sure what's going on but as of late I can't seem to get any files copied to local from my remote server. I tried rsync from local and get the following error message: mut@mylocalcomputer:~$ ...
0
votes
1answer
41 views

rsync not skipping untouched files

I transferred about 300GB of files with scp from one server to another. Sadly I forgot the -r option, so part of the files that were in subdirectories were not copied. I verified that the files that ...
1
vote
1answer
882 views

How to copy directories into symbolic links using rsync?

I have directories and files locally and I want to copy them over to another box using rsync . The problem is that most of the data is already on the remove box, but not on the target disk. E.g., I ...
0
votes
0answers
85 views

rsync won't recognize my /user folder?

On OSX, 10.9.1. I'm trying to use rsync to sync the contents of local folder: (/Users/username/00_WORK/FolderX) with (/Volumes/WORK_BACKUP/01_Current_Work/FolderX) Obviously the destination is ...
3
votes
1answer
1k views

What does Rsync's output tell here about to-chk?

I was using rsync using --progress option , So the file transfer completed and I got following output receiving incremental file list Makefile 9,935 100% 9.47MB/s 0:00:00 (xfr#1, ...
2
votes
1answer
2k views

Rsync: after copying as root, change owner to the web user

I seem to come up against this issue occasionally: serverA assets: owned by user: apache (no 'ftp' user present) serverB assets: owned by user: ftp (apache user present, but not used) I connect ...