Rsync is an open source command line tool for Unix and Windows systems that provides fast incremental file transfer to copy files and synchronize directories locally or between remote hosts. It is widely used for mirroring and backup.

learn more… | top users | synonyms

32
votes
17answers
44k views

Best way to copy millions of files between 2 servers

I have roughly around 5 million small (5-30k) files in a single directory that I would like to copy to another machine on the same gigabit network. I tried using rsync, but it would slow down to a ...
1
vote
1answer
2k views

Rsync and Windows 7

Can someone give me any tips on setting up some sort of Rsync server/client on Windows 7 to run rsync between both my web hosting server, and a backup server that I have running Ubuntu? I've tried ...
3
votes
3answers
3k views

Any faster scp or rsync?

How can I transfer a large file quickly, when I don't really require a secure connection? Often I would like to copy a large file from local machine to remote machine. The file does not already ...
1
vote
2answers
381 views

Copy ubuntu from one PC to another

I have a very weird scenerio here, I have installed some softwares on my Ubuntu machine, Now I want to move everything (including windows)to another Machine B, notice Machine has no access to Internet ...
4
votes
1answer
2k views

rsync between two local directories

I am trying to do rsync between two directories I have in two filesystems connected to the same computer. I want to do it without deleting the extra files in b. I though a simple rsync -a a/dir b/dir ...
9
votes
3answers
10k views

rsync - does it create a temp file during transfer?

As far as I can see rsync doesn't create the file in the target directory until it is complete. This must mean that it creates the file in a temp directory somewhere and copies the file into the ...
3
votes
2answers
3k views

Rsync to multiple directories with one command?

I want to rsync my working directory to two other directories. One would destination would be for backup, another for testing it. I tried this: rsync -av --exclude='.git' source dest1,dest2 But ...
53
votes
16answers
70k views

How to synchronize the home folder between multiple computers?

I have three computers at home, and would like to have the /home/ folder tree synchronized between the three. Any files/folders (except those hidden) that are modified/added/removed in one of the ...
0
votes
1answer
2k views

How to remove/delete a vanished file?

[macek ~]$ rysnc -av /src /dest building file list ... file has vanished: "/src/file.ext" This will not generate error: [macek /src]$ rm file.ext Also strange: [macek /src]$ fi Tab The path will ...
1
vote
0answers
56 views

Is cp -au / cp --archive --update contradictory?

I want to update a directory on a backup disk with new content added to the source, and of course rsync is too slow. cp -a does the trick but is a bulldozer; cp -u should leave existing files alone. ...
8
votes
3answers
6k views

Why cwRsync doesn't work on Windows?

I'm trying to run rsync on Windows with the cwrsync port. I'm issuing the following command (note that this is just a dry run): rsync -nPaAz foouser@webxx.example.com:/home/foobar/webapps/barbaz/ ...
2
votes
1answer
79 views

What are the rsync equivalent command/s to mv /a/b /c/d?

What are the rsync equivalent command/s to mv /a/b/ /c/d/, i.e, to move a directory tree from one location to another, possibly on another file system? That is, not just copy, but delete the files ...
2
votes
2answers
4k views

How do I replicate the directory structure to a remote Linux server?

I want to replicate a complex directory structure from one Linux server to another remote server start at a certain sub-directory. I don't want the contents of the files, I just want to preserve the ...
9
votes
10answers
6k views

Something like Dropbox for local use with a NAS as storage server

I am looking for a solution to sync folder pairs between a NAS and multiple local macs. Each of the macs could edit files and the other macs should then get synced automatically. Basically my own ...
0
votes
1answer
93 views

Prevent rsync out of memory

I am trying to rsync 55000 jpg files in 10000 folders (from an Android phone to a Linux box). Usually, I did this: rsync -rOi --size-only --delete --timeout=60 --progress --no-perms ...
0
votes
2answers
332 views

How to intelligently sync two directories under OSX

This may seem like a simple request at first glance, but I believe it is not due to some critical subtleties I wish to keep track of. Here is the scenario: I have a directory on my home workstation, ...
0
votes
0answers
17 views

Improve accuracy of rsync's speed reporting

I am doing an rsync from a local SSD (super-fast) to a samba NAS (super-slow). When I sync (e.g.) ~400MB files, it seems to read the first entirely into memory before it does any writing, and thus I ...
0
votes
2answers
65 views

How to migrate a root filesystem to another partition from within running system?

I try to migrate a root filesystem from ext4 to another partition type (zfs) and started with a simple rsync -a --exclude=/boot --exclude=/mnt --exclude=/media --exclude=/tmp --exclude=[all zfs ...
0
votes
0answers
36 views

How to tell if a directory is completely synced by rsync?

I'm rsync'ing some directories from one machine to another, in a cronjob. Each directory is complete set of data files. On the destination machine, is there a sane way to tell if a directory is ...
0
votes
0answers
39 views

Periodic backup of changes on another disk from full backup

I am looking for a backup solution that will enable me to backup only changes to an internal disk after a full backup to an external disk. Also, this should run snapshot backups every day until the ...
0
votes
1answer
114 views

Merging two directory trees with same structure, but different contents in Unix

I have two directories with the following structure: A |_0 |_1 |_2 B |_0 |_1 |_2 each 0,1,2 directory contains unique contents. I want all of the contents to simply exist as: A |_0 |_1 |_2 Is ...
7
votes
5answers
29k views

What similar rsync tool for Windows? [closed]

What can you recommend of an application that looks the same on rsync for windows? I will be doing migration where I need to transfer data from Windows to Linux. I like the rsync on linux where you ...
2
votes
0answers
225 views

External drive has strange files “_*” appearing - culprit is maybe rsync or SMB access?

I'm using rsync to back up folders from my iMac to a WD MyBook and the process seems to leave files such as these: _0O1MN~H.MP4 _2C0X2~3.AVI _ALLYW~B.SRT _VUC69~7 The last one is the most ...
3
votes
2answers
95 views

How to use rsync to make a copy of a huge source split into two destinations?

I have 3 drives. For this purpose, we will call them A, B and C respectively. A is 2 TB B is 1 TB C is 500 GB I have already run rsync -avz -progress A B Now B is full and I want to move/copy ...
0
votes
2answers
389 views

rsync on Windows not transferring large files

I'm noticing a strange issue where rsync isn't transferring large (approx >3GB) files. I'm running MSYS rsync and I'm transferring on my LAN from my Windows box to my Linux based NAS over ssh. Most of ...
0
votes
1answer
54 views

Rsync unhides files when syncing

When I tell rsync to sync 2 folders with -a /source/path/ /destination/path, hidden files become suddenly visible. Is there a way to tell rsync not to unhide those files, but simply copy the folder as ...
9
votes
2answers
14k views

Rsync only changed files

Is it possible to have rsync transfer all files in a directory that have changed or have been created? I don't need something as sophisticated as a CRC diff check; files with different timestamps ...
7
votes
5answers
5k views

Only show changed files while syncing from ext4 to NTFS

I would like rsync to print modified and deleted files. The verbose option (-v) does print modified files but also the list of subdirectories, maybe because touched directories are considered ...
0
votes
1answer
112 views

Is there any advantage of using rsync over plain cp when copying from a samba share?

I am currently configuring Dirvish on a Linux computer for backing up the contents of a share on a Windows computer. Dirvish creates a new directory every time it is called. It internally uses rsync ...
0
votes
1answer
65 views

Does Killing A Bash Process Affect Children Processes [duplicate]

The Scenario I'm automating backups via a bash script. The script is executed via cron. The script archives directories recursively using tar, and then rsync's them to the remote host. Here is a ...
12
votes
1answer
8k views

Does rsync delete files, folders at destination by default?

Files Suppose that /foo/src contains only A.c and that /foo/dest contains both A.c and B.c. And suppose I run the following command: rsync /foo/src/ /foo/dest Will rsync erase B.c? Folders Now ...
2
votes
1answer
3k views

Rsync difference between two directories to a 3rd directory

How would I go about rsyncing the difference between two directories to a 3rd directory. For example, if I have the command rsync -av --progress --stats /hdd1/ /hdd2/ I would like to copy the ...
1
vote
0answers
260 views

Run cygwin command from windows command line

I am trying to make PhpStorm run vagrant rsync when files are changed, because built-in rsync-auto fails with my deep directory structure. I am able to setup watcher in PhpStorm settings, but I don't ...
0
votes
0answers
147 views

Rsync takes longer time “receiving incremental file list”

I am using rysnc to copy files from remote host to local machine using a cron job. Every time I need the rsync to copy new files only from remote host. But its getting struck at this line "receiving ...
1
vote
1answer
221 views

Check difference in file structure

On Linux I've done a rsync syncing supposedly everything in /home to /backup. Both of these disk are the same size, but after the sync du -h shows around 50G difference. I want to know what the 50G ...
1
vote
1answer
239 views

Special characters on Synology NAS

I've been backing up my photos and mp3s from my MacBook Pro to a Synology NAS. Many files have names containing special characters like é and ö. I've been copying them with Rsync, as follows: rsync ...
2
votes
4answers
8k views

How to print Linux command output to a file?

I am creating a script to sync my important documents between two system. I want my script to generate a log file for the last action. can you suggest me a way to achieve this. Question: If I execute ...
9
votes
3answers
10k views

Is it possible to rate-limit an scp/sftp/rsync/etc transfer from the command-line? ie, manual QoS on a single command?

Specifically, I am looking to rate-limit an scp or sftp session (or other arbitrary network call) in the call itself. For example, let's say I want to copy 100MB to one server, and 1GB to another. ...
1
vote
1answer
716 views

RSync move files to a different directory if they were deleted on the server

So right now I use rsync to synchronize files with a server. Sometimes the files on the server are deleted (I know about the --delete option and I do not use it) and those files are left in my local ...
0
votes
0answers
34 views

Is there a simple way to automatically send through scp any file modified in a folder to a remote folder?

Put it simply: I have a folder full of files and subfolders with more files. I would like to have a mirror of that folder in a remote server so anytime a file is modified on my machine it gets ...
4
votes
3answers
3k views

Stop a Mac from sleeping while a bash script is running, then allow it to sleep as normal once complete

I've got my Mac to wake up during the night and run an rsync to back it up. This is configured through Energy Saver > Schedule. However it seems that it's going back to sleep before it can get ...
1
vote
3answers
298 views

Incremental Backup

How do I write a script which creates incremental backups using rsync or what are the parameters to do the incremental backup? Here's the scenario: I have a samba file server and I need to backup to ...
0
votes
1answer
50 views

Rsync: Mirror only certain directories based on presence of dotfile

I have a large amount of files on a server and I have a partial mirror of these files on my Mac (i.e. only some folders exist in the mirror). I would like to have a sync from the Mac to server using ...
1
vote
2answers
429 views

SCP remember password

On debian, is there anyway for scp to remember the password I've typed in for the duration of my ssh logon, similar to how sudo does? i.e. PC ---ssh(putty)---> server 1 ---scp---> server 2 ...
4
votes
1answer
457 views

How to encrypt rsync communication? (rsyncd + stunnel, vs rsync over SSH, etc)

I don't know much about the specifics of rsync, SSH, stunnel, etc, but I'm looking for a way to make rsync backups, but the transfer of data over the network must be encrypted (the credentials, and ...
3
votes
2answers
4k views

rsync only a specific subset of directories

I need to use rsync to synchronize several directories from an rsync server. The whole rsync module is quite large and I like to avoid coping the other, not required parts as well. I have the wanted ...
1
vote
1answer
209 views

SSH connection with key, not working with rsync

After creating a key with no password, I can login with SSH to my remote host: $ ssh root@machine.local Welcome to Ubuntu 14.04.1... However when I try to use SSH to rsync to the same remote host: ...
25
votes
7answers
19k views

Parallelizing rsync

I just moved and discovered after some trial and error that somewhere between my home and my remote server, there is some throttling going on...but the throttling is not very intelligent. It only ...
0
votes
2answers
114 views

“2 way” NAS 'time-machine' backup [closed]

For off-site backup purposes I am looking at putting a NAS in 2 places (my own and another) and having them sync. This can be done with rsync obviously, but I'm looking for a bit more security ...
3
votes
2answers
1k views

rsync --remove-source-files but only those that match a pattern

Is this possible with rsync? Transfer everything from src:path/to/dir to dest:/path/to/other/dir and delete some of the source files in src:path/to/dir that match a pattern (or size limit) but keep ...