Questions tagged [rsync]
Rsync is an open source command line tool for Unix and Windows systems that provides fast incremental file transfer to copy files and synchronize directories locally or between remote hosts. It is widely used for mirroring and backup.
1,398
questions
192
votes
4
answers
271k
views
What are the differences between the rsync delete options?
I see on the rsync man page that there are a number of delete options, but don't really understand the differences between them. What are the differences between these options?
--del ...
- 2,207
140
votes
16
answers
207k
views
How to synchronize the home folder between multiple computers?
I have three computers at home, and would like to have the /home/ folder tree synchronized between the three.
Any files/folders (except those hidden) that are modified/added/removed in one of the ...
134
votes
2
answers
261k
views
rsync and symbolic links
I want to backup my home directory to an external drive nightly using a cron job to execute rsync. I am unsure of the exact behavior of rsync's symbolic link flags.
rsync's -a flag includes the -l ...
- 1,495
118
votes
13
answers
236k
views
How to speed up rsync between two local disks?
I'm running rsync to sync a directory onto my external USB HDD. It's about 150 gigs of data. 50000+ files I would guess.
It's running it's first sync at the moment, but its copying files at a rate of ...
- 4,424
102
votes
2
answers
161k
views
What is Git Bash for Windows anyway?
I have happily been using Git and Git Bash from https://git-scm.com/. There is a page with more information here: https://git-for-windows.github.io/.
Yesterday I ran into a problem with rsync, and I ...
- 1,400
93
votes
3
answers
70k
views
Do I need to have a passphrase for my SSH RSA key?
Before I started at my current job (at a small business), my office had no firewall on the network and literally nothing was ever being backed up. Now that I've signed on as a dedicated sysadmin / one-...
- 1,174
76
votes
1
answer
84k
views
Does rsync delete files, folders at destination by default?
Files
Suppose that /foo/src contains only A.c and that /foo/dest contains both A.c and B.c. And suppose I run the following command:
rsync /foo/src/ /foo/dest
Will rsync erase B.c?
Folders
Now ...
- 952
68
votes
6
answers
99k
views
Why is rsync -avz faster than scp -r?
I'm a bit puzzled by this? Why is rsync faster than scp? Doesn't rsync use scp beneath or does it do something more efficient? Is there some way to speed up scp?
Note! The question is about a fresh ...
- 2,564
62
votes
11
answers
102k
views
Run rsync with root permission on remote machine
I want to sync a folder from my machine with a folder on a remote machine. The remote folder can only be manipulated by root. I have an account on the remote machine which can use sudo. How can I run ...
- 931
61
votes
1
answer
67k
views
Which is the rsync command to “smartly” merge two folders?
I have some folders that got duplicated by mistake and I have to merge them together. Let's say folder A contain file X and folder B contains file Y, I would like to get file X copied to folder B. But,...
60
votes
7
answers
90k
views
How to print files that would have been changed using rsync?
Is there a way to get rsync to print the full filepaths to all files that are different without actually transferring any files?
Alternatively, I need a way to diff the files across two trees (over ...
- 609
57
votes
8
answers
63k
views
How to do Rsync-like encrypted backup?
I want to save a backup of my data on a remote server, but never want the backup server to see the data unencrypted.
Editing a single file and backing up should not result in everything being ...
user67194
56
votes
5
answers
54k
views
Rsync files via intermediate host
I am currently away from my LAN and I need to do a backup of my laptop. I have a somewhat recent copy of my laptop on my server and I usually back the laptop up using rsync. Now I wish to do that, but ...
- 811
55
votes
5
answers
59k
views
Resume rsync over SSH after broken connection?
I have to transfer large amounts of data (>80 GB) over ssh using rsync. Everything is working fine, but the DSL connection where the backup data is sent from will drop once every 24h for up to 3 ...
Andreas
55
votes
3
answers
64k
views
rsync - does it create a temp file during transfer?
As far as I can see rsync doesn't create the file in the target directory until it is complete.
This must mean that it creates the file in a temp directory somewhere and copies the file into the ...
- 1,253
46
votes
2
answers
38k
views
Make rsync use less bandwidth?
I use rsync to backup and synchronize network shares and my computer. I have this figured out.
The problem is, when using rsync to transfer files, it uses all the bandwidth it can. I want to cap it's ...
- 2,206
46
votes
7
answers
186k
views
How to use rsync from Windows PC to remote Linux server?
How do I use the rsync command from a local Windows 7 machine to a remote Linux server?
What software is needed on the Windows 7 machine to use rsync?
The remote Linux server is using Amazon's Linux ...
- 9,225
45
votes
2
answers
52k
views
Use rsync to copy all files except for certain filenames with a certain extension
I have two same-size flash cards, and I want to copy contents of one to the other based on the following rules:
I want all directories and subdirectories in place
I want to exclude files of type .FOO,...
- 6,205
42
votes
17
answers
103k
views
Best way to copy millions of files between 2 servers
I have roughly around 5 million small (5-30k) files in a single directory that I would like to copy to another machine on the same gigabit network. I tried using rsync, but it would slow down to a ...
41
votes
5
answers
46k
views
RSYNC Does Not Delete Source Directories
I'm using rsync to essential fetch files off a server, then delete the files from the server once I have them locally. The full command I'm running is below.
This does successfully delete the files ...
- 533
41
votes
1
answer
30k
views
How to make rsync copy actual data instead of symlinks? [duplicate]
In the source directory I have some symbolic links. These links point to some paths and data is present in those paths also; but using rsync it is copying only the link files, not the data that is ...
- 511
39
votes
7
answers
66k
views
Does rsync --inplace write to the entire file, or just to the parts that need to be updated? (for btrfs+rsync backups)
I was reading several guides how combine btrfs snapshots with rsync to make an efficient backup solution with history. However it all depends on if rsync --inplace modifies only those portions of ...
- 3,153
38
votes
8
answers
58k
views
rsync file permissions on windows
I have an rsync service that syncs files from remote machine to a machine that drops them on a network drive.
I need the copied files to take on the native permissions of the destination folder.
...
- 809
37
votes
9
answers
48k
views
Parallelizing rsync
I just moved and discovered after some trial and error that somewhere between my home and my remote server, there is some throttling going on...but the throttling is not very intelligent. It only ...
- 611
36
votes
5
answers
158k
views
How to Setup Rsync without password with SSH on UNIX / Linux?
How can I set up a rsync between two hosts without providing any password?
user2010736
36
votes
4
answers
111k
views
How to interpret and fix a Input/output error in Linux?
I am running a daily backup with rsync. Starting some days ago, one of the files has been throwing this error during the backup:
rsync: read errors mapping "/home/folder/file.ext": Input/output error ...
- 1,878
35
votes
1
answer
27k
views
rsync recursive on same mount only
Hopefully, this simple question has a simple answer.
I want to run rsync recursively using the -r option, but without descending into a different mount point (file system). i.e. like the -mount ...
- 692
35
votes
1
answer
17k
views
How to exclude rsync excludes from delete?
I'm using rsync to sync files across multiple machines, using the following:
rsync -az -e "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no" \
--delete --delete-excluded --force --...
- 729
34
votes
5
answers
24k
views
Rsync friendly gzip
I must not be the only one - I'm rsyncing .tar.gz files and notice that every time the full file gets rsynced rather than the differences. Reading into it it seems back in 1999 someone created an ...
- 3,313
30
votes
4
answers
38k
views
Why does't rsync use delta-transfer for local files?
I have a big iso image which is currently being downloaded by a torrent client with space-reservation turned on: that means, file size is not changing while some chunks in in (4 Mib) are constantly ...
- 3,081
30
votes
2
answers
55k
views
rsync files newer than 1 week
I want to run rsync on server A to copy all files from Server B when they are newer than 7 days.
find . -mtime -7
I don't want to delete the files on Server B.
- 403
29
votes
3
answers
32k
views
rsync --remove-source-files deletes source files one by one or after rsync completes?
I have a rsync command in an application that copies the contents of a folder to another machine. I have the option --remove-source-files because I'm only interested in having the files on the ...
- 393
28
votes
5
answers
6k
views
Is it safe to use a HDD while rsync is running?
I plan to backup my large HDDs by rsync, and anticipate that it takes a few days. Is it safe to use the original HDD (adding files) while rsync is working? Or it is better to leave the HDDs untouched ...
- 1,042
28
votes
4
answers
36k
views
rsync command deletion error "IO error encountered -- skipping file deletion"
I use rsync command to take backup of files from one of my ubuntu server to another ubuntu machine.
Backup server trigger a script that use rysnc command.
Here is the command I use
rsync -rltvh --...
- 507
28
votes
2
answers
18k
views
Can I hard link files with rsync instead of copying them?
cp -l hard links files instead of copying them, saving filesystem space. I need to use rsync instead of cp because of its --exclude capabilities.
So my question is, how do I get rsync to hard link ...
- 1,429
27
votes
2
answers
63k
views
Rsync new and changed files
Is it possible to have rsync transfer all files in a directory that have changed or have been created? I don't need something as sophisticated as a CRC diff check; files with different timestamps and/...
- 612
27
votes
3
answers
48k
views
is there equivalent of rsync in MS powershell?
Rsync is very useful, I don't have to copy all files in a directory. It updates only the newer files.
I use it with cygwin but I think there are some inconsistencies, which are not the main focus of ...
- 1,318
26
votes
1
answer
14k
views
What does Rsync's output tell here about to-chk?
I was using rsync using --progress option , So the file transfer completed and I got following output
receiving incremental file list
Makefile
9,935 100% 9.47MB/s 0:00:00 (xfr#1, to-...
- 383
26
votes
4
answers
66k
views
How to best clone a running system to a new harddisk using rsync?
I have a system running as a server that has a failing harddrive. While all important data is on a RAID and backed up and all that, I don't have an image of the system itself. There is no specific ...
- 639
23
votes
1
answer
17k
views
Resume transfer of a single file by rsync
In Ubuntu, I want to copy a big file from my harddrive to a removable drive by rsync. For some reason or other, the operation cannot complete in a single run. So I am trying to figure out how to use ...
- 17.3k
23
votes
7
answers
19k
views
Replacing Dropbox with: Amazon S3 + SSL + GPG/TrueCrypt + Mounting on OSX ?? [closed]
So, right now we're using Dropbox to share various data files around between approximately 10 Mac OS X systems.
However, we already have an S3 account and everyone on the lowest Dropbox plan of $10/...
- 775
22
votes
3
answers
68k
views
rsync: failed to set times on "<dir path>"
I am trying to rsync files from one server to another but getting this error
rsync: failed to set times on "/dept/intranet/dept/atest/.": Operation not permitted (1)
Can someone help me why so ?
...
TopCoder
22
votes
5
answers
38k
views
How to arbitrarily map user/group ownerships in rsync
I need to rsync a directory to a remote server so that all files belonging to user X and group Y on the source (local) machine are mapped to user W and group Z on the destination (remote) machine. If ...
- 4,349
22
votes
4
answers
8k
views
Moving 2TB (10 mil files + dirs), what's my bottleneck?
Background
I ran out of space on /home/data and need to transfer /home/data/repo to /home/data2.
/home/data/repo contains 1M dirs, each of which contain 11 dirs and 10 files. It totals 2TB.
/home/...
- 353
21
votes
5
answers
31k
views
Transfering about 300gb in files from one server to another
I have about 200,000 files that I am transferring to a new server today. I haven't done anything on such a large scale before, and wanted to get some advice on how I should go about this. I am ...
- 429
20
votes
2
answers
20k
views
How can I get rsync to ignore missing files?
I'm executing a command like the following to several different systems:
$ rsync -a -v [email protected]:'/path/to/first/*.log path/to/second.txt' /dest/folder/0007/.
Sometimes *.log does not ...
- 5,633
19
votes
4
answers
19k
views
Why cwRsync doesn't work on Windows?
I'm trying to run rsync on Windows with the cwrsync port.
I'm issuing the following command (note that this is just a dry run):
rsync -nPaAz [email protected]:/home/foobar/webapps/barbaz/ C:\...
- 301
19
votes
1
answer
14k
views
rsync by path/filename only?
Is there a way for me to rsync two directories comparing only filenames (and not file size, date modified, or any other criteria)?
- 357
19
votes
2
answers
34k
views
Rsync installed but still getting an error command not found
I am trying to sync a local folder to a remote server using a non standard port and my rsync command is as follows:
rsync -avz ~/Research/Folder1/folder2 -e "ssh -p 3345" [email protected]:/home/...
- 295
19
votes
1
answer
35k
views
rsync between two local directories
I am trying to do rsync between two directories I have in two filesystems connected to the same computer. I want to do it without deleting the extra files in b. I though a simple rsync -a a/dir b/dir ...
- 4,347