Questions tagged [rsync]
Rsync is an open source command line tool for Unix and Windows systems that provides fast incremental file transfer to copy files and synchronize directories locally or between remote hosts. It is widely used for mirroring and backup.
96
questions
18
votes
2
answers
26k
views
How to tell rsync to skip files on a damaged hard drive block, instead of being stuck trying to read it
I am trying to recover files from a disk that has hardware errors.
The disk was repaired by disk utility, but it told me that the disk is damaged, so I need to backup everything before a failure will ...
140
votes
16
answers
207k
views
How to synchronize the home folder between multiple computers?
I have three computers at home, and would like to have the /home/ folder tree synchronized between the three.
Any files/folders (except those hidden) that are modified/added/removed in one of the ...
102
votes
2
answers
161k
views
What is Git Bash for Windows anyway?
I have happily been using Git and Git Bash from https://git-scm.com/. There is a page with more information here: https://git-for-windows.github.io/.
Yesterday I ran into a problem with rsync, and I ...
56
votes
5
answers
54k
views
Rsync files via intermediate host
I am currently away from my LAN and I need to do a backup of my laptop. I have a somewhat recent copy of my laptop on my server and I usually back the laptop up using rsync. Now I wish to do that, but ...
18
votes
4
answers
10k
views
Is there a way to access files from one WSL 2 distro/image in another one?
I have several Unbuntu WSL 2 "installations" on my Windows 10 system and I'd like to be able to run tools like rsync and diff between them. Is it possible to mount/find where the files for ...
57
votes
8
answers
63k
views
How to do Rsync-like encrypted backup?
I want to save a backup of my data on a remote server, but never want the backup server to see the data unencrypted.
Editing a single file and backing up should not result in everything being ...
134
votes
2
answers
259k
views
rsync and symbolic links
I want to backup my home directory to an external drive nightly using a cron job to execute rsync. I am unsure of the exact behavior of rsync's symbolic link flags.
rsync's -a flag includes the -l ...
117
votes
13
answers
234k
views
How to speed up rsync between two local disks?
I'm running rsync to sync a directory onto my external USB HDD. It's about 150 gigs of data. 50000+ files I would guess.
It's running it's first sync at the moment, but its copying files at a rate of ...
55
votes
5
answers
59k
views
Resume rsync over SSH after broken connection?
I have to transfer large amounts of data (>80 GB) over ssh using rsync. Everything is working fine, but the DSL connection where the backup data is sent from will drop once every 24h for up to 3 ...
38
votes
8
answers
58k
views
rsync file permissions on windows
I have an rsync service that syncs files from remote machine to a machine that drops them on a network drive.
I need the copied files to take on the native permissions of the destination folder.
...
6
votes
2
answers
4k
views
rsync seems to overwrite already existing file on ExFat
I have a curious behavior from rsync between two hard drives containing the exact same four partitions. However on one partition rsync seems to always copy again the files, I don't understand why.
...
4
votes
2
answers
3k
views
rsync over SSH with cron in osx-environment
I want to automatically download files and folders from a Linux server to which I have an SSH (and FTP) account. The files shall be downloaded on a regular basis (I suppose a cron is the right tool to ...
4
votes
2
answers
6k
views
rsync as a background process?
I've been using rsync to sync a bunch of files between my two servers. I set a cron job that would run rsync every minute. However, I'd prefer if I could have it run always, so that files are synced ...
55
votes
3
answers
63k
views
rsync - does it create a temp file during transfer?
As far as I can see rsync doesn't create the file in the target directory until it is complete.
This must mean that it creates the file in a temp directory somewhere and copies the file into the ...
26
votes
4
answers
66k
views
How to best clone a running system to a new harddisk using rsync?
I have a system running as a server that has a failing harddrive. While all important data is on a RAID and backed up and all that, I don't have an image of the system itself. There is no specific ...
16
votes
4
answers
16k
views
Exact copy of files from NTFS to another NTFS using Linux
I try to copy all files from an NTFS to an external drive and want to preserve all permissions and attributes. The external drive has already an NTFS and contains other files.
I've considered using ...
15
votes
2
answers
34k
views
Behavior of rsync with file that's still being written?
If Apache is in the middle of writing a large file and an rsync cron job runs on that file, does rsync attempt to copy the file?
Example
Apache-1: Has file large being written to /var/www.
Apache-2: ...
15
votes
2
answers
15k
views
Why does rsync over SSH give me 10x the throughput of SCP?
scp user@aws-ec2:~/file file
rsync --partial --progress -Pav -e ssh user@aws-ec2:~/file file
scp only gives me 200K/s, but rsync gives me 1.9M/s
I tested several times, all same results.
rsync ...
12
votes
3
answers
29k
views
How do I compare two folders recursively and generate a list of files and folders that are different?
tl;dr and an example
I'm looking for a way to compare two folders recursively and output the relative paths all files (and folders) that are different (by size or by timestamp, à la rsync).
For ...
12
votes
1
answer
18k
views
How can I set environment variables for a remote rsync process?
When I use rsync to copy files from one computer to another, an rsync process is started on both ends. However, on the remote end, shell initialization files are apparently not read, so I cannot set ...
6
votes
1
answer
7k
views
Rsync is resending all files, because files have different timestamp (off by 1 second?)
I am using rsync to backup files from my Mac laptop to a USB drive (exFAT) on my Windows laptop. The USB drive is shared within my home network, and mounted on my Mac.
I noticed a weird problem when ...
5
votes
4
answers
13k
views
Make rsync case insensitive
I'm rsync'ing a bunch of files between a Windows and a Linux system. Since not all Windows care about case, some of the files on the Windows system no longer have the same casing as they had on the ...
5
votes
2
answers
9k
views
rsync slow copy from one disk to another
I got two drives of the same model which are connected using SATA.
When benchmarking read speeds using hdparm -t /dev/sdX, speeds of about 160MB/s are returned. When benchmarking write-speed using dd ...
5
votes
2
answers
1k
views
How do I efficiently move a large, empty disk image to another system
I'm currently trying to move a fairly empty, fresh qcow2 image from one system to another that's far away. I've basically got a fresh install of ubuntu LTS (real size 6gb, apparent size 200gb). Most ...
1
vote
2
answers
3k
views
How to improve performance of RSYNC's delta transfer on receiving host using software only?
I'm using RSYNC to backup VMs for Virtual Box from one server to some Synology NAS DS1512+. The important point is that I really want to backup the VM-images itself, NOT individual files within those ...
1
vote
2
answers
1k
views
sync local and remote directories using rsync?
I have two linux box that i would like to keep in sync. I see option -avz syncs the remote with local but new local files are not pushed. How do i do this?
93
votes
3
answers
70k
views
Do I need to have a passphrase for my SSH RSA key?
Before I started at my current job (at a small business), my office had no firewall on the network and literally nothing was ever being backed up. Now that I've signed on as a dedicated sysadmin / one-...
62
votes
11
answers
101k
views
Run rsync with root permission on remote machine
I want to sync a folder from my machine with a folder on a remote machine. The remote folder can only be manipulated by root. I have an account on the remote machine which can use sudo. How can I run ...
60
votes
7
answers
89k
views
How to print files that would have been changed using rsync?
Is there a way to get rsync to print the full filepaths to all files that are different without actually transferring any files?
Alternatively, I need a way to diff the files across two trees (over ...
46
votes
7
answers
185k
views
How to use rsync from Windows PC to remote Linux server?
How do I use the rsync command from a local Windows 7 machine to a remote Linux server?
What software is needed on the Windows 7 machine to use rsync?
The remote Linux server is using Amazon's Linux ...
41
votes
5
answers
46k
views
RSYNC Does Not Delete Source Directories
I'm using rsync to essential fetch files off a server, then delete the files from the server once I have them locally. The full command I'm running is below.
This does successfully delete the files ...
36
votes
4
answers
110k
views
How to interpret and fix a Input/output error in Linux?
I am running a daily backup with rsync. Starting some days ago, one of the files has been throwing this error during the backup:
rsync: read errors mapping "/home/folder/file.ext": Input/output error ...
35
votes
1
answer
27k
views
rsync recursive on same mount only
Hopefully, this simple question has a simple answer.
I want to run rsync recursively using the -r option, but without descending into a different mount point (file system). i.e. like the -mount ...
29
votes
3
answers
32k
views
rsync --remove-source-files deletes source files one by one or after rsync completes?
I have a rsync command in an application that copies the contents of a folder to another machine. I have the option --remove-source-files because I'm only interested in having the files on the ...
29
votes
2
answers
55k
views
rsync files newer than 1 week
I want to run rsync on server A to copy all files from Server B when they are newer than 7 days.
find . -mtime -7
I don't want to delete the files on Server B.
28
votes
4
answers
36k
views
rsync command deletion error "IO error encountered -- skipping file deletion"
I use rsync command to take backup of files from one of my ubuntu server to another ubuntu machine.
Backup server trigger a script that use rysnc command.
Here is the command I use
rsync -rltvh --...
23
votes
1
answer
17k
views
Resume transfer of a single file by rsync
In Ubuntu, I want to copy a big file from my harddrive to a removable drive by rsync. For some reason or other, the operation cannot complete in a single run. So I am trying to figure out how to use ...
22
votes
3
answers
67k
views
rsync: failed to set times on "<dir path>"
I am trying to rsync files from one server to another but getting this error
rsync: failed to set times on "/dept/intranet/dept/atest/.": Operation not permitted (1)
Can someone help me why so ?
...
22
votes
5
answers
38k
views
How to arbitrarily map user/group ownerships in rsync
I need to rsync a directory to a remote server so that all files belonging to user X and group Y on the source (local) machine are mapped to user W and group Z on the destination (remote) machine. If ...
19
votes
4
answers
19k
views
Why cwRsync doesn't work on Windows?
I'm trying to run rsync on Windows with the cwrsync port.
I'm issuing the following command (note that this is just a dry run):
rsync -nPaAz [email protected]:/home/foobar/webapps/barbaz/ C:\...
16
votes
6
answers
54k
views
rsync using regex to include only some files
I am trying to run rsync to copy some files recursively down a path based on their file name pattern, case insensitive. This is what I have done to run rsync:
$ rsync -avvz --include ='*/' --include='...
15
votes
2
answers
8k
views
Does anyone know why rsync would keep sending the files over and over again?
I'm trying to using rsync to backup some files, about half a TB. It's now it a state where it keeps sending the same files everytime it runs.
for example:
rsync -av /data/source/* user@host:/data/...
15
votes
1
answer
12k
views
Leave ACL handling to Windows with Cygwin rsync
I'm running an rsync command, copying my files from a server to my Windows machine like:
rsync -rt --partial-dir=".rsync" --del rsync://server/a/ a/
Whenever this command creates a directory, it ...
15
votes
3
answers
40k
views
How to rsync a large file, with as little CPU and bandwidth expense as possible?
I have a 500 GB file that I plan on backing up remotely. The file changes often. I'll be rsyncing it from a desktop to a server. Both can run rsync client or server.
What is the proper command for ...
15
votes
3
answers
17k
views
Stop a Mac from sleeping while a bash script is running, then allow it to sleep as normal once complete
I've got my Mac to wake up during the night and run an rsync to back it up. This is configured through Energy Saver > Schedule.
However it seems that it's going back to sleep before it can get ...
14
votes
3
answers
91k
views
rsync Permission denied backing up a remote directory to my local machine
I'm getting the error mentioned in the title.
I found this similar question: Run rsync with root permission on remote machine.
That doesn't answer my question.
I'm the admin on the remote server ...
13
votes
3
answers
13k
views
linux merge folders: rsync?
I have two copies of a folder
src/
dest/
I want to merge them, doing the following:
If a file is only in src, I want it to be moved to dest
If a file is only in dest, I want it ignored IE left ...
13
votes
5
answers
5k
views
How to get rsync to not print directories
I use rsync to create backups on an external hard drive. I usually run a dry-run first just to sanity check. Specifically, I will use
rsync -auvn path/to/source path/to/dest
However this prints all ...
13
votes
1
answer
28k
views
Understanding rsync's include / exclude patterns
I want to transfer N files from a remote system, using a pattern:
ssh remote ls target/
2013-08-01.mjson.gz
2014-07-04.mjson.gz
2014-07-09.mjson.gz
2014-08-12.mjson.gz
...
I only want the 2014-07* ...
12
votes
4
answers
15k
views
Rsync over ssh, or something else?
I've been trying to use rsync on OSX to Linux or Unix box over which I don't have much control. I've been doing something like this
rsync -avz -e ssh remoteuser@remotehost:/remote/dir /this/dir/
...