Questions tagged [rsync]

Rsync is an open source command line tool for Unix and Windows systems that provides fast incremental file transfer to copy files and synchronize directories locally or between remote hosts. It is widely used for mirroring and backup.

463 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
9 votes
1 answer
4k views

Make rsync show progress on checksum comparison, even if no difference is being found

I run this command to check the identicalness of two folders and put all differences into a log file: rsync -av --delete --checksum --dry-run --stats --itemize-changes --out-format='%t %i %f ' SOURCE ...
Rotareti's user avatar
  • 504
7 votes
1 answer
7k views

Create a differential backup with rsync of a directory on my local drive to another directory on the same drive

How can I use rsync (but neither rsnapshot nor rdiff-backup nor any other application) to create a differential backup of a directory located on my local drive to another directory located on that ...
Yekutiel's user avatar
7 votes
1 answer
3k views

Rsync backups on btrfs very slow

My environment is Ubuntu 15.04 with kernel 3.19.0-28-generic and Btrfs v3.17. I have two identical external USB hard disks that I use with my backup script. One of them is formatted with btrfs and ...
matpen's user avatar
  • 215
6 votes
0 answers
3k views

ignoring No data available (61) errors with rsync

Is there a way to suppress a warning where the origin changes during the rsync? $ rsync --archive server::somemodule/ /dest/dir rsync: read errors mapping "/somefile" (in somemodule): No ...
Aleksandar Ivanisevic's user avatar
5 votes
0 answers
655 views

How to get output to STDOUT in Rsync while only logging changes to “--log-file” the way Rsync 2.6.9 did with Rsync 3.x

As a longtime macOS user I have used the built-in/bundled rsync binary that has been stuck at version 2.6.9 for various licensing reasons. I recently decided to start using modern rsync via Homebrew. ...
Devin Emke's user avatar
5 votes
0 answers
638 views

rsync over ssh tunnel: What does this repeated message mean? debug2: channel 2: window 1966080 sent adjust 131072

I'm rsyncing through an SSH tunnel (with ssh -vv). It's not very fast and I'm getting this message about every second: debug2: channel 2: window 1966080 sent adjust 131072 Once in a while I also get ...
the's user avatar
  • 2,819
5 votes
2 answers
900 views

Issue with China's Great Firewall (or other odd networking issue)

We periodically deploy an application to a computer located in China (from Canada). We deploy the changes using Subversion - we're talking a ~100Mb diff. We've noticed that the speeds are good until ...
Jason Kealey's user avatar
5 votes
1 answer
2k views

Copy folders and keep their modified date

I want to copy a couple of folders and preserve their modified date for historic reasons and not to change the sorting I'm used to. I have tried using rsync with flags to preserve times and the ...
soupdiver's user avatar
  • 397
4 votes
0 answers
786 views

Copying files of Windows partition using rsync gives readlink_stat(...) Input/output error (5)

I want to copy all files of a Windows 10 system partition (NTFS) to another partition (NTFS). I am using rsync under Ubuntu like this: sudo rsync -axHAWXS --numeric-ids /media/windows /mnt/windows-...
Willi Mentzel's user avatar
4 votes
1 answer
5k views

rsync issue failed: No such file or directory (2)

I am trying to use rsync files from my home directory. The error I get is: rsync: link_stat "/home/xxx/.mozilla/firefox/**/*.sqlite" failed: No such file or directory (2) I have checked and the file ...
axd's user avatar
  • 145
4 votes
1 answer
116 views

Does Postgres ever write to tables without file system timestamps getting updated for a long time?

I'm running Postgres 9.6 and backing it up once a while simply by stopping the cluster and using rsync on file level. One day I've recognized that some old, already backed up files for tables still ...
Thorsten Schöning's user avatar
4 votes
0 answers
5k views

How to always let rsync continue when (permission denied) error occurs

The following error happened with a rsync task. rsync: send_files failed to open "/home/user/scripts/@eaDir/script.js@SynoEAStream": Permission denied (13) rsync error: some files/attrs were not ...
JJD's user avatar
  • 199
4 votes
0 answers
617 views

Is there a way to verify the integrity of a rsnapshot backups?

I'm using rsnapshot to make daily backups to a second hdd. But how can I make sure, that no file in my backup is corrupted? If I use the checksum option from rsync, the whole backup will take much ...
schlicht's user avatar
  • 141
4 votes
2 answers
2k views

rsync --remove-source-files but only those that match a pattern

Is this possible with rsync? Transfer everything from src:path/to/dir to dest:/path/to/other/dir and delete some of the source files in src:path/to/dir that match a pattern (or size limit) but keep ...
user avatar
3 votes
1 answer
996 views

Rsync from Mac OS to synology with btrfs having issues with file and directories that contain umlauts and similar characters

I want to copy a folder from my local Mac (Catalina with APFS) to my Synology (btrfs filesystem) on my local network. And all files that have an umlaut in it (ö,ä,ü,....) gets in trouble somehow. ...
TomBlue's user avatar
  • 43
3 votes
0 answers
1k views

Is rsync's --link-dest option "better" than --backup-dir?

In his popular article Easy Automated Snapshot-Style Backups with Linux and Rsync, Mike Rubel asserts that using rsync to create hard-linked snapshots with --link-dest is "a better way" to perform ...
jth's user avatar
  • 141
3 votes
0 answers
125 views

How to copy Epichrome (SSB Chrome) apps to other workstations with active Extensions?

Epichrome is an absolutely great tool when it comes to site-specific-browsing. I've found it's useful to be able to copy Epichrome apps to other systems, however it appears that Chrome itself must ...
ylluminate's user avatar
  • 1,159
3 votes
0 answers
904 views

How to stop rsync from canceling when encountering a symlink without a referent

I use rsync to backup home dir of my computer 1 to computer 2 like this: rsync --recursive --archive --copy-unsafe-links --hard-links \ --delete-after --fuzzy --compress --human-readable \ ...
CrabMan's user avatar
  • 218
3 votes
0 answers
94 views

When do I need `--exclude=*` in an `rsync` call?

I sometimes read --exclude=* in rsync calls. But I don't really understand why I need them there. What is the effect of it? In wich use-cases should it be used? The question is not about --exclude ...
buhtz's user avatar
  • 329
3 votes
0 answers
330 views

systemd managing cgroups: assign specific binary to a cgroup

SHORT VERSION: I would like systemd to assign any instance of the rsync binary to a specific cgroup, automatically. How do I set up systemd to do this? LONG VERSION: When I run rsync for (large) ...
reikred's user avatar
  • 522
3 votes
0 answers
271 views

Copying of large files failed

I have been running simulations in supercomputer often writing large output files in h5 format (around 750MB). For backing up data after simulations to my local computer, I used rsync command. At the ...
SKPS's user avatar
  • 130
3 votes
2 answers
1k views

Creating a hard linked differential backup (trying with rsync)

I am trying to create a script that creates both a snapshot backup and a differential backup, but with hard links instead of duplicated files. In other words: All unchanged files between the current ...
Fabiano's user avatar
  • 756
3 votes
0 answers
535 views

Rsync strangely gets auto killed and then binary is broken

I got a strange situation with Rsync on my remote Ubuntu 14.04.3 LTS (Trusty Tahir) instance. Whenever I execute it, it runs for a while (not completed) then seems get auto killed. After that it's ...
Arcobaleno's user avatar
3 votes
0 answers
2k views

rsync SSD migration: lsetxattr failed: Operation not supported (95) -- consequences?

I just got an SSD and have been trying to migrate to it, poring through tons of how-to's I finally decided on this command (I have multiple partitions that needed migrating): time rsync -avHAXx /...
insaner's user avatar
  • 325
3 votes
0 answers
456 views

ssh proxycommand equivalent on server

Lets say we have three machines me, proxy, server I would like to have an ssh connection to server, so normally I would do me$ ssh proxy proxy$ ssh server server$ # and here we are ssh proxy ...
zidarsk8's user avatar
  • 213
3 votes
1 answer
959 views

rsync ... blocked for more than 120 seconds

I'm running an rsync between two external USB 2TB drives to make a copy on a CentOS 5.9 server. It appears to have stalled, and it hangs when I cd to the source drive in the copy, while the ...
Edward's user avatar
  • 509
3 votes
0 answers
495 views

Rsync, how can I delete remote duplicates?

I'm trying to delete remote duplicate files without transferring the non-existing files, and without updating the existing files. If I specify both --existing and --ignore-existing (along with -av --...
BlakBat's user avatar
  • 1,258
3 votes
1 answer
3k views

Rsync to a WebDAV filesystem on OS X copies all files regardless of being changed or not

I am trying to sync from my (Mac) desktop to an iPad and an iPhone. MacOS X mounts WebDAV as a native file system, but syncing results in all files in my directory structure being copied again. This ...
user avatar
3 votes
2 answers
5k views

Are cp/rsync asynchronous?

We're running a backup script which first copies a file to a destination and then runs tar over it. DIR2BCK='/foo/bar' TMPDIR=$(mktemp -d) rsync -a ${DIR2BCK} ${TMPDIR}/ > /dev/null 2>&1 ...
nKn's user avatar
  • 5,647
3 votes
1 answer
1k views

Rsync: Only preserve meta (time, group, etc) on files and sub-directories, not root directory

I am copying some files (all except hidden ones) using rsync from one place to another using this command: rsync -Cav --delete --exclude=.* /Some/Directory/ other-host:/Other/Directory It works nice ...
Svish's user avatar
  • 39.4k
2 votes
0 answers
279 views

Does rsync behave differently if it detects that a mount point is from a network instead of a local/external drive?

Note: I don't think I did a good job of explaining what I was looking for. The problem I'm running into is NOT which files rsync is selecting to mirror. The problem is that rsync is taking WAY ...
Jeremy W's user avatar
2 votes
0 answers
91 views

use rsync with autossh

I'm trying to use rsync with autossh on my laptop, so it can also work when internet connection is less reliable, but it doesn't really do as I'd like it to do. The starting point is that I have an ...
Richard Rosner's user avatar
2 votes
0 answers
22 views

How to change directories recursively in huge files

I tried to copy data between two huge NFS folders. I copied all the data, but I noticed that by mistake I made duplicate directories only in the 6th depth. For example, instead of copying: /mnt/nfs1/a/...
Yagel's user avatar
  • 121
2 votes
0 answers
2k views

Rsync not deleting files in destination directory despite “--delete” being set

I have the following rsync command: rsync -ahv --progress --delete \ /source_dir1/file1.txt \ /source_dir2 \ /dest_dir/ It syncs everything as expected, but does not delete anything from ...
willco's user avatar
  • 21
2 votes
0 answers
577 views

rsync with -X fails due to dot underscore files

I need to maintain sync copy from one external drive to another, on my macOS (Monterey). ext drive 1 (source) = ExFAT ext drive 2 (target) = APFS rsync v3.2.3 is used (not macOS default v2.6.9) rsync -...
C. Derx's user avatar
  • 121
2 votes
1 answer
309 views

rsync adding executable flag when syncing over network

I’m running rsync to synchronize two Debian machines. Both the source and the destination file systems are ext4. Naturally I want to preserve the permissions so I’m running the command with the ...
thuswm's user avatar
  • 21
2 votes
0 answers
891 views

rsync --ignore-missing-args not working

I'm trying to use rsync actually inside loop to backup a list of directories from remote. I just want to ignore if src directory do not exists. I got from most of the forums that --ignore-missing-args ...
Adil Saju's user avatar
  • 163
2 votes
0 answers
338 views

Can rsync adjust sym-links to reflect their new location?

I have a directory tree that has internal, absolute sym-links, and I want to rsync it with a different disk. My problem is sym-links. After I rsync, the sym-links on the new disk all point to the ...
Chris Miller's user avatar
2 votes
0 answers
691 views

How to efficiently distribute large, incrementally updated sqlite database?

What is an efficient way to transfer deltas from a large sqlite database? Note: this question is NOT about how to copy/distribute a "live" database. I.e. I am not concerned about corrupted ...
thegreatemu's user avatar
2 votes
0 answers
49 views

Copy symbolic link and its target subdir to new dir

I have a dir with a symbolic-linked subdirectory: greg:/testA> ls foo -> alice alice/ I need to copy the link "foo-> alice" (preserve it at destination) AND the subdirectory alice/ to a new ...
G. D'Seas's user avatar
2 votes
0 answers
880 views

rsync: send_files failed to open "filename": Invalid argument (22)

I use rsync to copy files from a mounted davfs box.com folder to an ext4 partition. For files containing the symbols [, ", and {, I receive the following error: rsync: send_files failed to open "...
Reid's user avatar
  • 21
2 votes
0 answers
28 views

rsync: can rsync adjust to the directory layout insteaed of transferring?

I have three systems, but only the target and a outdated mirror are on a fast link. Master_SLOW: On a super slow link. Contains a updated directory structure and files OldMirror_FAST: On a fast link ...
Waxhead's user avatar
  • 1,240
2 votes
0 answers
459 views

Using rsync through intermediate host with `sudo` on jump-host

I am user1@host1 and would like to transfer data via rsync from user3@host3. However, I can access host3 only via user3@host2. The catch is, that I can become user3 on host2 only via su or sudo, but ...
janoliver's user avatar
  • 183
2 votes
0 answers
812 views

Writing files to Android Phone SD card over SSH?

I successfully set up SSHDroid on my phone so I could both login and use rsync to copy files FROM the phone to the computer. However, when I try to push files back the other way I get the error: $...
SurpriseDog's user avatar
2 votes
0 answers
295 views

Stop rsync from deleting large amounts of data when disk accidentally umounted

I use rsync to mirror my hard drive onto a couple of devices with cron. The command is essentially this: rsync -avr --delete /home/username [email protected]:/home/username The problem I have is ...
Sarrusophone's user avatar
2 votes
0 answers
675 views

rsync: listing only changed files/attributes

I am trying to achieve the following: In order to check whether I correctly rsynced all data from my old NAS to my new, I am executing: rsync --rltD -hvinu /path/to/old/NAS/ /path/to/new/NAS Now: -...
quaylar's user avatar
  • 161
2 votes
0 answers
978 views

Rsync and accents in filename on macOS

When I use rysnc to backup my media hard drive that's plugged into my Ubuntu server using the command rsync -av --delete source destination it deletes and recopies all files that have accents in them, ...
cclloyd's user avatar
  • 752
2 votes
0 answers
265 views

MacOS 10.13.3 rsync and terminal: filenames with scandic letters

I have a rsync backup script that copies my personal files to an external drive with rsync: rsync -vr —-delete --progress /Users/omistaja/Documents/ /Volumes/MacExternal/Backups.Omistaja/ Edit: It ...
VeliJussi's user avatar
2 votes
0 answers
971 views

Remote sync without rsync

My setup is as follows: Two Linux machines running Debian 9 A "storage box" on the internet rented from a cloud hosting company. This box can only be accessed via the SFTP, Samba and WebDAV protocols ...
Kadabash's user avatar
2 votes
0 answers
406 views

Best Practices for Transferring Data Between Hadoop Clusters

Situation: I need to transfer around 100TB from an old Hadoop Cluster to a new Hadoop cluster. The two clusters have a direct 1GbE connection from the namenode on each cluster. The datanodes are not ...
tbenz9's user avatar
  • 7,095

1
2 3 4 5
10