2
votes
1answer
16 views

Understanding rsync's include / exclude patterns

I want to transfer N files from a remote system, using a pattern: ssh remote ls target/ 2013-08-01.mjson.gz 2014-07-04.mjson.gz 2014-07-09.mjson.gz 2014-08-12.mjson.gz ... I only want the 2014-07* ...
0
votes
0answers
15 views

Is there a tool for syncing changes to individual nodes within HDF5 files?

I'm looking for a tool for efficiently syncing large scientific datasets in HDF5 format: Each file is ~20-30GB in size and contains many large floating point arrays, which are usually chunked and ...
0
votes
1answer
20 views

rsync and symbolic links

I want to backup my home directory to an external drive nightly using a cron job to execute rsync. I am unsure of the exact behavior of rsync's symbolic link flags. rsync's -a flag includes the -l ...
0
votes
1answer
16 views

automatic file encryption on NFS or RSYNC for remote incremental backup?

What solutions should I be looking at to allow an incremental backup via rsync over nfs4 to allow files to be encrypted on the backup host? I have zero control over the backup host's software so the ...
0
votes
1answer
26 views

Source host doesn't have access to Internet, but can SSH to host that has; how can I rsync to a third, Internet host?

This is the command I used to run from my SOURCE machine rsync -Rrlt --progress --delay-updates ./my_folder root@machine-on-same-network:/home/me/. Now I need to copy over this same folder to a ...
1
vote
0answers
24 views

Rsync doesn't allow me copy files into mounted filesystems

I configured rsyncd.conf for public access in my rpi: log file = /var/log/rsyncd.log timeout = 600 max connections = 20 uid = nobody gid = nogroup read only = yes list = yes [public] path = ...
0
votes
0answers
19 views

Trying to rsync and use sudo on other end

Scenario: Mac OS X Machine - old computer - user: john.doe Mac OS X Machine - new computer - user: john.doe Goal: Copy /Users/john.doe/Desktop and /Users/john.doe/Documents via rsync Problem: ...
0
votes
2answers
24 views

Syncing directory between Kubuntu and Windows 7

I recently came across an issue on my work computer. I'm running Kubuntu 14.04 as my main development OS and Windows 7 for compatibility with some company software and network stuff. My problem is ...
0
votes
1answer
31 views

Do GUI and Terminal execution (of Back In Time/Grsync/other Linux apps) produce the same results?

I've opened Back In Time GUI and copied the whole rsync ... line from the last log into Terminal and executed it in Terminal. Question #1 is: if I were to do it the normal way (click 'Take snapshot' ...
3
votes
1answer
34 views

Using pv(1) to limit rsync speed

I'm trying to use pv rather than rsync's builtin --bwlimit to limit the bandwidth of an rsync transfer because --bwlimit is implemented in an eventually-consistent way, and my transfers are so short ...
0
votes
0answers
40 views

Synology NAS resume rsync backup after connection loss

I am trying to do a backup from a synology NAS to an offsite NAS. I am using rsync through DSM5 as of now, but it doesn't give me the possibility to resume the backup if a connection loss occurs. This ...
1
vote
1answer
30 views

Rsync transfers everything instead of changed

I have tried many many different combinations of rsync commands, but whatever platform I use it on, from whatever source to whatever destination. It always transfers everything no matter if it's ...
1
vote
1answer
35 views

Find files with different inodes in same directory structure

We have a slight problem with our backups. Our setup is as follows: We create a backup using the rsync command. The backup uses the option --link-dest option, to create incremental backups with ...
0
votes
1answer
84 views

File synchronization between two Linux servers

We have a two web servers running CentOS. We need to sync images that users uploaded. No servers should be needed to synchronize those, because we need to handle failover. Also, we need to do a ...
0
votes
0answers
102 views

How to always let rsync continue when (permission denied) error occurs

The following error happened with a rsync task. rsync: send_files failed to open "/home/user/scripts/@eaDir/script.js@SynoEAStream": Permission denied (13) rsync error: some files/attrs were not ...
1
vote
2answers
183 views

Properly escaping forward slash in bash script for usage with sed

I'm trying to determine the size of the files that would be newly copied when syncing two folders by running rsync in dry mode and then summing up the sizes of the files listed in the output of rsync. ...
0
votes
0answers
393 views

Can't git push to Bitbucket.org — with ssh keys

I'm running doku-wiki on may (Ubuntu) laptop for some note taking. So, to keep backup, I was using Bitbucket and periodically pushed my wiki files there and everything worked smoothly. (a) Some time ...
0
votes
1answer
22 views

rsync per-site configuration file?

I know how to configure a per-site entry for ssh, but is there any kind of a client configuration for rsync that allows per-site configuration options and aliases or similar shortcuts like the ...
0
votes
0answers
83 views

File transfer tools for linux with support of zero-copy transfer

Are there small and simple file transfer tools for Linux, which can use "zero-copy" style of transferring files from HDD (SSD) to tcp socket, and back? I want to often send huge amount of ...
1
vote
1answer
50 views

Setting up Debian server for remote backups

I am running a Debian Wheezy server at home, and am offering an organization to let them do rsync backups to their own folder on my system. I created the unique user for them, and got a strong ...
0
votes
1answer
90 views

ntfs disk dying or..? rsync and mkdir failing, strange behavior

System root@host:~# uname -a Linux host 3.2.0-4-686-pae #1 SMP Debian 3.2.57-3+deb7u1 i686 GNU/Linux Mounts root@host:~# mount | tail -2 /dev/sda1 on /mnt/sda1 type ntfs ...
0
votes
1answer
13 views

rsync exclude config.js but include lib/project/config.js

I have a folder like this / config.js /lib /project config.js In the root directory I execute rsync rsync -azvr --exclude config.js --include lib/project/config.js . dest The ...
3
votes
1answer
71 views

How to control permissions on external USB hard disk

I'm using RHEL 6.5, andhave an auto-mounted external USB drive. Any and all files created in the mounted filesystem are given permission 700 for dirs and 777 for files. The chmod command does not ...
0
votes
0answers
11 views

Using an SD card for regular backups? [duplicate]

I recently updated to a 128 GB solid state drive, and I've gotta say, I love the boot times and app load times, however I'm not keen on the increased possibility of spontaneous failure. I figure I've ...
0
votes
1answer
63 views

How to make rsync completely ignore permission from source

I'm trying to use rsync to copy from an Android source where the sdcard mount has rather nonsensical permissions (---rwxr-x) and rsync fails because it creates the destination directories with the ...
0
votes
2answers
41 views

Rsync folders automatization

I've got a folder on the remote server, to which I'm connecting via SSH. When I need to synchronise folders I'm using manually access via scp. (I'm synchronising my folder with the folder on the ...
0
votes
1answer
30 views

Merging two directory trees with same structure, but different contents in Unix

I have two directories with the following structure: A |_0 |_1 |_2 B |_0 |_1 |_2 each 0,1,2 directory contains unique contents. I want all of the contents to simply exist as: A |_0 |_1 |_2 Is ...
0
votes
1answer
144 views

Exact copy of files from NTFS to another NTFS using Linux

I try to copy all files from an NTFS to an external drive and want to preserve all permissions and attributes. The external drive has already an NTFS and contains other files. I've considered using ...
0
votes
0answers
47 views

How to output file sizes in human-readable format in an rsync logfile

I know that the -h option outputs filesizes in human-readable format when, ex, displaying the transfer progress with --progress, but is it possible to get this same file size formatting in the log ...
0
votes
1answer
42 views

Fast splitting of list of files in chunks of the same size (memory)

See title. Assume I have the following list of files and relatives sizes: cat list_of_files.txt: 1243 001.txt 124 002.txt 4132 003.txt 3000 004.txt I want to be able to split into multiple chunks ...
0
votes
0answers
33 views

web interface for server-to-server file transfer [duplicate]

I'm looking for a way to transfer files between two servers. I currently ssh into one server and use rsync inside a screen session to initiate a transfer from the other host. This works, but I'm ...
0
votes
2answers
31 views

rsync from two servers to a common location

I have two servers and I want to rsync files from both servers to a single location. Server 1 has files A and B Server 2 has files A and B After rsync server 3 has file A and B When a user ...
0
votes
0answers
25 views

How to rsync merge two paths into one; opposite of --inplace

I have a large remote directory: remote://server/pathA/zupertoll And a large local directory that is 95% the same as the remote directory: local:///home/pathB/zupertoll I want to copy zupertoll to ...
0
votes
1answer
94 views

Transparent failover for NFS home directory (with LDAP users)

Background: In our work environment, we have LDAP users together with NFS homes. So you sign on LDAP server, autofs mounts the homes, etc. Pretty neat so far. However, from time to time, the NFS or ...
0
votes
0answers
59 views

ssh proxycommand equivalent on server

Lets say we have three machines me, proxy, server I would like to have an ssh connection to server, so normally I would do me$ ssh proxy proxy$ ssh server server$ # and here we are ssh proxy ...
0
votes
0answers
27 views

mv backup but instead of creating tilde directories, create tilde files in subdirectories?

I have 2 deep directory structures that are (probably) identical and I want to end up with a single one, but not auto-resolve existing conflicts (I'm guessing rsync would clobber something). I just ...
0
votes
0answers
30 views

How to backup incremental with rsync with diffrent location for incremental changes?

I want to do incremental backup with rsync in centos. but I want to have a full backup in a specific folder and for incremental changes I use to another folder with time stamp. Is it possible and how? ...
0
votes
1answer
43 views

rsync don t work as expected

I have a script which download and compress a folder of a remote server (example: /home). I m trying to make the "restore" script, but I have a strange behavior. restore.sh: ($1 is IP adress, $2 is ...
0
votes
1answer
64 views

Error message using scp over ssh

I'm not sure what's going on but as of late I can't seem to get any files copied to local from my remote server. I tried rsync from local and get the following error message: mut@mylocalcomputer:~$ ...
0
votes
0answers
40 views

Copy files to location, skipping duplicates (in source and dest. location)

I have an external hardrive I think is running down breaking (unable to delete some files from it). I copied a bunch of files over already but ran out of space. How do I dedup while copying/How to I ...
0
votes
0answers
70 views

Backup with rsync in an encrypted folder

I want to backup a folder from a remote server in a local folder encrypted with encfs. The files in the remote folder are owned by different users and I want to keep ownerships. The local folder is in ...
0
votes
1answer
44 views

cwrsync and long file names issue

I am trying to use cwrsync on my windows server to backup to my linux server, but am running into an issue where the file names become too long. So a user on one of the systems will have a mapped ...
0
votes
1answer
53 views

rsync over ssh but block changes to authorized_keys file on remote computer

OK I finally got rsync over ssh setup with authorized keys. Since I don't want my users browsing the filesystem, I have modified the authorized_keys file to run an rsync script. That way they can't do ...
2
votes
1answer
284 views

rsync on NTFS - permissions

I'm backing up directories on a Fedora machine onto an NTFS drive (fuseblk according to df -T) and it seems to go just fine with this: ls -t | sed -e "1,"$n"d" | xargs -d '\n' rm -rf nice rsync -azv ...
0
votes
2answers
330 views

RSYNC over SSH requesting password

I looked at numerous question but don't seem to find quite what I'm looking for. I've created a script in which I'm trying to run rsync over ssh, this will eventually be put into a cron job. The ...
0
votes
1answer
68 views

Watch a filesystem and rsync changes in a cross-platform way with minimal dependencies

I'd like to start a long-running Ruby or Python script, and have the script watch a local directory for changes to files and new files. Periodically, the script will trigger a one-way rsync to a ...
0
votes
1answer
60 views

Ununtu 12.0.4 LTS Boots to some unknown mount point

I installed Ubuntu 12.0.4 LTS to a 2TB Sata with a swap partition and a data partition. I installed mdadm and setup an incomplete raid. I later unmounted that raid (to which it erred as non-existent) ...
2
votes
1answer
391 views

Rsync very slow

I have just setup a Debian server for backup of Windows PCs. I am currently testing various methods including: rsync (using Deltacopy on Windows), SMB via Samba and scp. Currently rsync (without SSH ...
2
votes
2answers
309 views

Is there really no way in Linux to get creation time for files on cifs/smb share?

I am embarrassingly far along in a bash backup script and have missed something major. I know that ext3 filesystems don't track creation date, but copying files within ext3 updates the changed time as ...
1
vote
1answer
125 views

Automatically purge rotated backup without breaking rsync

How do I make sure locally deleted files have been rotated before deleting them on the server, whithout breaking incremental rsyncing? I usually rsync my files into /backup/active/ on my server, ...