I have two directories, dirA whicht contains N gb of data and dirB which is supposed to contain only the newest M gb of data from dirA. When files are added to dirA, they sould also be added to dirB, while the oldest files in dirB should be deleted.
Is that possible with rsync? or any other software?
Here's an attempt. A word of caution: I haven't tested the following code extensively.
dirA=/path/to/dirA
dirB=/other/path/to/dirB
tmpdir=/path/to/tmpdir
cd "$dirA"
find . -type f -printf '%T@ %s %p\0' | sort -nr -z \
awk 'BEGIN {RS="\000"; ORS="\000"; maxsize=2147483648}
{size += $2; if (size > maxsize) exit; print $3}' | \
rsync -Rpt --link-dest="$PWD" --files-from=- -0 . "$tmpdir" && \
rsync -rpt --delete "$tmpdir" "$dirB"
Some explanations:
find, sort, and awk parts is to get a machine-parsable (i.e. null delimited) list of the newest files, sorted by newest first, and then truncate that list when the accumulated size reaches 2 GiB (the 2147483648 is simply 2*2^30 bytes, or 2 GiB; adjust this number to taste).awk part means the size is a hard upper bound (an "at most" condition), so it can result in situations like the following: if the newest file was 4 GiB, nothing would be included in the sync. Move the print $3 to the beginning of the { } to get the opposite behavior ("at least"), in which case you would get the 4 GiB file and end up syncing twice as much as you intended. I assumed the first was more desirable.rsync extracts those files to a temporary location ("$tmpdir") but preserves path (-R a.k.a. --relative). Since it uses hard links (using --link-dest) and not a full copy, this step uses negligible space, but $tmpdir must reside in the same file system as $dirA.rsync invocation does the actual syncing with $dirB and --delete-ing of old files in $dirB not found in $tmpdir. The reason for using the temporary directory and doing this in two steps is I haven't yet found a way to get the deletion part to work in just one step.
