8

I'm trying to transfer my Aperture library to the NAS. It contains few hundred of thousands of files (quite small ones).

After OS X finishes copying the big files, the estimated time of the copy operation goes from 2 hours to 4–5 days (and growing).

I know that a solution could be to tar them and copy the tarball, but the problem is that the library I want to copy is about 80GB and I have only 10GB left on my Mac.

Any ideas how to copy them in such conditions?

Improve this question

4 Answers 4

Reset to default
8

The problem probably is that you are working with many small files, and most copying tools have an overhead per file. It's possible to tar the files up, pipe the data over to your destination, and unpack the archive on the other end, without ever actually creating a tarball anywhere on the system.

tar cf - "~/Pictures/Aperture Library" | (cd "/Volumes/NAS/" && tar xf -)

The downside of this is that you won't have any kind of progress meter, but it should be significantly faster than a more conventional copy.

Improve this answer
3
  • Awesome, I thought that it is possible to create a pipe with tar.
    – OgreSwamp
    Commented Aug 22, 2013 at 22:04
  • 4
    You could pipe into pv to get an indication of what's going on.
    – slhck
    Commented Aug 22, 2013 at 22:11
  • thank you @evilsoup :)
    – minisaurus
    Commented Jun 2 at 17:45
7

You can use the rsync command line tool to do a one-way synchronization of your files. The way rsync works makes it easy to interrupt the process if you don't have the patience for it—you can easily resume it later on. Only the files that are still to be transferred will be copied.

You need to open Terminal.app, and then call rsync like so:

rsync -avh --progress "~/Pictures/Aperture Library" "/Volumes/NAS/"

Here, the first path is pointing to your library, which by default should be under Pictures. If you're unsure, you can drag and drop the library to the terminal command line and it will fill the path automatically for you. The same goes for the path of your NAS.

The -a option enables the archive mode, which sets some defaults, including recursive copying. -v will make the command more verbose. -h turns on human-readable file sizes.

Rsync will show you a progress meter. If you want to cancel the process, press CtrlC. You can call the command again to have rsync continue.

Improve this answer
5
  • Thanks. But as far as I know the only advantage will be that I can continue rsync later. But I don't feel like ready to wait 4 days to copy 80GB.
    – OgreSwamp
    Commented Aug 22, 2013 at 21:54
  • May be there is a solution for "tar" to network, not to my SSD with 10GB left. If I can copy folder as one file - it will be copied in couple hours.
    – OgreSwamp
    Commented Aug 22, 2013 at 21:59
  • I think the overhead when using Finder is bigger. Also the calculation could be quite off. evilsoup's solution is nice though.
    – slhck
    Commented Aug 22, 2013 at 22:12
  • The problem are small files. I've tried rsync and it still spend 1 second to copy about 10-20 files (even if their size is around 1K). That really slowed down the copy process. Hope the tar pipe will help to cpu it (there is no progress indicator, but it looks like it copies faster (I've checked what was copied on NAS already)
    – OgreSwamp
    Commented Aug 22, 2013 at 23:18
  • Late addition to this, but what seemed to have helped me is using the -W option which copies the whole file at once (I also had the issue of having lots of smaller files)
    – Jann
    Commented Aug 13, 2018 at 7:23
0

if external device supports full ssh shell, you can do:

tar cf - /your/dir|ssh -C 'tar xf - -C /directory/where/you/unpack'

it is not recommended use -c option for tar because you have many files. printing him cause speed degradation. flag -C for ssh is needed for compression on the fly on network. small piece of data cause small data to crypt/decrypt, this increase speed. but if it is still to slow, you can change crypt chiper from 3des to twofish or if it's possible to 1des. last is not secure but fastest.

otherwise you can use rsync, but rsync should be installed on external device. see man. rsync can transport data by rsync server or ssh.

Improve this answer
0

Try the MHISoft fastcopy opensource. It copy files and directories recursively. https://github.com/mhisoft/fastcopy/releases

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .