1

I'm copying the Xcode.app from one computer to another using scp and it seems to have copied more than the file size and it's still copying. It 4.94GB (I didn't compress it), but nettop on the destination machine says 7430MB have been copied so far and it hasn't stopped yet. nettop on the source machine doesn't have any activity that corresponds to the copy.

The command I used 

scp -rp paul@192.168.16.178:/Applications/Xcode.app /Applications/

nettop on the destination machine:

ssh.636                                                                                         7662 MiB          21 MiB   178 KiB  2029 KiB  1044 B
tcp4 192.168.16.108:50145<->192.168.16.178:22                    en1     Established        7662 MiB          21 MiB   178 KiB  2029 KiB  1044 B    13.53 ms   512 KiB   128 KiB        BE

Any ideas why it has copied over 7GB when the file is only 4.94GB? Source and destination machines are iMac and Mac mini, copying over a wifi local network.

Update: Copy finished and when I check Xcode's size on the destination machine it's a whopping 10.07GB. I don't understand how Xcode should have doubled in size during the copy. My only theory is that .app directories are already compressed and that scp uncompresses them.

Cheers!

Improve this question
3
  • 2
    Perhaps there are symbolic links in the Xcode.app directory which get copied (but are not counted on the source machine)?
    – mtak
    Jun 21, 2014 at 18:19
  • Maybe, running du -L to get file size with links followed results in du: Too many symlinks at.
    – paulvs
    Jun 21, 2014 at 18:45
  • That was spot on @mtak!
    – paulvs
    Jun 24, 2014 at 20:42

1 Answer 1

Reset to default
2

Here is a page describing how the SCP protocol works. The notable fact is that it can only transfer files and directories; there's no protocol support for symbolic links. In fact, SCP will treat each symbolic link as if it were an actual file or directory.

My copy of Xcode.app is 4,936,956,797 bytes according to the OS, and find tells me it has 7,499 symbolic links. We can trick scp into pretending to copy Xcode.app in order to see how much data it writes:

$ scp -rf Xcode.app < /dev/zero | wc -c
 8746268376

And we can do the same with tar:

$ tar cf - Xcode.app | wc -c
 5018552320

Tar can handle symbolic links, so it only writes a little more data than is actually in the directory structure. Scp can't handle symbolic links, and it's expanding each link into an actual file or directory.

Improve this answer
3
  • +1. For a copy I suggest tar (and gzip if your connection is slow but your CPU is fast), then copy that, then untar on the receiving end. I believe rsync also can preserve symlinks. It's worth noting that @paulvs should really redo this with proper symlink copies - its more than just taking up too much space, those symlinks exist for a reason, so that multiple locations can refer to the same file and if that file changes or is updated, the updated version is also accessible through the symlinks. You could end up with a very unstable xcode some day if you don't.
    – Jason C
    Jun 22, 2014 at 0:04
  • Excellent answer @Kenster, using the scp command you posted I got the same size as what what arrived at the destination machine. Curiously the tar command gives an output 1.4GB larger than what Finder says the size is (they both should be ignoring symbolic links as I understand).
    – paulvs
    Jun 24, 2014 at 20:41
  • Tar has some overhead. There's an I think 512-byte header for each file, plus the file contents. Symbolic links have to be represented. I don't know how the finder counts symlinks when it's calculating the size of an app. Tar format is described here if you're interested.
    – Kenster
    Jun 24, 2014 at 20:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.