Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I want to copy a local disk image to another computer in the local network. It is a large directory (around 15 GB) and to speed it up I use this little trick:

cd /Volumes/myserver
tar -cvzf image-dir.gz  /Users/francis/image-dir/

For the unpacking I can use:

cd /
tar -xvzf /Volumes/myserver/image-dir.gz

The idea is improve performance by sending only compressed data over the network. However, it's not clear if tar actually does the compression before or after the data has been sent over the network. Does anyone know?

Alternative solutions are also welcome.

share|improve this question

2 Answers 2

up vote 2 down vote accepted

How effective this is depends on where your commands are executed, and where the data is.

Assuming:

  1. /Volumes/myserver is a network share,
  2. /Users/francis/image-dir is on the client's local drive, and
  3. both creation and extraction commands are being executed on the client,

I think you're correct -- no uncompressed data is crossing the network.

  • In the create case, you're reading uncompressed data from the local drive, compressing it, and writing the compressed data to the network drive. Network transmission is compressed.

  • In the extract case, you're reading compressed data from the network drive, uncompressing it, and writing the uncompressed data to the local drive. Again, network transmission is compressed.

share|improve this answer
    
and this is just analyzing your current process with an eye towards the network data transfer; this is not to say another process won't be more efficient/faster/easier. –  quack quixote Jan 22 '10 at 14:33
    
sounds ok for me. –  akira Jan 22 '10 at 14:47

I've seen friends pipe tar output over netcat but I'm at a loss for the syntax. The man pages and a quick web search should be of some help.

share|improve this answer
    
receiver first: netcat -l -p 7000 | tar x ... then sender: tar cf - /Users/francis/image-dir | netcat otherhost 7000 –  quack quixote Jan 22 '10 at 18:05
    
caveat: "You'll probably find your hard disk drive is a bottleneck on a fast network. Compression is useless if you can't saturate your network and will probably just slow things down." source: compsoc.dur.ac.uk/~djw/tarpipe.html –  quack quixote Jan 22 '10 at 18:06
    
@quack: I can send data at maximum 12 MB/s over my network. My hard disk is probably much faster than that. Anyway I should time the difference and see for myself.. –  StackedCrooked Jan 22 '10 at 21:46
    
@StackedCrooked: that syntax example doesn't do compression over necat; just add the 'z' or 'j' flags to both tar commands for that. –  quack quixote Jan 23 '10 at 7:48

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.