Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Note that I can't first store the file locally -- it's too big.

This (obnoxious) page (scroll all the way to the bottom) seems to give an answer but I'm having trouble disentangling the part that's specific to tape drives:

http://webcache.googleusercontent.com/search?q=cache:lhmh960w2KQJ:www.experts-exchange.com/OS/Unix/SCO_Unix/Q_24249634.html+scp+redirect&cd=3&hl=en&ct=clnk&gl=us

To make this more concrete, here's how you would think it might work:

On local machine:

% echo "pretend this string is a huge amt of data" | scp - remote.com:big.txt

(That's using the convention -- which scp does not in fact support -- of substituting a dash for the source file to tell it to get it from stdin instead.)

share|improve this question

migrated from stackoverflow.com Jun 2 '11 at 0:39

This question came from our site for professional and enthusiast programmers.

    
Can you post the url of your google result? Experts Exchange only shows the answer at the bottom if your referrer is google... –  Jon Jun 21 '10 at 19:52
    
Thanks for pointing that out; I changed the link. –  dreeves Jun 21 '10 at 20:00
add comment

4 Answers 4

You can pipe into ssh and run a remote command. In this case, the remote command is cat > big.txt which will copy stdin into the big.txt file.

echo "Lots of data" | ssh user@example.com 'cat > big.txt'

It's easy and straightforward, as long as you can use ssh to connect to the remote end.

You can also use nc (NetCat) to transfer the data. On the receiving machine (e.g., host.example.com):

nc -l 1234 > big.txt

This will set up nc to listen to port 1234 and copy anything sent to that port to the big.txt file. Then, on the sending machine:

echo "Lots of data" | nc host.example.com 1234

This command will tell nc on the sending side to connect to port 1234 on the receiver and copy the data from stdin across the network.

However, the nc solution has a few downsides:

  • There's no authentication; anyone could connect to port 1234 and send data to the file.
  • The data is not encrypted, as it would be with ssh.
  • If either machine is behind a firewall, the chosen port would have to be open to allow the connection through and routed properly, especially on the receiving end.
  • Both ends have to be set up independently and simultaneously. With the ssh solution, you can initiate the transfer from just one of the endpoints.
share|improve this answer
    
Dang. Out-answered by a minute! –  Barry Brown Jun 21 '10 at 19:59
    
If it's any consolation, I'm inclined to mark yours accepted since it explains nicely what's actually going on. (If you wanted to really clinch it, you could include the FIFO pipe and netcat solutions with some guidance on why you might prefer one or the other! :) –  dreeves Jun 21 '10 at 20:06
    
Done. Netcat is a handy utility. :) –  Barry Brown Jun 21 '10 at 20:37
    
As in the other comment, if you are piping through tar, you can use a process substitution: tar -cvzf >(ssh destination 'cat > file') huge_directory_tree –  Taywee Jul 6 '13 at 0:49
add comment

Using ssh:

echo "pretend this is a huge amt of data" | ssh user@remote.com 'cat > big.txt'
share|improve this answer
    
Ah, beautiful, thank you! Any reason to prefer this or FIFO pipe solution? –  dreeves Jun 21 '10 at 19:58
    
I think the mknod approach accomplishes the task in exactly the same way except for the named pipe. –  BipedalShark Jun 21 '10 at 20:03
add comment

Use a FIFO pipe:

mknod mypipe p
scp mypipe destination &
ls > mypipe
share|improve this answer
    
I couldn't get this to work on a Linux system. scp complained that mypipe was not a regular file. –  Barry Brown Jun 21 '10 at 20:05
1  
Didn't work on a Mac, either, for the same reason. (I had to use mkfifo to create the pipe, though.) –  Barry Brown Jun 21 '10 at 20:24
    
Didn't work here either. But if it does work for you, and you have bash or zsh, you could better achieve this with a process substitution, like for this example: scp <(ls) destination –  Taywee Jul 6 '13 at 0:48
add comment

Use nc (Net Cat), which doesn't need to save the file locally.

share|improve this answer
    
Ah, thanks! Want to include the equivalent of my "echo .. | scp .." example? And any reasons you know of to prefer this to the other answers? –  dreeves Jun 21 '10 at 20:09
    
Actually, I like Barry Brown's answer. –  mcandre Jun 21 '10 at 20:28
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.