34

Can you scp a file to multiple locations in the same command?

For example:

scp file.txt user@ip-1.com:, user@ip2.com:

Or would it be more practical to create a bash script that has all the hosts already in it and it would just take a file as the argument?

8 Answers 8

28

Let's say you have a file (destfile.txt) with user@host-values, one on each line. Then you could do like this:

while IFS= read -r dest; do
  scp ourfile.txt "$dest:remote/path/"
done <destfile.txt
3
  • This works! Still a handful to type out though. Perhaps there's a way to simplify that.
    – Andrew
    Sep 22, 2011 at 21:02
  • 3
    @Andrew, if you change the for-loop into while read dest; do, it will read from standard input. Put it in a script and feed the destfile.txt into it (e.g., ./thescript.sh <destfile.txt). Sep 22, 2011 at 21:19
  • I'm doing the former, but with rsync and some of my nodes are not updating. Any idea why?
    – Soubriquet
    Sep 22, 2018 at 17:50
9

Looks like a job for parallel-scp(n)(t) - this implements a set of commands that allow for scp commands to be run on multiple systems at once. It will allow for the copying of files in parallel to a set of machines.

4
cat file.txt | tee >(ssh user@ip1.com "cat > file.txt") \
                   >(ssh user@ip2.com "cat > file.txt")

tar cz file1 file2 file3 | tee >(ssh user@ip1.com "tar xz") \
                               >( ... )
1

If you have a consistent naming convention going on for multiple servers, you can do something neat like:

for x in st1-abc-{11..20}.acme.com; do scp filez.tgz user@$x; done
1

You can also use xargs.

hosts.txt :

user@ip-1.com:
user@ip2.com:

Then run this command :

cat hosts.txt | xargs -I HOST scp file.txt HOST
0

Another alternative (and a one-liner) would be to instead use pdsh to connect to each target node and trigger a get from there:

pdsh -w^destfile.txt scp hostname:/path/to/file /path/to/destfile

This of course requires one more information (the local host) and different user rights, but you avoid looping in bash and using indirection to read the file.

0

Here's another option, with a one line shell script.

cscp.sh 20337.patch < hosts.txt

It uses two files, one for the loop, and one for the server host list. It reads $1 for the first parameter from the CLI as the filename to SCP

cscp.sh

#!/bin/bash
while read host; do
  scp $1 ${host}:
done

hosts.txt

project-prod-web1
project-prod-web2
project-prod-web3

Usage

Copy file to multiple hosts:

cscp.sh file < hosts
0

You can use pssh or parallel-scp packages for ubuntu

http://manpages.ubuntu.com/manpages/trusty/man1/parallel-scp.1.html http://manpages.ubuntu.com/manpages/trusty/man1/parallel-ssh.1.html

1
  • Welcome to Super User! Please quote the essential parts of the answer from the reference link(s), as the answer can become invalid if the linked page(s) change.
    – DavidPostill
    Jul 3, 2020 at 12:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.