0

I have around 1000 text files. 0001.txt 0002.txt .....to 1000.txt

Every text file have 50 direct image links. how i can batch download these images in there respective folders named 0001, 0002.

Also rename files to maintain downloaded order 1 to 50.

Example links https://xyz dot com/photos/hdi3bfkdn497ndbdj3.jpg . . .

1
  • Please note that superuser.com is not a free script/code writing service. If you tell us what you have tried so far (include the scripts/code you are already using) and where you are stuck then we can try to help with specific problems. You should also read How to Ask. – DavidPostill Mar 25 '20 at 9:26
2
#! /usr/bin/env bash

if [ -z $1 ] || egrep -qv '^https?:\/\/[^[:space:]]+' $1; then
  echo "Usage: $0 FILE"
  echo "FILE must be a newline-separated list of URLs."
  exit
fi

INPUT_FILE=$1
OUTPUT_DIRECTORY="${INPUT_FILE%.*}"

mkdir -p $OUTPUT_DIRECTORY

i=1
while read URL; do
  FILENAME="${URL##*/}"
  curl -L $URL > "${OUTPUT_DIRECTORY}/${i}_${FILENAME}"
  i=$(($i+1))
done < $INPUT_FILE

This script will download all the URLs in a given file into a directory named after the input file with the extension stripped.

[gnubeard@mothership: ~]$ cat example.txt
https://upload.wikimedia.org/wikipedia/commons/a/a9/Example.jpg
https://upload.wikimedia.org/wikipedia/commons/7/70/Example.png
[gnubeard@mothership: ~]$ ./download_from_file example.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  9022  100  9022    0     0   477k      0 --:--:-- --:--:-- --:--:--  489k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2335  100  2335    0     0   117k      0 --:--:-- --:--:-- --:--:--  120k
[gnubeard@mothership: ~]$ ls example
1_Example.jpg  2_Example.png

If you wanted to use this with, say, a thousand text files, you can use a for loop like so:

for FILE in $(ls *.txt); do ./download_from_file $FILE; done
7
  • @gnubread Hey, Thanks. it work for one txt file. But i don't able to do with multiple files. Right now say 50 files. Can you explain more of that loop thing? sorry noob – Roxion Apr 4 '20 at 4:39
  • The last part of my answer has an example of how you could call the script on an arbitrary number of files. What part of it is unclear to you? – gnubeard Apr 4 '20 at 4:48
  • ibb.co/9VFWxFf $sh script.sh 0001.txt works good, you can see in screenshot, it created folder 0001. I did it for three files. But how do it in batch all these file? – Roxion Apr 4 '20 at 5:38
  • Have you seen the last line of my answer? The last line of my answer is the way to apply the script to all the files in a directory. Instead of $ sh script.sh 0001.txt, the command $ for FILE in $(ls *.txt); do sh script.sh $FILE; done would do the trick. If you want more clarification, please make an actual, honest effort to understand and ask clear, specific questions about what confuses you. – gnubeard Apr 4 '20 at 5:42
  • Thanks. This works now. – Roxion Apr 5 '20 at 6:17

Not the answer you're looking for? Browse other questions tagged or ask your own question.