Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

There's this beautiful boy who periodically uploads pictures of himself to his website. I am trying to automate the process of downloading these images to my computer.

So far, I'm able to download his webpage and parse it for jpg files. I end up with a file like this.

http://stat.ameba.jp/user_images/20120129/19/maofish/f9/60/j/o0480064011762693689.jpg
http://imgstat.ameba.jp/view/d/70/stat001.ameba.jp/user_images/20120127/22/maofish/f7/3e/j/t02200293_0480064011759076335.jpg
http://imgstat.ameba.jp/view/d/70/stat001.ameba.jp/user_images/20120125/18/maofish/80/46/j/t02200293_0480064011755033425.jpg
http://imgstat.ameba.jp/view/d/70/stat001.ameba.jp/user_images/20120120/20/maofish/3c/99/j/t02200290_0480063311745603530.jpg
http://stat.ameba.jp/user_images/20100219/16/maofish/33/0b/j/t01400198_0140019810420649113.jpg
http://stat.ameba.jp/user_images/b0/09/10101851128_s.jpg
http://stat.ameba.jp/user_images/9c/26/10027225053_s.jpg

I can get any of those images by doing a wget on them, but I would like to automate this process to get everything in the list. I tried piping and redirecting to wget, but it doesn't work. How can I accomplish what I'm trying to do?

share|improve this question
    
What have you already tried? –  r4. Jan 30 '12 at 9:10
1  
I have tried wget | list and wget < list. –  tony_sid Jan 30 '12 at 9:12

2 Answers 2

You can use -i option of wget such as:

$ wget -i input_file.txt

You will get all files downloaded in the current directory. You can see man wget for more options.

share|improve this answer
    
Is there a way to tell wget to ignore images that are smaller than a certain size? –  tony_sid Jan 30 '12 at 9:22
1  
If the target is to remove the small files (less than some threshold), you can use find to delete them automatically after download. –  Khaled Jan 30 '12 at 9:26

minsize="50" #grab all over 50kb

for x in $(cat list)
do
if [ "$(echo $(GET -Ssed $x | grep Length | awk '{print $2}') /128 |bc)" -ge $minsize ]; then
wget -q $x
fi
done

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.