Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

There's this beautiful boy who periodically uploads pictures of himself to his website. I am trying to automate the process of downloading these images to my computer.

So far, I'm able to download his webpage and parse it for jpg files. I end up with a file like this.

I can get any of those images by doing a wget on them, but I would like to automate this process to get everything in the list. I tried piping and redirecting to wget, but it doesn't work. How can I accomplish what I'm trying to do?

share|improve this question
What have you already tried? – r4. Jan 30 '12 at 9:10
I have tried wget | list and wget < list. – tony_sid Jan 30 '12 at 9:12

You can use -i option of wget such as:

$ wget -i input_file.txt

You will get all files downloaded in the current directory. You can see man wget for more options.

share|improve this answer
Is there a way to tell wget to ignore images that are smaller than a certain size? – tony_sid Jan 30 '12 at 9:22
If the target is to remove the small files (less than some threshold), you can use find to delete them automatically after download. – Khaled Jan 30 '12 at 9:26

minsize="50" #grab all over 50kb

for x in $(cat list)
if [ "$(echo $(GET -Ssed $x | grep Length | awk '{print $2}') /128 |bc)" -ge $minsize ]; then
wget -q $x

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .