I am trying to mirror a website.

It follows a simple scheme of .htm files but there is one catch. Each file contains the following line:

<img width="100%" src="http://DOMAIN.org/CREATE_IMAGE.php">

I need to download this image, but httrack and wget both ignore the script outputs when downloading a file.

The php script does not take any parameters. It determines which image to output internally. Each .htm file gets a different output. So I can't simply call the php file. It needs to be downloaded as some kind of dependency of the corresponding .htm file.

How can I configure wget or httrack or really any other tool to get what I need?


You can't download and run the .php since it is supposed to run server-side.

If the website is such as:

<img width="100%" src="http://DOMAIN.org/CREATE_IMAGE.php">

When you get the page source you can't actually see that (client-side). What you should see is:

<img width="100%" src="http://DOMAIN.org/IMAGE_LINK_GENERATED_BY_THE_PHP_SCRIPT">

You could then wget the html, parse it, and download the image. You have to either know the pattern of the filename or the possible extensions. If you don't, you can iterate over a list of possible extensions. For only one image it should look something like this:

# Get the html for us to parse
wget http://DOMAIN.org/index.html -O index.html
# List of common image extensions
exts=("png" "jpg" "jpeg" "bmp" "gif")
for ext in ${exts[@]}
    # Parse the html looking for an image
    # You'll have to adapt the regex, of course
    # man grep to see what each letter stands for
    img=`grep -shoiP "DOMAIN\.org/.+\.${ext}" index.html`
    if [ "$img" = "" ]
wget $img

Of course you'll have to adapt that.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.