How to automatically get images from an SSL Site that uses Java Servlets

Question

This is tricky. I want to use curl, wget or any other tool to login into a SSL Website which provides a Login Form. Then I want to visit several Links within that Domain and fetch certain images.

I got it to work with this in bash:

curl -c /tmp/cookie.txt -d "login=username&password=passw&send=submit" https://URI

Use the cookie later with

curl -b /tmp/cookie.txt https://URI

The trick was that you have to submit the credentials to the action= adress of the html form field.

Another problem I face now is that this writes no image because the URL of the image is constructed out of some Servlet URI:

<img src="URI/servlet/manyParametersWith?And=AndLotsOf&">

Answer 1

Try using Python and mechanize (available for Perl too). You can do something like this:

import mechanize
br=mechanize.Browser()
br.open('http://www.yourfavoritesite.com')
br.select_form(nr=0) #check yoursite forms to match the correct number
br['Username']='Username' #use the proper input type=text name
br['Password']='Password' #use the proper input type=password name
br.submit()
br.retrieve('https://www.yourfavoritesite.com/pagetoretrieve.html','yourfavoritepage.html')

Answer 2

Ok only possibility to write the image is to display it in a browser, for e.g. echo via php and then write it out to disk via php. Some users reported that might work.