Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm trying to use cURL to download the contents of webpages that require forms to get to.

In a browser it looks something like this

1.) Login using POST
2.) Pick which page to go to using another POST
3.) Pick another page... using POST
4.) etc.. until I get to the page I want, 
    then download all textfiles linked to on that page.

I am attempting to do this using a bash script and some loops with the values that change for each POST.

My problem is how do I do multiple POSTs with cURL? Must there be cookies involved?

FYI the website is http://metagenomics.nmpdr.org/ (MG-RAST).

share|improve this question
add comment

1 Answer

I'm beginning to realize that I only post questions to stack* because I know that as soon as I do, I will figure out the answer.

The answer is indeed cookies:

$ curl -d "who=your_mom&when=last_night" -D mycookies http://outbacksteakhouse.com/date.cgi
$ curl -d "what=took_her_out_for_a_steak" -b mycookies http://outbacksteakhouse.com/date.cgi

-D saves to mycookies
-b loads mycookies

Add another -D to the second command and you might make it to 2nd base.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.