Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Forgive me for the simple question... I haven't used curl before. I'm trying to automatically download some data from this form:

which posts to available_records_process.cfm. I would use:

curl -F "fromdate=1988-08-26;todate=2009-09-30;rtype=3;site_no=12040500"

but the webmaster has limited access to available_records_process.cfm, so I can't post to it directly (I get a 'specified URL could not be found' error). Next, I tried to use curl directly on the preceding page, like:

curl -F "fromdate=1988-08-26;todate=2009-09-30;rtype=3"

But this doesn't seem to post the form, despite trying many variations of flags and post formatting. Any thoughts?

share|improve this question

You might need to use --data-urlencode but this will only work with POST requests. For GET requests you simply append the params separated by & to the URL, e.g. ...available_records.cfm?sn=12040500&fromdate=1988-08-26&todate=2009-09-30&rtype=3

share|improve this answer
Thanks for the tip. I'm still having trouble POSTing to the second .cfm (available_records_process). Using cookies seems to at least get a response: curl -d "site_no=12040500" -D mycookies and then curl -d "fromdate=1988-08-26&todate=2009-09-30&rtype=3&submit1:Retrieve+Data" -b mycookies Returns "JRun Servlet Error" rather than "URL not found". I'm still stuck on getting a valid POST through to it, though. – Stochtastic Jan 21 '12 at 17:02
The key is to emulate exactly what the browser does. Use Firefox+Firebug or Chrome+F12(Network) to see exactly what the browser sends. Also you could try using WireShark to compare the requests sent by curl and by browser. – ccpizza Jan 21 '12 at 19:13
Aha, digging through it with firebug did it for me. I'd forgotten to include --referer http://..., which was indeed included in the POST. Thanks! – Stochtastic Jan 23 '12 at 22:33

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.