5

I want to download content from a site. Originally, I used curl/wget, however, the page demanded a session cookie + had multiple redirects. The page administrator then told me to use Firefox to download the content. This is of course pretty difficult because I want to call the download from a script.

I figured out that

firefox https://url.to.page.com/download_content

works and the download file dialog appears. So I am almost there. Of course, as I want to call the download from a script I don't want to press the "ok" button but download it instantly. Are there any additional command line settings to surpress the download file dialog in firefox and download the file immediately?

Improve this question
  • 1
    Thanks for your suggestion, however, using additional programmes is not possible... – Syd Amerikaner May 6 '18 at 14:45
  • Does Using curl with cookies help? – Andrew Morton May 6 '18 at 15:00
  • No, it does not work. The webpage adminstrator I contacted also sounded as it was very difficult to realize it with curl/wget and he recommended to do it with Firefox. – Syd Amerikaner May 6 '18 at 15:07
  • So you don't have to log into an account to download the file? Then curl might indeed work. Or maybe a python script. – SpiderPig May 6 '18 at 15:31
  • Nope. You can try it yourself, the url is piratenpad.de/p/qwq (export url: piratenpad.de/p/qwq/export/txt). curl -L -b cookies.txt the_url doesnt work; calling it with firefox gives the above described behavior – Syd Amerikaner May 6 '18 at 21:30
3

Solved the same problem as follows, using Firefox:

  • Open "Network" tab of "Web Developer" tool: Ctrl-Shift-E
  • Visit the page you want to save (e.g. a photo behind a login)
  • Right click the request and choose 'Copy'->'Copy as cURL'

This will give you a command that has all your cookie credentials e.g.

curl 'https://mysite.test/my-secure-dir/picture1.jpg' \ 
-H 'User-Agent: Mozilla/5.0 ...' \
-H 'Cookie: SESSIONID=abcdef1234567890'

You can modify the URL in the command to fetch whatever you want and run it in your shell.

Improve this answer
-1

As already stated above, you can simply use the 'Copy cURL-Adress function' of the webdevolper tools in Firefox/Chrome/Safari.

For those who don't know where to find the specified option in their browser, here's an article on where to find it exactly (in all mentioned browsers):

https://ec.haxx.se/usingcurl-copyas.html

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.