10

I want to download content from a site. Originally, I used curl/wget, however, the page demanded a session cookie + had multiple redirects. The page administrator then told me to use Firefox to download the content. This is of course pretty difficult because I want to call the download from a script.

I figured out that

firefox https://url.to.page.com/download_content

works and the download file dialog appears. So I am almost there. Of course, as I want to call the download from a script I don't want to press the "ok" button but download it instantly. Are there any additional command line settings to surpress the download file dialog in firefox and download the file immediately?

Improve this question
5
  • 1
    Thanks for your suggestion, however, using additional programmes is not possible...
    – Syd Amerikaner
    May 6, 2018 at 14:45
  • Does Using curl with cookies help?
    – Andrew Morton
    May 6, 2018 at 15:00
  • No, it does not work. The webpage adminstrator I contacted also sounded as it was very difficult to realize it with curl/wget and he recommended to do it with Firefox.
    – Syd Amerikaner
    May 6, 2018 at 15:07
  • So you don't have to log into an account to download the file? Then curl might indeed work. Or maybe a python script.
    – Tesseract
    May 6, 2018 at 15:31
  • Nope. You can try it yourself, the url is piratenpad.de/p/qwq (export url: piratenpad.de/p/qwq/export/txt). curl -L -b cookies.txt the_url doesnt work; calling it with firefox gives the above described behavior
    – Syd Amerikaner
    May 6, 2018 at 21:30

2 Answers 2

Reset to default
10

Solved the same problem as follows, using Firefox:

  • Open "Network" tab of "Web Developer" tool: Ctrl-Shift-E
  • Visit the page you want to save (e.g. a photo behind a login)
  • Right click the request and choose 'Copy'->'Copy as cURL'

This will give you a command that has all your cookie credentials e.g.

curl 'https://mysite.test/my-secure-dir/picture1.jpg' \ 
-H 'User-Agent: Mozilla/5.0 ...' \
-H 'Cookie: SESSIONID=abcdef1234567890'

You can modify the URL in the command to fetch whatever you want and run it in your shell.

Improve this answer
1
  • This will not work if the target web site uses Cloudflare protection or a similar system that uses TLS fingerprinting.
    – Robert
    Apr 21, 2022 at 7:36
-1

As already stated above, you can simply use the 'Copy cURL-Adress function' of the webdevolper tools in Firefox/Chrome/Safari.

For those who don't know where to find the specified option in their browser, here's an article on where to find it exactly (in all mentioned browsers):

https://ec.haxx.se/usingcurl-copyas.html

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .