56

Let's say I want to download something with wget but the website that has the files I need redirects to a site which automatically chooses a mirror for me (and there's no static file URL provided).

Downloading from such sites works with Firefox, but how do I get real url (not only the link to the redirect page) so I could download these files with wget?

| improve this question | | | | |
  • 3
    Breakthrough's answer works, but what's the point then? Just let FF finish the download. Can you add more details to your question? I'm going to guess that you're trying to automate downloading via some sort of script, and not use wget just for gits-n-shiggles. – Yoopergeek Aug 21 '09 at 16:40
  • 2
    when you download with firefox right click on your download in downloads windows and select copy download link – Am1rr3zA Sep 10 '09 at 13:11
  • 2
    @Yoopergeek: His answer helps if the file is extremely big and you want to put it into a download manager. ;) – cp2141 Sep 10 '09 at 19:17
  • @Yoopergeek: The OP also mentioned "wget" which is a linux/unix command-line tool to transfer a file without any windowsy UI at all. Tools like this come in handy when you're running a virtual machine in the cloud and need to download software. Since wget is just trying to get a file, not display advertising while it waits for a mirror to be selected, the direct url is essential. – mightypile Jul 1 '14 at 23:29
  • 1
    Cliget addon gives you a curl command to copy and paste in the download dialog. addons.mozilla.org/en-US/firefox/addon/cliget – Dagelf Aug 9 '17 at 5:24

15 Answers 15

54

When you've started the transfer in Firefox, cancel it, and right click the download and hit "Copy download link". If you're using an older version, right click it and click on "Properties", and the link will be in the window which opens.

In Chrome - run download as normal - then go to Menu - Downloads - and you should see the direct link which was used. Or press Ctrl + J to open the window.

| improve this answer | | | | |
18

You can use the LiveHTTPHeaders extension to determine the actual URL of the file being downloaded. (Keep an eye on the GETs in particular.)

| improve this answer | | | | |
  • 1
    that's cool idea, i'll try this. i would vote up if not my low rep ;) – Phil Aug 21 '09 at 15:08
  • anything for chrome? – Anwar Sep 16 '15 at 10:12
  • I know this is an old thread but wanted to give my +1 for this response. The best solution is definitely acceptable for most cases. In mine, I have an inventory web gui that I wanted to easily export all my devices from in CSV format using their export button. To make this scriptable, I took your "keep an eye on GET" advice and checked the network tab in F12 to see the file that was being downloaded by export. From there, in chrome at least, you can right click that file and do copy > copy as cURL. I substituted the device's name as a variable in the link & put it in a for loop in bash to curl – user2856414 Aug 7 '19 at 15:06
16

While I like wget too, I use another similar tool: cURL. cURL specifically has a feature to follow redirects (-L / --location), and it's also free like wget. I suggest keeping both in your toolbelt; they have some complementary features.

Here's an interesting article I came across on how to use either wget or cURL to download from a site that uses cookies for authentication. There's a mention of the cURL --location feature to follow redirects. (In fact, based on no special flag mentioned for wget, it would seem to me that wget may follow redirects implicitly..?)

| improve this answer | | | | |
6

This is the answer you have been looking for!

There is only one fool-proof one-click solution: cliget

It gives you a curl command with headers, cookies and all, with a copy to clipboard button, right on the download dialog.

For Firefox: https://addons.mozilla.org/en-US/firefox/addon/cliget

I am not linking curlwget for Chrome here, because it's ugly. Please update this answer if Chrome gets an as elegant equivalent.

| improve this answer | | | | |
  • 1
    Thank you! It worked like a charm – mudassirkhan19 Sep 20 '19 at 13:04
  • 1
    Great this work well. – Ahdee Dec 16 '19 at 16:36
4

In those situations, First I starts the actual download and pause it and copy the URL.

| improve this answer | | | | |
  • 1
    copy it from where? – Phil Sep 10 '09 at 13:18
  • 1
    If you pause the download in the firefox download window before it completes, you can right-click and grab the URL (have done that to fire the URL on wget sometimes). – nik Sep 10 '09 at 13:19
3

I usually use the firefox DownThemAll addon when there are a lot of links to download and I need to select specific URLs.

It shows the full URL and allows you to add in paused mode so you can start the download when you like.

As an additional feature, you can grab the URL and skip the download altogether.
However, if you need to download the file, DownThemAll is quite good as an accelerator.

| improve this answer | | | | |
2

Can't you use wget directly using the --trust-server-name flag ? It will download and save using the filename it redirects to.

| improve this answer | | | | |
1

Your question is a bit unclear. There are 2 urls here : the one you use, and the one that came back after redirection. The first you should know, since it originated from your machine, while the second is the one that displayed in your browser. If the question pertains to where the file is on the redirected server, then there's no way to know.

If you're trying to debug the calls issued towards the server, you can use the firefox add-on of firebug, older versions of it exist here : https://addons.mozilla.org/en-US/firefox/addons/versions/1843.

| improve this answer | | | | |
  • 1
    honestly, i don't understand what's so unclear in my question, some posters below even actually understood ;) – Phil Sep 10 '09 at 13:24
  • If your only problem is just getting the url, then pausing the download only works for large files, unless you're very fast. You can much much simpler just find this information in the download manager's list : right-click and do Copy download link. – harrymc Sep 10 '09 at 13:55
  • To the downvoter: Don't waste your time here - this is a very old post and may not apply to you. – harrymc Aug 9 '17 at 16:06
1

You can use Curl to download a redirected URL:

curl --location http://domainName/fileName 2> /dev/null > fileName
| improve this answer | | | | |
0

You can use IDM (Internet Download Manager) when you click on the not direct download link it will redirect the link and showing up the direct link.

| improve this answer | | | | |
0

Have you tried to pass the redirect URL to wget? Sometimes they work for me.

Here a example from SourceForge:

$ wget http://sourceforge.net/projects/emule/files/eMule/0.49c/eMule0.49c-Installer.exe/download?use_mirror=ufpr,Centro%20de%20Computacao%20Cientifica%20e%20Software%20Livre
--14:23:20--  http://sourceforge.net/projects/emule/files/eMule/0.49c/eMule0.49c-Installer.exe/download?use_mirror=ufpr,Centro%20de%20Computacao%20Cientifica%20e%20Software%20Livre
Resolvendo sourceforge.net... 216.34.181.60
Connecting to sourceforge.net|216.34.181.60|:80... conectado!
HTTP requisição enviada, aguardando resposta... 302 Found
Localização: http://downloads.sourceforge.net/project/emule/eMule/0.49c/eMule0.49c-Installer.exe?use_mirror=ufpr [seguinte]
--14:23:21--  http://downloads.sourceforge.net/project/emule/eMule/0.49c/eMule0.49c-Installer.exe?use_mirror=ufpr
Resolvendo downloads.sourceforge.net... 216.34.181.59
Connecting to downloads.sourceforge.net|216.34.181.59|:80... conectado!
HTTP requisição enviada, aguardando resposta... 302 Found
Localização: http://ufpr.dl.sourceforge.net/project/emule/eMule/0.49c/eMule0.49c-Installer.exe [seguinte]
--14:23:21--  http://ufpr.dl.sourceforge.net/project/emule/eMule/0.49c/eMule0.49c-Installer.exe
Resolvendo ufpr.dl.sourceforge.net... 200.236.31.1, 200.17.202.1
Connecting to ufpr.dl.sourceforge.net|200.236.31.1|:80... conectado!
HTTP requisição enviada, aguardando resposta... 200 OK
Tamanho: 3342809 (3,2M) [application/x-msdos-program]
Saving to: `eMule0.49c-Installer.exe'

100%[====================================================================================>] 3.342.809   2,35M/s   in 1,4s

14:23:23 (2,35 MB/s) - `eMule0.49c-Installer.exe' saved [3342809/3342809]

The version I use:

$ wget --version
GNU Wget 1.10.2 
| improve this answer | | | | |
0

There was an addon for Firefox called Splitlink. It doesn't work for Firefox 3+, but it did work in Firefox 2. If you can find it anywhere (it's not on the official addons page anymore), it will give you more information about the real URL.

| improve this answer | | | | |
-1

The firefox copy url is definitely simplest. wget (as the question was tagged) is also a nice solution because you can see all the intermediate bounces, not just the final url:

$ wget 'http://sourceforge.net/projects/pdfcreator/files/PDFCreator/PDFCreator%200.9.8/PDFCreator-0_9_8_setup.exe/download'
--2009-09-10 09:59:53--  http://sourceforge.net/projects/pdfcreator/files/PDFCreator/PDFCreator%200.9.8/PDFCreator-0_9_8_setup.exe/download
Resolving sourceforge.net... 216.34.181.60
Connecting to sourceforge.net|216.34.181.60|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://downloads.sourceforge.net/project/pdfcreator/PDFCreator/PDFCreator%200.9.8/PDFCreator-0_9_8_setup.exe?use_mirror=voxel [following]
--2009-09-10 09:59:54--  http://downloads.sourceforge.net/project/pdfcreator/PDFCreator/PDFCreator%200.9.8/PDFCreator-0_9_8_setup.exe?use_mirror=voxel
Resolving downloads.sourceforge.net... 216.34.181.59
Connecting to downloads.sourceforge.net|216.34.181.59|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://voxel.dl.sourceforge.net/project/pdfcreator/PDFCreator/PDFCreator%200.9.8/PDFCreator-0_9_8_setup.exe [following]
--2009-09-10 09:59:54--  http://voxel.dl.sourceforge.net/project/pdfcreator/PDFCreator/PDFCreator%200.9.8/PDFCreator-0_9_8_setup.exe
Resolving voxel.dl.sourceforge.net... 74.63.52.167, 69.9.191.19, 69.9.191.18, ...
Connecting to voxel.dl.sourceforge.net|74.63.52.167|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 17695920 (17M) [application/octet-stream]
Saving to: `PDFCreator-0_9_8_setup.exe'

10% [=======> ] 1,818,064 609K/s

| improve this answer | | | | |
-1

If you want a download manager to handle the download instead of firefox, you could use the Flashgot firefox addon.

| improve this answer | | | | |
-1

You could use the netcat tool (which is often included with linux distributions) on port 80 of the website. This shows the full raw HTTP response, including the headers which contain the redirect destination URL. This is one of the most direct ways to get the redirect location, since you are doing the same thing as downloaders do when they follow redirects. Here is an example for the URL http://www.example.com/example.

printf 'GET /example HTTP/1.1\r\nHost: www.example.com\r\n\r\n' | netcat www.example.com 80

...or...

Alternatively, many browsers show the actual download URL in the Downloads section. In these, you can start the download and cancel it right away, then copy the URL (from the download history).

| improve this answer | | | | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.