GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

learn more… | top users | synonyms

88
votes
17answers
137k views

What is the Windows equivalent of Wget?

How can I download something from the web directly without Internet Explorer or Firefox opening Acrobat Reader/Quicktime/MS Word/whatever? I'm using Windows, so a Windows version of Wget would ...
10
votes
8answers
23k views

wget/cURL alternative native to Windows?

Is there a wget or cURL type command line utility native to Windows Vista? How does it work?
6
votes
2answers
17k views

How to download list of files from a file server?

How would I download a list of files from a file server like this one http://www.apache.org/dist/httpd/binaries/ ? I suppose I could use wget but then it tries to get all the links and the html file ...
7
votes
1answer
9k views

Make wget convert HTML links to relative after download if -k wasn't specified

So the -k option (or --convert-link) will convert links in your web pages to relative after the download finishes, such as the man page says: After the download is complete, convert the links in ...
4
votes
10answers
11k views

Where on my computer did Wget download this image?

I am using Windows XP. I just downloaded wget and put it into my C:\Windows folder. I then ran this command line (trying to download an image from a website): wget -r -A ".jpg,.gif,.png" ...
1
vote
2answers
396 views

WGet a Page that Requires Logging in

I’m trying to figure out a way to use WGET or a similar tool so that I can schedule a web page to be downloaded regularly as a sort of updating log. The problem is that the page requires that I be ...
25
votes
3answers
35k views

Save a single web page (with background images) with Wget

I want to use Wget to save single web pages (not recursively, not whole sites) for reference. Much like Firefox's "Web Page, complete". My first problem is: I can't get Wget to save background images ...
5
votes
5answers
9k views

How do you use WGET to mirror a site 1 level deep, recovering JS, CSS resources including CSS images?

Pretend I wanted a simple page copy to be downloaded to my HD for permanent keeping. I'm not looking for a deep recursive get, just a single page, but also any resources loaded by that page to be also ...
0
votes
2answers
3k views

How to download a file behind authentication via wget or curl [duplicate]

Possible Duplicate: WGet a Page that Requires Logging in I wish to download file from http://mbr.nlm.nih.gov/Download/MetaMapped_Medline/2011/ however in order to download the file, i need ...
3
votes
2answers
2k views

Persistent retrying resuming downloads with curl

I'm on a mac and have a list of files I would like to download from an ftp server. The connection is a bit buggy so I want it to retry and resume if connection is dropped. I know I can do this with ...
2
votes
1answer
245 views

Download an entire sites jpegs with wget instead of just thumbnails

When I try to pull all the jpegs off of this website (http://www.zodiackillerfacts.com/gallery/) using wget: wget -A jpg,jpeg -r http://www.zodiackillerfacts.com/gallery/ it only downloads the ...
21
votes
14answers
35k views

How to find out the real download URL on download sites that use redirects

Let's say I want to download something with wget but the website that has the files I need redirects to a site which automatically chooses a mirror for me (and there's no static file URL provided). ...
2
votes
5answers
7k views

How can I download an entire (active) phpbb forum?

One of the forums that I frequent (and have added a LOT of quality content too) seems to be having problems with their server. I am not confident in their ability to sort out the problems they are ...
8
votes
2answers
1k views

How to download parts of the same file from different sources with curl/wget?

I have a quite large file hosted on five different servers. I would like to be able to download different parts of the file from each server and subsequently concatenate the parts, in order to ...
7
votes
3answers
10k views

How can I make wget rename downloaded files to not include the query string?

I'm downloading a site with wget and a lot of the links have queries attached to them, so when I do this: wget -nv -c -r -H -A mp3 -nd http://url.to.old.podcasts.com/ I end up with a lot of files ...
3
votes
1answer
2k views

Downloading multiple files, and specifying output filenames with wget

I'm using wget with the -i option to download a list of files from a URL. However, I want to specify the names that these files will be saved with as well. I see you can do that with a single file ...
3
votes
8answers
5k views

How to recursively download an entire web directory?

i have a web directory that has many folders and many sub folders containing files. i need to download everything using wget or bash.
2
votes
2answers
430 views

Batch download pages from a wiki without special pages

From time to time I find some documentation on the web that I need for offline use on my notebook. Usually I fire up wget and get the whole site. Many projects however are now switching to wikis, and ...
7
votes
2answers
3k views

Call but not download a page with wget

I was wondering if there was any argument that allowed me to use wget and "call" a page, but without downloading the page. The problem is that when you call wget on a page, it downloads it to the ...
5
votes
3answers
9k views

How to set http proxy address for wget under windows?

If ran without parameters my wget prints: D:\>wget SYSTEM_WGETRC = c:/progra~1/wget/etc/wgetrc syswgetrc = c:/progra~1/wget/etc/wgetrc D:\Apps\Util\wget: missing URL Usage: D:\Apps\Util\wget ...
3
votes
5answers
1k views

Why does wget'ing an image give me a file, not an image?

I do wget http://services.runescape.com/m=itemdb_rs/3809_obj_sprite.gif?id=2 and it downloads the following file: 3809_obj_sprite.gif?id=2 How can I make it save as the following? 2.png
1
vote
1answer
190 views

How to mirror a website on linux?

There is this abandoned blog witch can disappear any moment, and I want to make a browsable copy this site. The saved pages should contain correct on-page links, liked images and other files like ...
1
vote
1answer
5k views

Downloading only images using curl or wget?

UPDATED: I've found using this Bash script fixes the problem of having GIF files with a .jpg extension. I'm attempting to download images from a forum who's url uses the following format: ...
0
votes
1answer
667 views

How to download images using wget from a txt file that contains links

I can download images using wget if I download from a website. But I have several links and I have saved them in a text file. For example: wget -r -A.jpg -np www.fragrancenet.com There are so many ...
6
votes
3answers
1k views

Make wget download page resources on a different domain

How do you use wget to download an entire site (domain A) when its resources are on another domain, (domain B)? I've tried: wget -r --level=inf -p -k -E --domains=domainA,domainB http://www.domainA
5
votes
3answers
2k views

wget not converting links

I am trying to mirror a fairly large site (20,000+ pages) prior to a major overhaul. Basically, I need a backup before cutting over to the new one in case we forgot something we need (we'll have ...
1
vote
2answers
38 views

Wget - Difficulty Excluding Domains and Directories

I am using Wget in Windows 7. I believe this restricts me to Wget 1.11.4. For anyone interested in the details, I have posted a sample wgetrc file. This sample is actually the one I am currently ...
1
vote
2answers
175 views

wget: different speed in sudo and user mode

If I execute wget from my terminal to download something, it realices multiple attempts to download it, and eventually it downloads it (or not!). For example, with three attempts and 10 seconds of ...
1
vote
1answer
2k views

Exclude list of specific files in wget

I am trying to download a lot of pages from a website on dial-up and it can be brutally slow. I have almost got the perfect wget command, but because I'm downloading pages from the same site wget ...
0
votes
1answer
511 views

How to mirror ENTIRELY a website with wget [duplicate]

I really need the command to download everything from a website. It is arnos.gr and has a lot of content on it which I want to browse locally without Internet I have an account for login
0
votes
1answer
90 views

Generate a single PDF from a websites HTML pages

Here's the problem: There's a website that I often need for reference and I would like an offline version that also works in mobile devices, a PDF comes to mind. I can make an offline copy of the ...
0
votes
1answer
302 views

using wget to download from website with case-sensitive URLs to Windows

I used Gnuwin32 wget to download emacs manuals with this command (takes about 30 minutes): wget --mirror --page-requisites --convert-links --no-parent --accept .html,.htm,.css,.js ...
0
votes
1answer
1k views

Downloading multiple files with wget and handling parameters

How can I download multiple files using wget? I also want to rename the files. Here are the commands I'm running one by one (copy/paste on terminal): wget -c --load-cookies cookies.txt ...
0
votes
1answer
970 views

wget - save all data (images) from given directory?

I want wget to scan for sub and sub-subdirectories of: domain.com/profile/username/albums/ Then grab every .jpg file from their sources. Wget should get files like: ...
-2
votes
1answer
197 views

How to move a file of 250 GB to another server? [closed]

The hard disk of one of my servers is full, so I need to move a file away from it to make room. I have tried tar -cvf but when I send it to the other server using wget -c, I see it is 100% complete. ...