GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

learn more… | top users | synonyms

3
votes
2answers
4k views

wget -o writes empty files on failure

If I write wget "no such address" -o "test.html" it first creates the test.html and in case of failure, leaves it empty. However, when not using -o, it will wait to see if the download succeeds and ...
0
votes
3answers
914 views

Problems with Wget to a CloudFlare hosted site: 503 Service Unavailable

I have seen other instances of 503 errors using Wget, but to no available I cannot solve this. When I try to download a certain website, I get a 503 Service Unavailable error. This does not happen to ...
1
vote
2answers
3k views

Download entire directory from a website using Powershell

I have been using the following command to pull back files from a web server and store them on my local machine: $client = New-Object System.Net.WebClient ...
-1
votes
1answer
18 views

why wget doesnt download full size of zip file? [on hold]

i uploaded a zip file to mediafire and now am trying to download it into my host in koding.com. the original zip file is 868KB but after wget downloads the file the file size is 93.500. the file name ...
1
vote
0answers
17 views

403 when downloading from cloudfront with wget (after 302)

I'm trying to download file from cloudfront. It is a file archive from scala-sbt website. When using browser (Chrome) everything is ok. Basically the flow looks like: ...
0
votes
0answers
10 views

Not able to download all files from nexus to git repository folder with wget

I am downloading files from the nexus repo using wget command. Using the command, I am able to download all the files in the C:/ area, but when I am using the same command inside ...
0
votes
0answers
6 views

Axel usage with script on UBUNTU

I write a shell-script to download files via axel but always I get a 403 FORBIDDDEN error. At the same time I copied the axel command which generated by my script to terminal manually and it works ...
0
votes
0answers
39 views

wget through proxy in virtual box (cloudera quickstart VM - Cent OS)

I have installed the Cloudera Quickstart VM 5.3.0 running on virtual box for windows 7 and I would like to run a wget command but I have troubles with my company proxy (which needs authentification). ...
3
votes
1answer
3k views

How can I download private repo from github?

What I want to do is to download private repository archive from GitHub, extract it, remove archive file and copy some directories that are inside downloaded project. I tried to use wget but I cannot ...
0
votes
1answer
19 views

apple store showing different page in wget

istore.apple.com live url is different that url which is downloaded using wget Open this url in browser https://itunes.apple.com/us/genre/ios-games-action/id7001?mt=8&letter=A You will see ...
2
votes
2answers
94 views

Fast way to download many small webpages?

I need to download lots of small pages as fast as possible. Average size for one page is 3kb. There are millions of pages. First, I tried curl but it was way too slow, probably because it was waiting ...
1
vote
0answers
17 views

Make static site from wordpress site using bash

I have tried a few other scripts that can make a static version of a wordpress site but none of them are designed for bulk sites and I haven't found any that will just serve a static version while ...
1
vote
0answers
39 views

How do I download an entire folder with aria2?

Aria2 is a great wget replacement, having a lot of features like multi-threading, split download,download resume, etc But having all this features and option can make it hard to do some specific ...
2
votes
1answer
36 views

Archiving websites like Internet Archive

Most of the data for my research needs are met by our Government websites. Problem is they take down websites or certain pages or break the links. But my research demands the access to information ...
0
votes
0answers
20 views

wget a whole website, no sound

My friends showed me a site and I want to download this site for offline use, in basic .html format Using the command: wget -r -k -p "www.flappyplane.net" I was able to obtain the site for offline ...
0
votes
1answer
36 views

Using Wget to download files with a specific name off a site

What would be the best way to look down through a (semi) top level domain and all it's subfolders for occurrences of certain number letter combinations. I have several folders with large lists of ...
1
vote
1answer
62 views

I can ping but cannot download from the internet

When using the system proxy, firefox has no access to internet. After changing the proxy to auto, firefox downloads fine. In the terminal, when I tried to download using wget, the ternimal stopped at ...
4
votes
1answer
64 views

Wget - Saving a file to a local directory named after link

I'm new to using wget, so I hope this is not a dumb question : How can I save a file to a local directory that has been named after the link to the file ? It does not seem to work if a redirection ...
0
votes
1answer
15 views

Wget having trouble just saving the files I want - exclude directories doesn't seem to work

I want to download all government spending over £500 by the Department of Energy and Climate change. These are .xls and .xlsx files, generated once per month. They are stored at locations like this: ...
0
votes
2answers
498 views

wget stops downloading: Cannot write to `-' (success)

Please have in mind that I don't know anything about servers, nor any of its vocabulary, I am a simple web coder, trying to figure my way on the universe! I'm trying to link dropbox on my server, I am ...
0
votes
0answers
52 views

Why do wget and curl fail to download page source?

Via a shell script, I would like to download the source for a web page with air quality data on aqicn.org. Here is the URL for the page with Beijing air quality data: http://aqicn.org/city/beijing/m/ ...
3
votes
1answer
2k views

How can I force wget.exe to talk HTTP/1.1?

I'm talking about Windows, not Linux. I tried: wget -O X:/test/some/dir/file.txt --tries=3 -c http://some.com/dir/data.txt wget -O X:/test/some/dir/file.txt --tries=3 http://some.com/dir/data.txt ...
2
votes
4answers
369 views

Download forum with wget with username and password?

I want to download a forum where I only have access via my username and my password. I tried the following: C:\wget.exe wget -k -m -E -p -np -R ...
5
votes
1answer
4k views

How do I use Firefox cookies with Wget?

wget --load-cookies will load cookies as a "textual file in the format originally used by Netscape's cookies.txt file". However, Firefox keeps its cookies in an SQLite database. Is there a way to ...
1
vote
1answer
37 views

How to read Telegraph's rss feed?

According to Telegraph.co.uk, the world news rss feeds should be this one: http://www.telegraph.co.uk/news/worldnews/rss However, with Chrome I only get a blank page. Even wget ...
0
votes
1answer
40 views

Downloading images via wget

When downloading an image with wget using wget link I get the following errors: HTTP request sent, awaiting response... 302 Moved Temporarily Location: [following] --2015-05-24 14:47:46-- ...
-1
votes
2answers
60 views

How to install exe versions of linux utilities on windows? [duplicate]

I am aware that we can get .exe versions of many linux utilities. I happen to have wget.exe, curl.exe and busybox.exe. But how can I install them so that instead of typing C:\Users\Rahul>cd ...
2
votes
1answer
526 views

Mirroring with wget - Exclude old files

I am using wget to mirror an ftp file-archive. This is my command: wget -m $mirror (Stripped all unneccessary parameters) So far, everything is okay, all differences to the online archive will be ...
0
votes
0answers
34 views

wget converting http but not https links for offline use

I am trying to download a website for offline use using the following wget command on an Ubuntu machine: wget --recursive --background --adjust-extension --continue --page-requisites --convert-links ...
5
votes
1answer
6k views

How to get past the login page with wget?

I am trying to use Wget to download my private GitHub pages, but I cannot get past the login screen. How do I send the login/password using post data on the login page and then download the actual ...
0
votes
1answer
63 views

Wget - repeat download of the same image from URL (with backup and timestamping)

I am trying to download images from a webcam that refreshes the image about every 3 seconds. I want to use the timestamping option in case their webcam stops updating the files (happens from time to ...
3
votes
5answers
11k views

How to use Wget with Tor Bundle in Linux

I'm a Linux Mint (Lisa) and Tor Bundle user trying to use wget over Tor. After following the directions I found here, all I get when running wget is an output file saying, "514 Authentication ...
1
vote
1answer
79 views

fix wget “invalid or incomplete multibyte or wide character”

Wget show me Can't write to ... (invalid or incomplete multibyte or wide character) when i try to dump a website with Persian or Arabic characters in urls. Have wget any flag to fix this characters ...
2
votes
2answers
1k views

linux: how to run aria2 or wget only through eth0

I have a computer under Linux with several network cards, for example: eth0, eth1, eth2, eth3. Is there some way to run any downloader, like aria2 or wget only through one interface, for example eth0? ...
2
votes
1answer
139 views

wget how to download zip files with query parameters attached to filename

I am sending a request to a clients server using wget. I want to download all the .zip files but the problem is the .zip files have key/values appended to the files. Like so: ...
1
vote
2answers
83 views

Modifying names of files BEFORE wget downloads them (not renaming local versions)

I have a website filesystem wherein each file has two copies, a summary copy and a large copy, with different names. For example, a large copy file might be pages/info/GK2grkG23r_lg.html while in the ...
0
votes
1answer
24 views

Craft HTTP GET on Mac to get file

So I'm trying to download a file on my mac but the requirements to get the file are very strict: Username: testx Password: testing123 Authorization You can craft the ...
0
votes
0answers
42 views

WGET convert links not working with URLs that start with slash (relative URL)

Knowing SuperUser family, I am attempting to download mirror a site with the following command: wget --recursive --background --adjust-extension --continue --quiet --page-requisites --convert-links ...
0
votes
0answers
28 views

How to make wget resolve HTTP queries? [duplicate]

Consider the download link http://trac-hacks.org/changeset/latest/accountmanagerplugin/tags/acct_mgr-0.4.4?old_path=/&filename=accountmanagerplugin/tags/acct_mgr-0.4.4&format=zip with the ...
7
votes
5answers
4k views

Trouble using wget or httrack to mirror archived website

I am trying to use wget to create a local mirror of a website. But I am finding that I am not getting all the linking pages. Here is the website ...
0
votes
1answer
132 views

Modifying wget permissions and download directories on Cygwin

I have Cygwin installed on my Windows 7 64 bit PC and I often download large files using wget. wget, however, saves all files with permissions that forbid me from executing the files when they're ...
17
votes
8answers
38k views

wget/cURL alternative native to Windows?

Is there a wget or cURL type command line utility native to Windows Vista? How does it work?
1
vote
1answer
119 views

Downloading an authenticated large file with command line tools

I need to download a file sized around 1.6GB. I's like to get it download via a command line tool like axel, wget or curl However, how can I download a file which needs to be logged in to the site ...
0
votes
0answers
24 views

wget grabbing too much

I am building myself an app to analyze MLB stats. I figured I'd be a better citizen by grabbing the data once and putting it on my hard drive rather than hammer the "API". I also thought my debugging ...
1
vote
2answers
4k views

wget not working on one machine: 404 Not Found

I am trying to use wget to debug and issue I am having. I have a remote machine from one server trying to reach query.json and it works fine, connects and downloads fine. I have another remote machine ...
3
votes
1answer
128 views

wget starting from arbitrary byte

Using the -c flag, I can tell wget to resume a download. wget will then query the size of the pre-existing file to tell the server from where to resume the download. Is there a way to tell wget to ...
0
votes
2answers
353 views

What tool does yum use to download rpm packages from remote server?

My system is centos, and now I am installing packages using yum command. Updating for dependencies: apr-util i386 1.3.9-1.el5 lxcenter-updates 203 k gcc-gfortran ...
0
votes
2answers
111 views

Why wget fails with 'https: Host not found' on Windows?

I'm trying to use wget on Windows 8.1 prompt. I installed wget.exe in the folder System32 and run wget on cmd. But the following error appear: M:\site>wget http://wordpress.org/latest.zip ...
0
votes
0answers
17 views

wget did not resume even with -c option

After downloading over half of an 8 GB file from the Internet Archive, I interrupted wget, and a few minutes later started it again, as I had already done successfully several times in the past: wget ...
7
votes
3answers
9k views

Why do images from some Tumblr pages not load, but using wget on them works?

Helping a friend out with their Internet connection because “some pages won’t load”, I noticed that the problem was that the images of certain blogs' image posts weren’t loading on the browser. I ...