GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

learn more… | top users | synonyms

-1
votes
0answers
8 views

Download multiple range of files with wget

I have seen this Loop over a range of numbers to download with wget As an extension of this, is it possible to download multiple range of files in single session. For example, my range could be 540....
-1
votes
1answer
10 views

How can I filter a list of files to download those that don't already exist?

I have downloaded some files into a folder, but the download was interrupted and not all the files were downloaded. I do not want to redownload the gigabytes-worth of files I already have; I now want ...
0
votes
1answer
3k views

bash- how can I use wget to send POST data with files?

I want to make following POST request to my server using wget: email=abc@abc.com&file1=@FILE_HERE&file2=@FILE_HERE In above request, there are three POST parameteres called email , file1 ...
0
votes
0answers
28 views

Schedule on crontab to reboot router

I would like your much aprreciated help. I have a ISP router (gateway) that doesen't have a schedule reboot on the web interface. I have a raspbian and I would like to schedule on crontab to reboot ...
9
votes
3answers
24k views

How to download list of files from a file server?

How would I download a list of files from a file server like this one http://www.apache.org/dist/httpd/binaries/ ? I suppose I could use wget but then it tries to get all the links and the html file ...
30
votes
6answers
49k views

how to download dropbox files using wget command?

Seems I can only download dropbox files using explorer such as chrome and firefox. If I use wget to download, then I would get a file which is in html format. Why? For example you can open this link ...
0
votes
1answer
42 views

Stop Chrome ignoring ERR_CONTENT_LENGTH_MISMATCH for downloads?

I sometimes find downloads made using Google Chrome get interrupted, causing files to be truncated. Chrome seems to detect this situation as ERR_CONTENT_LENGTH_MISMATCH, but then ignores the error and ...
0
votes
0answers
24 views

How do I get wget to grab the specific directory I want multiple levels down from the home page?

I have a website, lets call it WEB I am trying to get the directory and everything under it from WEB/a/b/ I don't want anything from a/ but I want b/ and whatever else it may link to (the furthest may ...
10
votes
3answers
26k views

Recursive download (`wget -r`) equivalent for Firefox?

I have a website, and I want to download all of the pages/links within that website. I want to do a wget -r on this URL. None of the links go "outside" of this specific directory, so I'm not worried ...
2
votes
1answer
25 views

openssl s_client returns DNS error

When trying to connect to a site using openssl, I get the following error. $> openssl s_client -connect www.citibank.com:443 getaddrinfo: No address associated with hostname connect:errno=2 But ...
-1
votes
0answers
26 views

download data from website using scripting

I need to download data from following webpage: https://www.nseindia.com/corporates/corporateHome.html?id=eqFinResults. The issue is that there are more than 2000 possible values of companies and 2 ...
1
vote
2answers
5k views

Where should I put libintl3.dll to make wget work

I've downloaded wget binary for windows. I placed it in System32 folder. When I try to run the command, an error pops up: The program can't start because libintl3.dll is missing from your computer....
1
vote
1answer
236 views

Is there an alternative to httpfs2 to mount web urls in Linux?

Currently I use httpfs2 to mount direct web urls in the filesystem. For example a rar archive which has to be decompressed, but I don't have enough space on my VPS to download AND then decompress the ...
0
votes
0answers
11 views

Saving of pages of the site vk.com by wget

I need to download many pages from the site vk.com. Example of link: http://vk.com/wall245887788?offset=140. On various sites users recommend to use the wget. At me it turns out to download ...
0
votes
0answers
20 views

What does block the crawl of my website by Httrack or Wget?

I am attempting to clone my website to show it for a presentation offline. However I tried either with Httrack either with Wget and both are stoping to the second level of the source tree. What could ...
0
votes
0answers
4 views

Wget recursively use stop after 2nd level

So I'm trying to do a proper mirror of a website I've done to present it offline. Strangely this is stoping at the second level even since I'm using -linf wget -r -linf -k -p -E robots=off http://...
0
votes
1answer
43 views

wget script exit abnormally with error 126

Can't seem to get this script to run, I have user access to the data I'm trying to download but it just won't run. If anyone can help with wget scripts it would be appreciated. for i in $(seq 1 9); ...
3
votes
1answer
2k views

Batch download of URLs from command line multithreaded

I have 100,000 URLs of small files to download. Would like to use 10 threads and pipelining is a must. I concatenate the result to one file. Current approach is: cat URLS | xargs -P5 -- curl >>...
0
votes
2answers
723 views

Does wget have a download history?

I have used wget to download several compressed packages(in a failed attempt to manually emulate apt-get, if that is of any significance), and I don't remember the packages' locations. Is it possible ...
0
votes
2answers
37 views

How can I archive someone's Facebook profile page?

I thought about using archive.is or archive.org. I tried to view the page in a private browser, but it asks to log in, so that site will see the same thing. I tried to open the developer console and ...
0
votes
1answer
32 views

How is wget able to download an iso file when only iso.gz is listed under the FTP library

I hit a behavior of wget from FTP server (URL: ftp://foo.com/bar/...) that was unfamiliar to me. Can you direct me to some documentation or explain how is the following possible? If I wget the folder'...
0
votes
0answers
6 views

Does wget retrieve files in above levels when using --no-parent and --page-requisites?

Here's the scenario: Site with file www.dom.com/foo/bar/page.html, which contains a reference to file www.dom.com/foo/page.css Case 1) Would wget --page-requisites --no-parent http://www.dom.com/...
0
votes
2answers
22 views

Wget does -m need to be used with -r --no-parent argument

Title pretty much says it all. If i want to mirror a website directories and sub directories. should i use wget -m -r --no-parent bla.com or should i use wget -r --no-parent bla.com. Is -m needed?
5
votes
3answers
5k views

wget -o writes empty files on failure

If I write wget "no such address" -o "test.html" it first creates the test.html and in case of failure, leaves it empty. However, when not using -o, it will wait to see if the download succeeds and ...
0
votes
1answer
58 views

How can you “press” a button of a webpage from the command line to restart a cable modem?

I'm trying to reboot my cable modem programmatically. I used to be able to do this by using wget: wget http://192.168.100.1/reset.htm?reset_modem=Restart+Cable+Modem However I changed my cable ...
0
votes
0answers
47 views

wget issue with a tunnel connection using ssh

I'm working on a campus where the network is highly insecure (certain people are doing packet sniffing) and proxy servers are used for internet access. I have access to a server (communication ...
0
votes
1answer
30 views

Psexec with curl or wget

I recently found out about psexec and was wondering if this program could help me run wget or curl commands on a remote computer. is this even possible?
26
votes
8answers
58k views

Wget/cURL alternative native to Windows?

Is there a Wget or cURL type command line utility native to Windows Vista? How does it work?
7
votes
3answers
15k views

wget for ftp using a password containing @

I am trying to get some files from my ftp server from the command line. I am using wget to download the whole folder at once. The command is: wget -m ftp://username:password@ftp.hostname.com:/path/to/...
2
votes
4answers
5k views

wget - download all files but not preceding folders

I'm using wget to download all files from within a folder using the -r and -np options. However this also downloads the preceding folders, which I don't want. For example: wget -r -np ftp://user:...
5
votes
4answers
12k views

How to combine wget and grep

I have a html-page url and I want to grep it. How can I do it by wget someArgs | grep keyword? My first idea was wget -q -O - url | grep keyword, but wget's output bypass grep and arise on the ...
3
votes
3answers
4k views

Is it possible to force wget to use one connection?

I'm trying to download about 20GB of data with FTP with my personal account (no scp available and tar-ing is not possible). I'm using the following standard command: wget -m ftp://user:pass@host....
0
votes
0answers
26 views

gnuwin32 wget Unable to locally verify the issuer's authority

I have just installed gnuwin32 wget and have had some issues when running it. I appear to have issues with https sites... I get a "Unable to locally verify the issuer's authority." error. I have very ...
3
votes
2answers
30 views

How can I download a file the moment it becomes available on linux?

How can I set up something on a Linux machine so that the moment a file on a web server returns something other than a 404 error it will download it. I would like it to stop checking the moment the ...
5
votes
4answers
3k views

wget not converting links

I am trying to mirror a fairly large site (20,000+ pages) prior to a major overhaul. Basically, I need a backup before cutting over to the new one in case we forgot something we need (we'll have ...
1
vote
1answer
37 views

wget for FTP end in “connection refused”

I want to copy a FTP directory to my server (Debian) using wget command. I tried : wget -m --user=user --password=passftp://ftp.domain.com I've got this error : connection... failed : Connection ...
178
votes
15answers
383k views

How to download files from command line in Windows, like Wget is doing?

How can I download something from the web directly without Internet Explorer or Firefox opening Acrobat Reader/Quicktime/MS Word/whatever? I'm using Windows, so a Windows version of Wget would ...
0
votes
0answers
34 views

How can I plot cwnd vs. time and queue occupancy vs. time graph of the buffer from wget output?

I had used a smaller buffer for 2 host network h1 and h2 using sudo ./run-minq.sh command. Now at first, I observed the wget from h1 to h2. Next I start iperf and wget simultaneously and observed the ...
0
votes
1answer
57 views

Download the directory structure of a website without its files?

I googled for this but couldn't find an example of how to use eg. wget to download the directory structure of a website without downloading any of its files. At this point, I just need to understand ...
0
votes
1answer
69 views

Downloading with wget using referer + cookie

I'm trying to download a file from a website that requires login. This command retrieves the cookie.txt and the downloaded html shows I'm logged in (ok): (I broke the command in lines for readability)...
0
votes
0answers
12 views

mirror site behind login

I would simply like to mirror all the pages (millions) on a site behind a login. This is for personal use. I have a login for the site in question but I want to scrape it into a database. I ...
1
vote
1answer
3k views

WGET downloading extremely slow

I am using the following command wget -r -np -k -p http://www.site.com/dir/ To download a small archive of files(there is a couple hundred files so I don't want to click download on each one and ...
1
vote
2answers
58 views

How to wget a webpage's actual code for a <div> element

I use wget to save a webpage to a file 'annie.html'. % wget -qO- http://www.swagbucks.com/watch/video/523803/annie > annie.html Inside this file is this line: <div id="...
51
votes
4answers
66k views

Save a single web page (with background images) with Wget

I want to use Wget to save single web pages (not recursively, not whole sites) for reference. Much like Firefox's "Web Page, complete". My first problem is: I can't get Wget to save background images ...
0
votes
1answer
595 views

How can I download files from shared Amazon Cloud Drive links through the command-line?

I have been using Amazon Cloud Drive (ACD) for a while now and I am very happy someone has developed a command-line-client for it (called acd_cli). It allows me to keep my files in "sync" with my ...
0
votes
0answers
26 views

How to make wget IGNORE certain files?

How do I make wget IGNORE certain files? I ask, since it downloads them and deletes them afterwards, since they're not required (they're excluded). But is there an option to IGNORE them BEFORE even ...
0
votes
1answer
57 views

Cannot write to '-' (Broken pipe) OVH server?

I am trying to run this command to install Windows on my OVH VPS and I get Cannot write to '-' (Broken pipe); root@rescue-pro:~# sudo -i wget -O- 'http://mirror.joodle.nl/Windows_Server2012R2.gz' | ...
0
votes
1answer
29 views

Downloading multiple files in parallel with changed names

I'm writing bash script and don’t know how to solve my problem. I want to to download multiple files at the same time , but ... I want to save the files under modified names. I have a variable ...
0
votes
1answer
15 views

Neither wget nor curl downloads this wikipedia api request

I can perfectly download this link and save it as a text file in a web browser environment. But I can't download and save that wikipedia api request using neither wget nor curl. I use wget -O file.txt ...
0
votes
1answer
33 views

Wget download speed fluctuating

So I'm downloading an ISO from the Uberstudent website using the wget command displayed on their website. But the download speed is constantly fluctuating and I am wondering what could be causing this?...