GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

learn more… | top users | synonyms

0
votes
1answer
594 views

WGET downloading extremely slow

I am using the following command wget -r -np -k -p http://www.site.com/dir/ To download a small archive of files(there is a couple hundred files so I don't want to click download on each one and ...
0
votes
2answers
113 views

Can wget decode uri file names when downloading in batch?

Normally when downloading files using wget, the filenames are encoded. In example: wget -c --content-disposition "https://class.coursera.org/ml-005/lecture/download.mp4?lecture_id=1" and the saved ...
0
votes
0answers
134 views

Parallel download for a list of urls and renaming

I have a list of tab-separated urls, and target file names, urls_to_download.txt, for example: first_file.jpg\thttps://www.google.co.il/images/srpr/logo11w.png ...
0
votes
1answer
78 views

How to navigate PHP-based pagination with WGET?

I'm trying to come up with a list of possible names for our next baby boy and have been looking through the site IndiaParenting. Each name on the site has a detail page at a URL like this one ...
0
votes
1answer
53 views

How to automatically save a list of files with different names?

I have a text file of URLs, each in the form of: http://domain.com/a/b/c/d.jpg I want to download all of these, but save each file under the name: c_d.jpg In other words, for each file, I want to ...
-2
votes
1answer
49 views

wget doesn't match patterns with wildcards

I'm trying to download certain url's which I would like to match with the next pattern using wildcards: http://example.org/subtitles?q=NUMBER_en&format=srt where NUMBER = [0-9]* the output of ...
0
votes
2answers
607 views

Using wget to get files from local to server to remote server

We have a local server where we carry out much of our web development and then we push this to a live environment so that clients can access. We often end up with huge tarballs (this one in ...
1
vote
1answer
43 views

How do I make wget match exact domain names?

I'm using the following command to mirror a site: wget -r -p -E -N -l 10 -k -nH -H -Dfoo.com,www.foo.com foo.com The problem I am having is that somewhere on foo.com there are links to domains ...
0
votes
0answers
55 views

How to download all images from a gallery

I'm searching for a way to download all images (not the thumbnails) of this gallery http://kinki.de/#/?gallery=list&gid=4804 . As I can see, the images are all located at ...
0
votes
0answers
171 views

How can I use Wget / Curl / Plowshare to download from 1fichier.com as a premium member?

How can I download files from 1fichier.com (I am a premium member) with the help of Wget / Curl / Plowshare (Plowdown) or other CLI / command line tool / script?
0
votes
1answer
56 views

Trace web redirection in bash script

I want to make a script to download the image at best resolution (not the preview image) from a deviantART link, as if I clicked the "Download" button. However, it seems that deviantART redirect the ...
0
votes
0answers
212 views

OpenWRT wget in background

I start first script /etc/rc.local : sh /root/script1.sh & In file /root/script1.sh I connect by wget to server (while for tries) And after it I run : # say hello tries=1 while [ 1 ]; do ...
0
votes
1answer
33 views

wget only saving first line of cookies

I want to download and save all the cookies from a particular web-page when using wget. However, wget is only saving the first line. I want the saved cookies from wget to be the same as the cookies I ...
0
votes
0answers
96 views

How can I extract stock price data from a specific webpage?

I would like to extract the stock data (i.e. daily stock price) from the following webpage: http://www.onvista.de/aktien/Alcatel-Lucent-Aktie-FR0000130007 This page shows a chart graph of a stock ...
0
votes
0answers
92 views

how to use wget and/or python with an SAP HANA XSJS web page

I have an SAP HANA XSJS web page that I would like to retrieve using wget. I have tried using wget --save-cookies with password data in --post-data but the cookies.txt file comes back empty and the ...
0
votes
0answers
24 views

make wget refer to a local copy without redundantly downloading files

I want to archive a message board,I do that by using wget with the parameters: --page-requisites, --span-hosts, --convert-links and --no-clobber. The problem is that using --convert-links disables ...
-1
votes
1answer
38 views

Is there a way to read a simple one line asp page using a batch file?

I have a simple asp page in which i write an ip address, and nothing more. Now i need a batch file to read the contents of this page and pass it to a variable. I'm not good with batch files but ...
0
votes
1answer
134 views

WGET/cron downloading webcamera file

our new town doctor put a web camera to his waiting room for people to see how much crowded it is. As my mom and grandparents go to this doctor, I wanted to download the files periodically and if they ...
1
vote
1answer
98 views

What is the curl equivalent to wget -x?

I want to use curl to save multiple files with the same local paths as would be produced by wget -x (ie, cleaned up versions of the full urls). But I can't find the right option in the curl docs. Is ...
0
votes
0answers
40 views

How to wget range that fills a PHP URL

I try to wget from 1 to 724.. When I try this line in the terminal : wget http://exemple.fr/resultats.php?ID={1..725}.html everything works fine except that the pages downloaded are no filled with ...
1
vote
1answer
768 views

Why I can't use wget command in SSH shell?

I have the following problem. I am using my RaspBerry (with RaspBian installed) by SSH. My problem is that I can't use wget command because it fails. For example if I try to execute the following ...
1
vote
0answers
54 views

Does aria2 support write small files in batch?

I'm using aria2 to download 8 million jpg from flickr. Each image is about 100KB. I got a list of urls of these images in a txt file, the format is: ...
0
votes
1answer
45 views

How can I get a notification if wget can not reach a site in x seconds?

How can I set a timeout for wget and get a notification the connection times out in 20 seconds? So far I've tried the following command wget --spider www.example.com
0
votes
1answer
51 views

How I can download PDFs of a website by using only the root domain name?

I am using this command: wget -nd -e robots=off --wait 0.25 -r -A.pdf http://yourWebsite.net/ but I can't get PDFs from the website. For example I have a root domain name: www.example.com and ...
0
votes
1answer
90 views

Downloading entire web pages within domain [duplicate]

My computer in on LAN. I'm connected to the local domain at MyDomain. Within that domain exists something like WIKI for a training (http://training-wiki/). I would like to download entire WIKI into my ...
3
votes
1answer
150 views

How to use wget to download images from messages on facebook?

This look a bit harder than usual. I'm a new user wget and i was thinking that maybe it's possible to somehow login into facebook and download only images from a specific conversation. I mean, i'd ...
1
vote
1answer
105 views

Ping works, other commands fail

I'm behind a proxy at work. When I try to ping google, it works. When I try to wget, I got stuck on the "Connecting to [proxy name]". I tried both via the "google.com" adress and via the IP adress I ...
1
vote
1answer
84 views

Firefox - Download files using external bash script

Is possible to issue external bash script when files are being downloaded in Firefox? My goal is high customization. E.g. scripting something that will save files to specified folder based on ...
1
vote
1answer
113 views

Wget - Difficulty Excluding Domains and Directories

I am using Wget in Windows 7. I believe this restricts me to Wget 1.11.4. For anyone interested in the details, I have posted a sample wgetrc file. This sample is actually the one I am currently ...
1
vote
0answers
50 views

wget put all prerequisites in flat subdirectory, but not root page?

I'm trying to get wget to save a page + prerequisites in a format resembling that of a web browser: article.html article_files/img.jpg article_files/script.js I am able to get almost this behavior, ...
0
votes
1answer
42 views

How to autostart wget to download files when internet connection gets up?

I am using Windows XP. i have created a batch file containing wget -i links.txt --read-timeout=2 -t 0 -w 0 --waitretry=0 -c --no-check-certificate -b and the file "links.txt" contains URLs pointing ...
0
votes
0answers
63 views

wget sometimes fails, curl succeeds

I am continuously downloading some packets via wget. Once in a while, the wget fails. I have used retries as 2 and timeout as 10 seconds. When the failure occurs, I try the same URL again, but with ...
0
votes
0answers
61 views

Extract files from WARC and correct URLs

wget has an option of correcting the URLs when recursively downloading a website, so the website is browsable offline. It makes a folder structure that looks like this: Is there a program that can ...
1
vote
1answer
126 views

Mirroring a web site having pages that uses simple JavaScript [duplicate]

There are simple and great web site download tools that allow to create a local mirror of simple sites having no JavaScript infrastructure behind. However, sometimes there cases when a site looks ...
0
votes
0answers
35 views

Gzipped file arrives unzipped

I've recently set up an ArchLinux system at work and added the ArchHaskell repo. This is far from the first time I do this. I have a number of systems where it's worked just fine, but this last time ...
2
votes
1answer
1k views

How can I download private repo from github?

What I want to do is to download private repository archive from GitHub, extract it, remove archive file and copy some directories that are inside downloaded project. I tried to use wget but I cannot ...
2
votes
1answer
1k views

How can I force wget.exe to talk HTTP/1.1?

I'm talking about Windows, not Linux tried: wget -O X:/test/some/dir/file.txt --tries=3 -c http://some.com/dir/data.txt wget -O X:/test/some/dir/file.txt --tries=3 http://some.com/dir/data.txt wget ...
0
votes
0answers
416 views

Wget with input file says “No such file or directory”

I want to use Wget for Windows to retrieve some files on URL's which I have specified in a file called WgetInput.txt After reading the documentation I figured out the command I should use, which is: ...
0
votes
1answer
532 views

How can I encode an url for wget?

I am looking for a way to convert a string to a clean url. For example : wget http://myurl.com/toto/foo bar.jpg This is going to download http://myurl.com/toto/foo and http://bar.jpg. I want to ...
2
votes
1answer
279 views

How can I get a list of all messages in my gmail account in linux command line?

How can I get a list of all messages in my gmail account in linux command line? Ideally I would want to see something similar to: from:sender1@domain.com to:me@gmail.com Date:11/22/33 subj:Foo1 ...
1
vote
0answers
120 views

MacOS Mavericks: wget getting stuck with local ip but not with localhost

I am facing a weird problem on my Macbook (Mavericks). I'm serving some files over http. When I try to get the files using wget: it works perfectly using localhost or 127.0.0.1 it starts but get ...
0
votes
0answers
118 views

Using Wget with JQL Query

We are trying to run a Wget command wget -O xyz.xls --user=COldPolar --password=GlacierICe --ignore-length=on ...
0
votes
4answers
231 views

Mirroring a wordpress website with wget

I'm trying to download a wordpress web-site, my blog actually, and to get the php files as well. So far I've tried - wget -rkp -l3 -np -nH --cut-dirs=1 http://www.sharons.org.uk/ wget -r ...
1
vote
2answers
593 views

wget error 400 CentOS

I have problem with calling wget command in CentOS 6.5. Im using following command: (I spaced the https, because of my user level on thi forum) wget --certificate=/root/o2key.pem ...
0
votes
0answers
106 views

Pinging local machine with external IP

On a Gentoo server, I am trying to wget or curl a page from its url: wget http://example.com/test.html (I can get result for this url from my browser) All I receive is Timeout. With cUrl I get ...
0
votes
1answer
150 views

wget - difficult - How to download all links from a page including different path ones?

To make explanation easy and a bit entertaining let's imagine I want to download wikipedia pages of all the people mentioned here with one wget command, possibly with reasonable amount of other pages ...
0
votes
0answers
145 views

save content and header into different files using wget

I want to get url content and response header data into two file using wget. I see man of wget but i cant find useful method for this. somthing like this: wget -q --output-document=content.html ...
0
votes
0answers
228 views

wget input/output error (on linux CentOS / VMware)

I have booted a Linux OS (CentOS-6.5-x86_64-LiveCD.ISO), in VMWare Workstation (host windows Home). When i type this command i get the error as follows: $ wget input/output error I wonder if it is ...
0
votes
0answers
20 views

wget - (Not enough space) in headers

I'm trying to run wget as: wget URL -r Once I am connected, I get this error: "HTTP request sent, awaiting response... Read error (Not enough space) in headers" What causes this? Is there ...
0
votes
1answer
3k views

How to mirror ENTIRELY a website with wget [duplicate]

I really need the command to download everything from a website. It is arnos.gr and has a lot of content on it which I want to browse locally without Internet I have an account for login