GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

learn more… | top users | synonyms

0
votes
0answers
21 views

How to check what kind of files web page consists

Is there a way to check what files web page consists. For example I want to download .txt files from the web page, but I dont want all .txt files, some of them, and before I want to check are there ...
0
votes
1answer
242 views

Is there a way to download all .txt files from the page via curl or wget command

My question is in a title: Is there a way to download all .txt files from the page via curl or wget command? Thanks
0
votes
0answers
43 views

Listing the Files in a Web Page Directory via Wget

What should I write in the command line to list the files in this webpage: "http://download.xbox.com/content/5454085c/"? Files are supposed to be in ".cab" format.
1
vote
1answer
277 views

How to make wget create appropriate zero-byte files?

I'm trying to use wget to download entire directories with their contents from a webserver. Some of the files there have zero length. Is there a way to make wget create the 0-byte file on the client? ...
0
votes
0answers
847 views

Why is cURL slower than wget

Test page: https://www.beobank.be/nl/Home.aspx This is the result using wget (or a real browser): # time wget https://www.beobank.be/nl/Home.aspx -O /dev/null --2015-01-26 12:05:46-- https://www....
2
votes
1answer
3k views

Can ping but can't connect (curl, wget, apt-get, etc.) (rasberry pi server)

Background: I am trying to set up a rasberry pi server. Everything was going well and my pi could apt-get update etc, until I got the port forwarding to work and ssh to work outside of the LAN. I ...
0
votes
0answers
150 views

wget working on linux but not on mac os

I'm trying to download this file: http://www1.caixa.gov.br/loterias/_arquivos/loterias/D_megase.zip using wget, like this: wget http://www1.caixa.gov.br/loterias/_arquivos/loterias/D_megase.zip ...
0
votes
1answer
233 views

wget mangling output written to stdout, works only with -q

I have a minimal HTTP server written in python (http://pastebin.com/mjLDk7Gk) that simply answers every GET request with status 200 and body "Done". If I do wget "http://localhost:9001" -O - -q ...
0
votes
1answer
69 views

how to download pictures with wget

i would like to download pictures from http://myhomepage.com/plots/20150111-03/4711/4712.png every 3 hours. "20150111-03" is variable, rest is stable. That means every 3 hours the download looks like: ...
1
vote
1answer
1k views

wget missing from cygwin install

I was trying to install oh-my-zsh and found an article requiring 'wget' to do that. My install of cygwin64 on Windows 7 did not contain 'wget' -- I searched for it in the search filter of the setup ...
1
vote
3answers
203 views

Bash: executing concatenated variables

I have a Linux bash script, to the effect of: COMMAND_TO_EXECUTE="foo "$1" --option1 --option2 "$2 exec $COMMAND_TO_EXECUTE Problem: the script fails Debugging clue, If I echo $...
0
votes
0answers
462 views

Make wget cache DNS resolution results

I'm using wget in a script to download a changing file from Dropbox: wget -4 -O filename.ext "https://www.dropbox.com/s/<fixed-hash-id>/filename.ext?dl=1" I notice that wget resolves the IP ...
0
votes
1answer
61 views

Download file on unix from target url which autodownloads file on page visit

I'm trying to download a file to a unix machine on the command line. Accessing a GUI on this particular machine would be quite difficult. The file is autodownloaded when visiting the url in a web ...
0
votes
0answers
70 views

Http Starts failing after a while using windows 8.1

I have problem with my connection on windows 8.1, it looks fine when restart my system, for this wget call. wget http://www.google.com/ as following result. --2014-11-26 17:59:37-- http://www....
0
votes
0answers
35 views

How to tell wget to skip arbitrarily nested, named directory?

I know how to tell wget to skip an absolute directory, e.g. -X images/thumbs. But I can’t seem to get wget to skip all directories named thumbs, no matter what the ancestor directory is named. -X '*/...
1
vote
2answers
155 views

Why doesn't wget -m save the index.html documents

I'm making a copy of a website with wget -m -p -q http://example.com The process works fine, but the copy is missing all index.html files, even so I see them being sucessfully requested and all ...
2
votes
2answers
2k views

wget stops downloading: Cannot write to `-' (success)

Please have in mind that I don't know anything about servers, nor any of its vocabulary, I am a simple web coder, trying to figure my way on the universe! I'm trying to link dropbox on my server, I am ...
0
votes
1answer
164 views

Are there any programs that supports NRPC? (Lotus Notes RPC) [closed]

Are there any implementations of the Lotus Notes RPC (port 1352)? Example: wget NRPC://<Server-hint-or-host>/bar/foo.xls -O foo.xls So that we can download a file that is only reachable via ...
0
votes
1answer
59 views

can't connect to local web server

when i try to connect to a web server installed on a pc in lan i can't get the index page, the computer is well configured because i can ping and ssh it: ping $ ping 172.16.69.196 PING 172.16.69.196 ...
0
votes
0answers
27 views

use wget for the most updated file on http server

I need to download the most updated files with the name suite. I am using windows server to run the command lines. I wrote the following command but not sure of how to get the most updated one wget ...
0
votes
1answer
272 views

Where is the default directory of saved files using wget in crontab?

I want to download some files using wget command in crontab. I use a text file to make a list of my desired urls. wget's -O parameter works well for setting a destination folder for single files, but ...
1
vote
1answer
124 views

Download all files with some strings in their names

On http://www.inf.usi.ch/carzaniga/edu/algo08f/schedule.html, I would like to download all pdf files of the name *-2up.pdf, for example, http://www.inf.usi.ch/carzaniga/edu/algo08f/intro-2up.pdf Can ...
2
votes
1answer
308 views

How to force wget to output a summary?

I have a bash script that launches a series of wget commands and log the output. I use the following options: wget --no-verbose --page-requisites --adjust-extension --convert-links --backup-converted ...
0
votes
2answers
22 views

Wget does -m need to be used with -r --no-parent argument

Title pretty much says it all. If i want to mirror a website directories and sub directories. should i use wget -m -r --no-parent bla.com or should i use wget -r --no-parent bla.com. Is -m needed?
2
votes
0answers
395 views

wget: unable to resolve host address

I've RedHat Enterprise version 5.2. I run this wget --http-user=celee --http-password=1010$ -k -nd -np -r http://sp.kch.de/RandD/DesSup/Mask%20Tooling%20Runsets/ -o wget.log The wget.log file ...
0
votes
0answers
143 views

Does wget download files more than once?

Suppose there is a website where Page A links to Page B and pages A and B both link to page C. Somthing like A -> B -> C | v C If I run recursive wget on A to begin downloading the pages ...
4
votes
0answers
89 views

Mirror a blog with wget

I am trying to mirror a blog, eg www.example.com, with wget. I use wget with the following options (shell variables are substituted correctly): wget -m -p -H -k -E -np \ -w 1 \ --random-wait ...
0
votes
0answers
348 views

Download file vis cURL, but not with Wget

I have a link I'm trying to download. Before, we could download with either Wget or cURL because the website (which runs under Tomcat) didn't require a user name or password in order to download. ...
1
vote
1answer
503 views

How to restart a router with curl or wget to pass URL as well as POST parameters?

I have an Edimax router and from wireshark I see that the router restarts with the POST http://192.168.2.1/goform/formReboot and the form elements reset_flag=0&submit-url=%2Fadmin_restart.asp ...
1
vote
1answer
252 views

Is there an alternative to httpfs2 to mount web urls in Linux?

Currently I use httpfs2 to mount direct web urls in the filesystem. For example a rar archive which has to be decompressed, but I don't have enough space on my VPS to download AND then decompress the ...
0
votes
0answers
22 views

How to Download HTML Pages of a WebSite, Based on a Time Stamp?

I know this might be a repeated question, and I'm sorry for that, but I failed to find any answer for my problem . I Just want to download the Pages html files, without any other contents, and (If ...
1
vote
1answer
171 views

Wget downloaded more data than the actual file

I have downloaded lubuntu image using wget with mentioned file size 694MB , but the downloaded file is of size 789MB. when i do wget -c it gives that the file is already downloaded. But the file size ...
0
votes
1answer
138 views

Mirror an ftp site, but create empty files instead of full files to save bandwidth

I want a listing of all files and directories at ftp://ftp.ncbi.nlm.nih.gov/snp/. But I don't want to download the contents of all the files. To save bandwidth, I hoped that wget would have an option ...
1
vote
1answer
64 views

amazon ec2 intance rhel7 ftp not downloading

this is driving me insane. i am trying to download files using FTP. i tried with PHP's FTP, the FTP binary, WGET, none of them work. They all have the same problem. When get command is initiated, the ...
2
votes
1answer
263 views

wget how to download zip files with query parameters attached to filename

I am sending a request to a clients server using wget. I want to download all the .zip files but the problem is the .zip files have key/values appended to the files. Like so: http://example.com/...
0
votes
1answer
65 views

can't open .gzip archives retrieved with wget

I'm trying to download the content of a directory hosted on a server using wget in a cygwin terminal. I have no problem getting the files but they are all .gz archive and once downloaded they are ...
1
vote
2answers
106 views

Modifying names of files BEFORE wget downloads them (not renaming local versions)

I have a website filesystem wherein each file has two copies, a summary copy and a large copy, with different names. For example, a large copy file might be pages/info/GK2grkG23r_lg.html while in the ...
0
votes
1answer
308 views

How do I mirror sub directory of a website with wget

I need to mirror the following URL: https://web.archive.org/web/20100515230709/http://markowitzbocadentist.com/ But I need to mirror only files from this date '20100515230709' i have tried many wget ...
0
votes
1answer
139 views

Try to download mp3 with wget but it redirects to another site?

I am trying to download mp3 files from this adress hanialrefai using wget like this: wget -r -l0 -k -p -A.mp3 http://hanialrefai.com but it redirects me to http://quantsa.ru/?pw and then it say: ...
1
vote
1answer
284 views

Wget - Problems recursively downloading with authentication

So I'm trying to download the entire domain of a private wiki page. I've saved the cookies in a cookies.txt file and using it with wget for authentication like so: wget --load-cookies=cookies.txt --...
0
votes
2answers
730 views

What is the simplest way to download with wget in remote putty terminal using my local windows internet connection

I'd like to initate a download using wget in a remote putty terminal (i.e. an ubuntu server), but using my local windows internet connection. I guess there is two way to achieve this: run an HTTP ...
1
vote
1answer
145 views

Downloading an authenticated large file with command line tools

I need to download a file sized around 1.6GB. I's like to get it download via a command line tool like axel, wget or curl However, how can I download a file which needs to be logged in to the site ...
0
votes
1answer
190 views

download file from a url that returns a save dialog box

I have a url which on clicking returns a save dialog box for a pdf file. I would like to save this file using wget. Any ideas how to do this? Basically the url is not the url of a file but a server ...
0
votes
0answers
39 views

Do functions/aliases defined in a bash script that's downloaded from a server and executed not work?

I had a .bashrc which I moved to server so that I can download and execute it through wget. wget -O - http://xxx.github.io/dotfiles/bashrc | bash It does get executed but any functions and aliases ...
0
votes
1answer
83 views

Downloading Images through Wget With Proxy

I've written a ruby tool to download multiple images. However, this time I'm having to configure wget with a proxy. Prefix is the specified directory of where I want the images to go and md5_sum is a ...
0
votes
1answer
1k views

wget index.php redirection - The name is too long

I am trying to get a copy of the catalog. I would love to get the "List View" and "Show Object" details. The URL I am tring to wget is: http://catalog.quittenbaum.de/index.php?_function=list_objects&...
0
votes
2answers
121 views

wget doesnt create index file of directory

I try to download site with command: wget -r -l 0 -p -k -m -e robots=off --restrict-file-names=nocontrol,unix http://site.com When it finds link, for example, http://site.com/direction, it creates ...
2
votes
4answers
672 views

Download forum with wget with username and password?

I want to download a forum where I only have access via my username and my password. I tried the following: C:\wget.exe wget -k -m -E -p -np -R viewtopic.php*p=*,memberlist.php*,faq.php*,posting.php*...
1
vote
1answer
6k views

wget mirroring the site fails - 403 Forbidden, even with user-agent

I need to download suricata's latest rules from this url: http://rules.emergingthreats.net/blockrules/ I'm trying to compose my own rules based on different emerging rules and snort's rules too. ...
2
votes
2answers
2k views

How to download an image with wget and save it with the md5 hash as name?

How can I make a download an image, md5 hash the image, and save that image with the md5 hash as name to a directory using wget? # An example of the image link... http://31.media.tumblr.com/...