0
votes
1answer
70 views

wget - difficult - How to download all links from a page including different path ones?

To make explanation easy and a bit entertaining let's imagine I want to download wikipedia pages of all the people mentioned here with one wget command, possibly with reasonable amount of other pages ...
1
vote
1answer
88 views

How do you get the cookie of a website after an https ntlm authentification?

I am trying to store an cookie (which will help me authentificate myself elsewhere) given by a website after an authentification. The case is as follow : - get to the https page - get redirected by ...
1
vote
1answer
224 views

Using wget does not give same output as browser

I am using wget for Windows (gnuwin32 wget-1.11.4-1) in Windows 8 and using it for a helpdesk tool called Kayako, telling it to poll from an email queue. The command line looks like this: wget.exe -O ...
1
vote
1answer
5k views

Downloading only images using curl or wget?

UPDATED: I've found using this Bash script fixes the problem of having GIF files with a .jpg extension. I'm attempting to download images from a forum who's url uses the following format: ...
1
vote
1answer
419 views

Scrape a site with curl/wget appending a parameter to each site

I need to scrape a website for all sites and append an Paramater, e.g. www.theSite.com/allpages?myPara=us Now what I got is wget --recursive to get all sites, but that doesn't append the parameter ...
0
votes
1answer
68 views

Gentoo@neo wget problem

I have a neo freerunner, on which i'm installing gentoo. I need to run layman -S but it fails to download repositories.xml from www.gentoo.org. When i try to download the latter with wget it fails ...
2
votes
3answers
2k views

HTTP responses curl and wget different results

To check HTTP response header for a set of urls I send with curl the following request headers foreach ( $urls as $url ) { // Setup headers - I used the same headers from Firefox version 2.0.0.6 ...
0
votes
3answers
1k views

Wget hangs in the middle of get request

Sometimes wget or curl randomly hangs in the middle of get request in Ubuntu 10.10: curl "http://api.indeed.com/ads/apisearch?publisher=7570038743238473" wget ...
22
votes
4answers
10k views

wget HEAD request?

I'd like to send HTTP HEAD request using wget, is it possible?
2
votes
2answers
2k views

wget authenticated HTTP Connection Failed

First off, I am not a Linux guy, but I have to pretend to be one sometimes as a web developer. So, we have a Red Hat server and I'm using wget in crontab to run some PHP scripts. We've been doing ...
-2
votes
5answers
998 views

Increasing download speed by using multiple IP addresses? - Ubuntu

quite often sites limit download speed per ip address or identified user. Is it possible to use a tool such as wget in order to overcome this limitation? Ideas would be great.
0
votes
3answers
3k views

unauthenticated http proxy, wget fails, curl works

I'm running a ubuntu 10.04 i386 desktop edition. I have configured the http proxy via the http_proxy variable. When I use wget I get a connection refused but, curl, apt-get etc. work just fine. Any ...
17
votes
4answers
22k views

How can I do a HTTP PUT with Wget?

I am trying to use Wget to access a RESTful interface, but I can not figure out how to do HTTP PUT with Wget. How can I do it? Or isn't it prossible?
11
votes
2answers
19k views

Linux command line tool for uploading files over HTTP as multipart/form-data?

I can see that wget has a --post-file option, but the manpage says "Wget does not currently support multipart/form-data for transmitting POST data; only application/x-www-form-urlencoded. Only one of ...
7
votes
3answers
10k views

How can I make wget rename downloaded files to not include the query string?

I'm downloading a site with wget and a lot of the links have queries attached to them, so when I do this: wget -nv -c -r -H -A mp3 -nd http://url.to.old.podcasts.com/ I end up with a lot of files ...