1
vote
0answers
81 views

MacOS Mavericks: wget getting stuck with local ip but not with localhost

I am facing a weird problem on my Macbook (Mavericks). I'm serving some files over http. When I try to get the files using wget: it works perfectly using localhost or 127.0.0.1 it starts but get ...
0
votes
1answer
68 views

Download all images at distance 2 using Mac OS X

I need to download all the images from a website on my computer. The images are not listed all in one page. Instead I have an index page that links a set of pages (page1.html, page2.html, ...) each ...
1
vote
1answer
234 views

Do curl and wget open gzipped files automatically?

I made two http calls to the same gzipped resource from two different machines - RHEL and OSX. curl http://example.com/file.gz The first call returned an expected binary output: /�IQ���} The ...
2
votes
1answer
154 views

How do I disable my internet using curl or wget to hit my router in a script?

I want to be able to run some kind of shell script/curl commands to toggle my TV services by accessing a form on my router page using curl or wget. I have been able to get past authentication using ...
1
vote
0answers
285 views

Create a complete local copy of a website using Wget

OK, here is what I need : I'm on Mac (Mac OS X 10.6.8) I want to completely mirror a website on my hard drive (this is what I'm using as a test) I want all images and prerequisites there, so that ...
1
vote
2answers
3k views

crontab - /bin/sh: wget: command not found

I want to set up a crontab on my Macbook Pro running Lion. I run the usual crontab -e and provide a quick job to see if everything is working ok --- */10 * * * * wget -O - -q -t 1 ...
0
votes
0answers
156 views

Chrome downloads 60 times slower than wget?

I just downloaded a large file (the KyngChaos PostgreSQL package, an 8.5MB file). In Chrome, it was downloading at less than 1KB/sec, and said it would take over an hour to arrive. When I tried ...
0
votes
2answers
571 views

Can storing 300k files in one folder cause problems?

I'm crawling a large website (over 200k pages) using wget (is there a better tool btw?). Wget is saving all the files to one directory. The partition is HFS (I think), will it cause problems if I ...
5
votes
3answers
1k views

Where do I install certificates so that wget and other MacPorts programs will find them?

I have to install custom certificates so that they work with wget installed via MacPorts. I can't find the right directory. I've tried installing in /System/Library/OpenSSL/certs but that directory ...
7
votes
2answers
1k views

How to download parts of the same file from different sources with curl/wget?

I have a quite large file hosted on five different servers. I would like to be able to download different parts of the file from each server and subsequently concatenate the parts, in order to ...