curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

976
votes
6answers
861k views

What is the cURL command-line syntax to do a POST request?

How do I make a POST request with cURL command-line tool?
12
votes
8answers
28k views

wget/cURL alternative native to Windows?

Is there a wget or cURL type command line utility native to Windows Vista? How does it work?
48
votes
5answers
101k views

Run cURL commands from Windows console

Is there a way to install cURL in windows in order to run cURL commands from the command prompt?
82
votes
5answers
79k views

Getting curl to output HTTP status code?

I'm using curl at the command line on Linux to issue HTTP requests. The response bodies are printed to standard out, which is fine, but I can't see from the man page how to get curl to print the ...
12
votes
6answers
43k views

CURL to download a directory

I am trying to download a full website directory using CURL. The following command does not work: curl -LO http://example.com/ It returns an error: curl: Remote file name has no length!. But when ...
0
votes
2answers
3k views

How to download a file behind authentication via wget or curl [duplicate]

Possible Duplicate: WGet a Page that Requires Logging in I wish to download file from http://mbr.nlm.nih.gov/Download/MetaMapped_Medline/2011/ however in order to download the file, i need ...
3
votes
2answers
3k views

Persistent retrying resuming downloads with curl

I'm on a mac and have a list of files I would like to download from an ftp server. The connection is a bit buggy so I want it to retry and resume if connection is dropped. I know I can do this with ...
24
votes
3answers
22k views

How to wget a file with correct name when redirected?

So after some time of searching on google and superuser (and scanning man pages) I was unable to find an answer to something that (I think) should be simple: If you go here: ...
8
votes
2answers
2k views

How to download parts of the same file from different sources with curl/wget?

I have a quite large file hosted on five different servers. I would like to be able to download different parts of the file from each server and subsequently concatenate the parts, in order to ...
16
votes
2answers
5k views

Can I make cURL fail with an exitCode different than 0 if the HTTP status code is not 200?

I was always assuming that when curl got an HTTP 500 response it was returning an exit code that meant failure (!= 0), but that seems to be not the case. Is there a way I can I make cURL fail with an ...
2
votes
1answer
6k views

Downloading only images using curl or wget?

UPDATED: I've found using this Bash script fixes the problem of having GIF files with a .jpg extension. I'm attempting to download images from a forum who's url uses the following format: ...
2
votes
2answers
996 views

curl not uploading file to ftp

I am using curl to upload a file to ftp server. The same script is working on another system, but on this one after connecting to ftp it is not doing anything and connection times out. The detailed ...
4
votes
3answers
14k views

URL to reboot a WebSTAR DPC2100R2 cable modem with curl?

Does anyone know what the magic URL is for rebooting a WebSTAR DPC2100R2 with curl? I used to have a SurfBoard, and the curl command: curl ...
2
votes
1answer
670 views

PHP/cURL/Fedora issues resolving hostname possibly a DNS problem?

I have the following simple test script for cURL/php on a FC12/apache server that has PHP 5.3.2, cURL v7.19.7 (according to phpinfo()): $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, ...
1
vote
2answers
1k views

Why sudo curl ignores proxy settings?

$ echo $http_proxy http://my.proxy.com $ curl -v http://files.com/a.txt * About to connect() to proxy my.proxy.com # Correct downloading $ sudo echo $http_proxy http://my.proxy.com $sudo curl -v ...
1
vote
2answers
1k views

curl can't connect; wget can

trying to connect to my script on a remote server with curl but: col-pos-spd# curl -v http://***:80/monitor/reply.php * About to connect() to *** port 80 (#0) * Trying 83.xxx.xxx.2... Connection ...
0
votes
2answers
1k views

Search for text in web pages given a list of URLs

I have a list of several thousand URLs, and I'd like to search each of these pages for a given word. How can I do this programmatically on Windows, preferably using VBScript or Powershell?