curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

797
votes
6answers
700k views

What is the cURL command-line syntax to do a POST request?

How do I make a POST request with cURL command-line tool?
50
votes
4answers
51k views

Getting curl to output HTTP status code?

I'm using curl at the command line on Linux to issue HTTP requests. The response bodies are printed to standard out, which is fine, but I can't see from the man page how to get curl to print the ...
40
votes
5answers
86k views

Run cURL commands from Windows console

Is there a way to install cURL in windows in order to run cURL commands from the command prompt?
31
votes
5answers
19k views

PowerShell equivalent of curl

Is there an equivalent of curl in PowerShell? Does it have some similar built-in capability or is there a 3rd party cmdlet?
25
votes
2answers
20k views

How do you display POST data with cURL?

As an example, POSTing to a web server with the -v argument: curl -v http://testserver.com/post -d "firstname=john&lastname=doe" And the output > POST /post HTTP/1.1 > User-Agent: ...
17
votes
3answers
17k views

How to wget a file with correct name when redirected?

So after some time of searching on google and superuser (and scanning man pages) I was unable to find an answer to something that (I think) should be simple: If you go here: ...
17
votes
2answers
46k views

How to send POST with body data, headers, and HTTP params over the cURL command line?

I found a lot of examples on how to use simple POST commands in cURL, but I didn't find examples on how to send full HTTP POST commands, which contain: Headers (Basic Authentication) HTTP Params ...
14
votes
4answers
12k views

cURL: how to suppress strange output when redirecting?

I'm trying to print just the verbose sections of a cURL request (which are sent to stderr) from the bash shell. But when I redirect stdout like this: curl -v http://somehost/somepage > /dev/null ...
11
votes
6answers
38k views

CURL to download a directory

I am trying to download a full website directory using CURL. The following command does not work: curl -LO http://example.com/ It returns an error: curl: Remote file name has no length!. But when ...
10
votes
8answers
23k views

wget/cURL alternative native to Windows?

Is there a wget or cURL type command line utility native to Windows Vista? How does it work?
10
votes
1answer
5k views

Why can't cURL properly verify a certificate on Windows?

When I try to use Curl on windows, to retrieve an https url, I get the dreaded "connection error (60)." The exact error message is: curl: (60) SSL certificate problem, verify that the CA cert ...
9
votes
2answers
487 views

how to use curl to verify if a site's certificate has been revoked?

To check if the certificate for google.com has been revoked, I tried the following command: curl https://www.google.com --cacert GeoTrust_Global_CA.pem --crlfile gtglobal.pem -v , but I got the ...
8
votes
2answers
2k views

Can I make cURL fail with an exitCode different than 0 if the HTTP status code is not 200?

I was always assuming that when curl got an HTTP 500 response it was returning an exit code that meant failure (!= 0), but that seems to be not the case. Is there a way I can I make cURL fail with an ...
8
votes
2answers
1k views

How to download parts of the same file from different sources with curl/wget?

I have a quite large file hosted on five different servers. I would like to be able to download different parts of the file from each server and subsequently concatenate the parts, in order to ...
7
votes
4answers
3k views

How “legal” is site-scraping using cURL? [closed]

Recently I was experimenting with the cURL, and I found lot is possible with it. I built a small script that crawls a musical site, which plays online songs. On the way of my experiment, I found that ...
7
votes
1answer
16k views

cURL Upload file AND send POST data

I have a web server running some PHP that checks for an image (curl -F 'imageName=@myimage') and it also checks the POST data for username=&password=. When the PHP checks _REQUEST I can just do: ...
5
votes
2answers
4k views

POSTing a file's contents with CURL

(The following is backstory, you can jump down to "Question:" if you'd like) There's a webpage for one of my courses that accepts input text (expected to be Python code) and then gives some tokenized ...
5
votes
1answer
409 views

How did Google get on my Mac?

Am running a MacBook Pro, and have never installed Chrome, Google Earth, or anything blatantly Google. Just installed Little Snitch (are there no good free firewalls for Mac?) and see that CURL is ...
5
votes
1answer
7k views

CURL File with post parameter

How to make a request for uploading a file to facebook using graph api in curl I am making this request but getting Error: curl https://graph.facebook.com/<id>/photos -F "source=@me.jpg" -d ...
5
votes
1answer
2k views

cURL hangs trying to upload file from stdin

I'm trying to PUT a file with cURL. This hangs: curl -vvv --digest -u user -T - https://example.com/file.txt < file This does not: curl -vvv --digest -u user -T file https://example.com/file.txt ...
4
votes
4answers
13k views

Downloading file from FTP using cURL

I'm trying to use a cURL command to download a file from an FTP server to a local drive on my computer. I've tried curl "ftp://myftpsite" --user name:password -Q "CWD /users/myfolder/" -O ...
4
votes
2answers
1k views

How can I pass my Windows credentials in cURL?

I'm writing a script that uses cURL to pull data from an internal web server. I can explicitly use --user combined with --ntlm and everything works well, but I can't get it to recognize whomever's ...
4
votes
1answer
758 views

Why does sudo not work with curl?

I ran a command to download something with curl and it threw a permission denied error. So I did sudo curl and I got the same error... Why is that? I was able to run the install command without ...
4
votes
3answers
13k views

URL to reboot a WebSTAR DPC2100R2 cable modem with curl?

Does anyone know what the magic URL is for rebooting a WebSTAR DPC2100R2 with curl? I used to have a SurfBoard, and the curl command: curl ...
3
votes
3answers
2k views

Downloading XCode using wget or curl [closed]

Is there a way for downloading XCode using curl or wget? EDIT: Why is the following curl snippet not working? First, I login to Apple's site: curl -d "theAccountName=USERNAME&theAccountPW=PASS" ...
3
votes
2answers
876 views

Curl: Difference between 2 ssl commands

I cant tell the difference or when to use one over the other. (Should this be on serverfault?) -E/--cert (SSL) Tells curl to use the specified certificate file when getting a file with ...
3
votes
2answers
2k views

How to see HTTP request header by curl?

I can see HTTP header response by curl -I command. Is there any option to see the HTTP request header sent? # curl -I www.google.com HTTP/1.1 200 OK Date: Sat, 31 Dec 2011 00:55:53 GMT Expires: -1 ...
3
votes
1answer
557 views

How to make alias/function in .bash_profile for `curl` that takes an argument?

I found this tip super helpful, for using curl to resume an interrupted file copy. The full syntax is: curl -C - -O file:///Volumes/path/to/file I would like to add an alias or function called ...
3
votes
2answers
2k views

Persistent retrying resuming downloads with curl

I'm on a mac and have a list of files I would like to download from an ftp server. The connection is a bit buggy so I want it to retry and resume if connection is dropped. I know I can do this with ...
3
votes
3answers
686 views

What's the equivalent of “wget -i FILE” in Curl?

What's the equivalent of "wget -i FILE" in Curl?
3
votes
1answer
581 views

Batch download of URLs from command line multithreaded

I have 100,000 URLs of small files to download. Would like to use 10 threads and pipelining is a must. I concatenate the result to one file. Current approach is: cat URLS | xargs -P5 -- curl ...
3
votes
1answer
214 views

Why is there a very long 6 second delay in start of Curl

I did a full install of Cygwin on my Windows 7 x64 machine. After each machine reboot, Curl can take around 6 seconds to start (even if no command line options were specified). On subsequent uses of ...
3
votes
1answer
1k views

Use cURL with multiple POSTs

I'm trying to use cURL to download the contents of webpages that require forms to get to. In a browser it looks something like this 1.) Login using POST 2.) Pick which page to go to using another ...
3
votes
1answer
2k views

VPN routing issue in linux- how to make applications that can only bind to IP addresses work?

Right now I have a PPTP VPN set up on my remote ubuntu VPS. I would like to have certain applications use the VPN to access the internet (rtorrent, irssi, among others). Currently, it is set up like ...
3
votes
1answer
263 views

Can cURL use a keyring for client certificates?

I'm using cURL to fetch from an HTTPS location, which requires a client certificate. I have my .curlrc set up to tell cURL where it is, but I don't want to leave my unencrypted private key just lying ...
3
votes
1answer
911 views

Force curl to download an empty file

In a bash script, I am using curl command on linux to download files from a FTP server. The command I am using is the following (the information about the FTP server connexion is stored in a .netrc ...
3
votes
0answers
624 views

Specifying minor TLS version when using curl

Is there a way to specify curl to use a specific TLS version? Like 1.1 or 1.2? I can see only sslv3 and tlsv1 options in command help. I took latest src and compiled it with openssl 1.0.1e. Still dont ...
3
votes
1answer
326 views

How can I use a command line tool like wget to log into an OpenID site?

Specifically, I would like to be able to download specific pages from my user profile on the various SE sites. I would, however, like to do this automatically (using a cron job), from the command line ...
2
votes
2answers
1k views

Curl alternative on windows to get header information [duplicate]

Possible Duplicate: wget/cURL alternative native to Windows? Is there any tool (command line tool preferred) to get the HTTP request header information on Windows machine like Curl does on ...
2
votes
2answers
2k views

How do I unzip a compressed file as it's downloading?

I'm downloading a large file that's not an archive, and I want to combine the tasks of downloading and decompressing. How can I do them simultaneously?
2
votes
1answer
20k views

How to install curl on ubuntu

System: ibook G4 / linux ubuntu 2.6.28-6-powerpc OS : ubuntu 9.04(jaunty) PHP ver : 5.2.6-3ubuntu4.6 Please let me know how to install curl on my system. Synaptic Package Manager tells me to ...
2
votes
3answers
2k views

HTTP responses curl and wget different results

To check HTTP response header for a set of urls I send with curl the following request headers foreach ( $urls as $url ) { // Setup headers - I used the same headers from Firefox version 2.0.0.6 ...
2
votes
1answer
2k views

Using cURL through SSH tunnel or VPN

I would like to set my CURL to use SSH tunneling for certain domains. How can I accomplish that? I can also set up VPN or SOCKS or whatever, but I need to use CURL on local machine, but use the IP of ...
2
votes
1answer
886 views

Download a file from a website in command line in Linux

I need to download a file from a website which has a URL formatted like: http://www.server.com/downloads/1234/ This redirects to a .zip file which has to be saved. There is also a need to ...
2
votes
2answers
798 views

curl not uploading file to ftp

I am using curl to upload a file to ftp server. The same script is working on another system, but on this one after connecting to ftp it is not doing anything and connection times out. The detailed ...
2
votes
2answers
612 views

bash: source from URL

In addition to my own computer, I sometimes use an Ubuntu cluster at my school. Rather than manually keep my .bashrc's in sync, I would like to make the school cluster's .bashrc source my personal ...
2
votes
2answers
623 views

I am not able access certain website [closed]

I am trying to access www.psychologytoday.com and it fails every time. I tried proxysites and still but every time connection got refused. I've checked Google cached page of the same. Only a few ...
2
votes
1answer
2k views

Download a file from the latest stable Jenkins build

Using a batch file and wget/curl, how can I download the Multiverse-Core-.jar from here? http://ci.onarandombox.com/job/Multiverse-Core/lastStableBuild/artifact/target/ I want to use the timestamping ...
2
votes
1answer
243 views

Unable to resolve host using curl/wget while page loads in browser

The problem I am facing is that while I open a link in my browser it loads quickly but whenever I try using the link in terminal with for e.g. curl -0 www.google.com or wget ...
2
votes
1answer
2k views

curl: (7) couldn't connect to host

I know some people have asked this question before but I don't understand some of the answers. I'm a Mac user trying to instal Homebrew but I keep getting this error message: curl: (7) couldn't ...