curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

1135
votes
6answers
1.0m views

What is the cURL command-line syntax to do a POST request?

How can I make a POST request with the cURL command-line tool?
109
votes
6answers
113k views

Getting curl to output HTTP status code?

I'm using curl at the command line on Linux to issue HTTP requests. The response bodies are printed to standard out, which is fine, but I can't see from the man page how to get curl to print the ...
52
votes
6answers
114k views

Run cURL commands from Windows console

Is there a way to install cURL in windows in order to run cURL commands from the command prompt?
42
votes
5answers
34k views

PowerShell equivalent of curl

Is there an equivalent of curl in PowerShell? Does it have some similar built-in capability or is there a 3rd party cmdlet?
39
votes
2answers
35k views

How do you display POST data with cURL?

As an example, POSTing to a web server with the -v argument: curl -v http://testserver.com/post -d "firstname=john&lastname=doe" And the output > POST /post HTTP/1.1 > User-Agent: ...
28
votes
3answers
26k views

How to wget a file with correct name when redirected?

So after some time of searching on google and superuser (and scanning man pages) I was unable to find an answer to something that (I think) should be simple: If you go here: ...
23
votes
3answers
4k views

Curl local host names on Mac OS X Yosemite

I just upgraded from Mavericks to Yosemite, and now curl can't see loopback host names. Set up a simple http server to test: $ python -m SimpleHTTPServer Serving HTTP on 0.0.0.0 port 8000 ... Now ...
23
votes
2answers
7k views

Can I make cURL fail with an exitCode different than 0 if the HTTP status code is not 200?

I was always assuming that when curl got an HTTP 500 response it was returning an exit code that meant failure (!= 0), but that seems to be not the case. Is there a way I can I make cURL fail with an ...
23
votes
2answers
70k views

How to send POST with body, headers, and HTTP params using cURL?

I found a lot of examples on how to use simple POST commands in cURL, but I didn't find examples on how to send full HTTP POST commands, which contain: Headers (Basic Authentication) HTTP Params ...
19
votes
2answers
2k views

Is “curl -u username:password http://example.com” secure?

Is curl -u username:password http://example.com secure? If not, can you give a brief explanation of how someone could obtain your password?
18
votes
4answers
19k views

cURL: how to suppress strange output when redirecting?

I'm trying to print just the verbose sections of a cURL request (which are sent to stderr) from the bash shell. But when I redirect stdout like this: curl -v http://somehost/somepage > /dev/null ...
15
votes
8answers
34k views

wget/cURL alternative native to Windows?

Is there a wget or cURL type command line utility native to Windows Vista? How does it work?
12
votes
6answers
48k views

CURL to download a directory

I am trying to download a full website directory using CURL. The following command does not work: curl -LO http://example.com/ It returns an error: curl: Remote file name has no length!. But when ...
11
votes
1answer
8k views

Why can't cURL properly verify a certificate on Windows?

When I try to use Curl on windows, to retrieve an https url, I get the dreaded "connection error (60)." The exact error message is: curl: (60) SSL certificate problem, verify that the CA cert ...
10
votes
2answers
6k views

how to use curl to verify if a site's certificate has been revoked?

To check if the certificate for google.com has been revoked, I tried the following command: curl https://www.google.com --cacert GeoTrust_Global_CA.pem --crlfile gtglobal.pem -v , but I got the ...
9
votes
2answers
2k views

How can I pass my Windows credentials in cURL?

I'm writing a script that uses cURL to pull data from an internal web server. I can explicitly use --user combined with --ntlm and everything works well, but I can't get it to recognize whomever's ...
9
votes
4answers
3k views

How “legal” is site-scraping using cURL? [closed]

Recently I was experimenting with the cURL, and I found lot is possible with it. I built a small script that crawls a musical site, which plays online songs. On the way of my experiment, I found that ...
8
votes
2answers
1k views

How to download parts of the same file from different sources with curl/wget?

I have a quite large file hosted on five different servers. I would like to be able to download different parts of the file from each server and subsequently concatenate the parts, in order to ...
8
votes
1answer
19k views

cURL Upload file and send POST data

I have a web server running some PHP that checks for an image (curl -F 'imageName=@myimage') and it also checks the POST data for username=&password=. When the PHP checks _REQUEST I cannot just ...
6
votes
1answer
520 views

How can I use a command line tool like Wget to log into an OpenID site?

Specifically, I would like to be able to download certain pages from my user profile on the various Stack Exchange sites. I would, however, like to do this automatically (using a cron job), from ...
5
votes
2answers
8k views

POSTing a file's contents with CURL

(The following is backstory, you can jump down to "Question:" if you'd like) There's a webpage for one of my courses that accepts input text (expected to be Python code) and then gives some tokenized ...
5
votes
1answer
531 views

How did Google get on my Mac?

Am running a MacBook Pro, and have never installed Chrome, Google Earth, or anything blatantly Google. Just installed Little Snitch (are there no good free firewalls for Mac?) and see that CURL is ...
5
votes
1answer
10k views

CURL File with post parameter

How to make a request for uploading a file to facebook using graph api in curl I am making this request but getting Error: curl https://graph.facebook.com/<id>/photos -F "source=@me.jpg" -d ...
5
votes
1answer
1k views

cURL hangs trying to upload file from stdin

I'm trying to PUT a file with cURL. This hangs: curl -vvv --digest -u user -T - https://example.com/file.txt < file This does not: curl -vvv --digest -u user -T file https://example.com/file.txt ...
5
votes
2answers
14k views

how to get and use certificates with curl

1) Entering the following command curl -k email@mail.com:password https://www.site.com Returns the following error <HTML><HEAD><meta http-equiv="content-type" ...
4
votes
4answers
21k views

Downloading file from FTP using cURL

I'm trying to use a cURL command to download a file from an FTP server to a local drive on my computer. I've tried curl "ftp://myftpsite" --user name:password -Q "CWD /users/myfolder/" -O ...
4
votes
3answers
2k views

Downloading XCode using wget or curl [closed]

Is there a way for downloading XCode using curl or wget? EDIT: Why is the following curl snippet not working? First, I login to Apple's site: curl -d "theAccountName=USERNAME&theAccountPW=PASS" ...
4
votes
2answers
3k views

Persistent retrying resuming downloads with curl

I'm on a mac and have a list of files I would like to download from an ftp server. The connection is a bit buggy so I want it to retry and resume if connection is dropped. I know I can do this with ...
4
votes
1answer
858 views

How to make alias/function in .bash_profile for `curl` that takes an argument?

I found this tip super helpful, for using curl to resume an interrupted file copy. The full syntax is: curl -C - -O file:///Volumes/path/to/file I would like to add an alias or function called ...
4
votes
1answer
1k views

Why does sudo not work with curl?

I ran a command to download something with curl and it threw a permission denied error. So I did sudo curl and I got the same error... Why is that? I was able to run the install command without ...
4
votes
3answers
14k views

URL to reboot a WebSTAR DPC2100R2 cable modem with curl?

Does anyone know what the magic URL is for rebooting a WebSTAR DPC2100R2 with curl? I used to have a SurfBoard, and the curl command: curl ...
4
votes
1answer
3k views

Where's the ca cert bundle on OSX?

I'm debugging some curl ssl configuration stuff and was wondering where the ca cert bundle is installed on OSX.
4
votes
1answer
2k views

Specifying minor TLS version when using curl

Is there a way to specify curl to use a specific TLS version? Like 1.1 or 1.2? I can see only sslv3 and tlsv1 options in command help. I took latest src and compiled it with openssl 1.0.1e. Still dont ...
3
votes
2answers
1k views

Curl: Difference between 2 ssl commands

I cant tell the difference or when to use one over the other. (Should this be on serverfault?) -E/--cert (SSL) Tells curl to use the specified certificate file when getting a file with ...
3
votes
2answers
4k views

How to see HTTP request header by curl?

I can see HTTP header response by curl -I command. Is there any option to see the HTTP request header sent? # curl -I www.google.com HTTP/1.1 200 OK Date: Sat, 31 Dec 2011 00:55:53 GMT Expires: -1 ...
3
votes
3answers
2k views

HTTP responses curl and wget different results

To check HTTP response header for a set of urls I send with curl the following request headers foreach ( $urls as $url ) { // Setup headers - I used the same headers from Firefox version 2.0.0.6 ...
3
votes
1answer
1k views

Batch download of URLs from command line multithreaded

I have 100,000 URLs of small files to download. Would like to use 10 threads and pipelining is a must. I concatenate the result to one file. Current approach is: cat URLS | xargs -P5 -- curl ...
3
votes
1answer
380 views

Unable to resolve host using curl/wget while page loads in browser

The problem I am facing is that while I open a link in my browser it loads quickly but whenever I try using the link in terminal with for e.g. curl -0 www.google.com or wget ...
3
votes
2answers
812 views

bash: source from URL

In addition to my own computer, I sometimes use an Ubuntu cluster at my school. Rather than manually keep my .bashrc's in sync, I would like to make the school cluster's .bashrc source my personal ...
3
votes
3answers
735 views

What's the equivalent of “wget -i FILE” in Curl?

What's the equivalent of "wget -i FILE" in Curl?
3
votes
1answer
362 views

Why is there a very long 6 second delay in start of Curl

I did a full install of Cygwin on my Windows 7 x64 machine. After each machine reboot, Curl can take around 6 seconds to start (even if no command line options were specified). On subsequent uses of ...
3
votes
1answer
4k views

Download a file from the latest stable Jenkins build

Using a batch file and wget/curl, how can I download the Multiverse-Core-.jar from here? http://ci.onarandombox.com/job/Multiverse-Core/lastStableBuild/artifact/target/ I want to use the timestamping ...
3
votes
1answer
1k views

Use cURL with multiple POSTs

I'm trying to use cURL to download the contents of webpages that require forms to get to. In a browser it looks something like this 1.) Login using POST 2.) Pick which page to go to using another ...
3
votes
1answer
45 views

Why does “curl” command work normally while “ping” not?

I find a strange phenomenon in my corporate network: I can execute curl google.com command successfully: [root@localhost ~]# curl google.com <HTML><HEAD><meta ...
3
votes
1answer
542 views

How can I get a list of all messages in my gmail account in linux command line?

How can I get a list of all messages in my gmail account in linux command line? Ideally I would want to see something similar to: from:sender1@domain.com to:me@gmail.com Date:11/22/33 subj:Foo1 ...
3
votes
1answer
1k views

VPN routing issue in linux- how to make applications that can only bind to IP addresses work?

Right now I have a PPTP VPN set up on my remote ubuntu VPS. I would like to have certain applications use the VPN to access the internet (rtorrent, irssi, among others). Currently, it is set up like ...
3
votes
1answer
334 views

Can cURL use a keyring for client certificates?

I'm using cURL to fetch from an HTTPS location, which requires a client certificate. I have my .curlrc set up to tell cURL where it is, but I don't want to leave my unencrypted private key just lying ...
3
votes
1answer
1k views

Force curl to download an empty file

In a bash script, I am using curl command on linux to download files from a FTP server. The command I am using is the following (the information about the FTP server connexion is stored in a .netrc ...
3
votes
1answer
590 views

curl: (7) Failed connect to www.google.com:80;

I have installed Git for Windows and got cURL automatically. C:\>curl --version curl 7.30.0 (i386-pc-win32) libcurl/7.30.0 OpenSSL/0.9.8y zlib/1.2.7 Protocols: dict file ftp ftps gopher http https ...
3
votes
1answer
2k views

How can I download private repo from github?

What I want to do is to download private repository archive from GitHub, extract it, remove archive file and copy some directories that are inside downloaded project. I tried to use wget but I cannot ...