curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

1
vote
2answers
257 views

Curl Compressed Command not Recognized in Windows

I saw this post and successfully added curl to my Windows command prompt as instructed. However, when I attempted to run curl --compressed, it says: "the installed libcurl version doesn't ...
1
vote
2answers
1k views

Why sudo curl ignores proxy settings?

$ echo $http_proxy http://my.proxy.com $ curl -v http://files.com/a.txt * About to connect() to proxy my.proxy.com # Correct downloading $ sudo echo $http_proxy http://my.proxy.com $sudo curl -v ...
2
votes
1answer
2k views

curl: (7) couldn't connect to host

I know some people have asked this question before but I don't understand some of the answers. I'm a Mac user trying to instal Homebrew but I keep getting this error message: curl: (7) couldn't ...
1
vote
0answers
120 views

Download web server structure with empty files

I want to make a mirror of a Web server, but downloading the actual files will take too long. So I thought of having just the directory and file structure, and when I need the actual contents of the ...
5
votes
1answer
472 views

How did Google get on my Mac?

Am running a MacBook Pro, and have never installed Chrome, Google Earth, or anything blatantly Google. Just installed Little Snitch (are there no good free firewalls for Mac?) and see that CURL is ...
2
votes
2answers
2k views

How do I unzip a compressed file as it's downloading?

I'm downloading a large file that's not an archive, and I want to combine the tasks of downloading and decompressing. How can I do them simultaneously?
3
votes
1answer
712 views

How to make alias/function in .bash_profile for `curl` that takes an argument?

I found this tip super helpful, for using curl to resume an interrupted file copy. The full syntax is: curl -C - -O file:///Volumes/path/to/file I would like to add an alias or function called ...
0
votes
1answer
903 views

How do I enable the curl progress bar in cygwin?

When you curl a file, you get this progress bar whether you're uploading or downloading a file. % Total % Received % Xferd Average Speed Time Time Time Current ...
2
votes
1answer
239 views

curl works at the command line, but how do i put it in a php file?

The following curl format works for me from the command line. The line below is just sample data, but the format works. curl -u email@email.com:1234567 https://api.email.com/status How do I ...
0
votes
2answers
727 views

What should I do in order to build curl without an error?

This failed when I run ./buildconf The error information is as follows: [mirror@home curl]$ ls acinclude.m4 CMakeLists.txt GIT-INFO MacOSX-Framework packages TODO-RELEASE ...
0
votes
1answer
282 views

What tool does yum use to download rpm packages from remote server?

My system is centos, and now I am installing packages using yum command. Updating for dependencies: apr-util i386 1.3.9-1.el5 lxcenter-updates 203 k gcc-gfortran ...
1
vote
3answers
4k views

Silence curl with >/dev/null 2>&1 when it's piped

So I have a curl that is piped to a grep and a sed. Where would I apply the >/dev/null 2>&1 ? curl www.site.com | grep stuff | sed "other stuff" At the very end or after the curl?
1
vote
1answer
2k views

How do I remotely fetch files from redirected URLs from a terminal?

I want to fetch a tarball of this python library from terminal. https://github.com/simplegeo/python-oauth2/downloads However, I cannot simply call wget ...
1
vote
1answer
699 views

Grab source from Google Code using wget or curl?

Is it possible to grab the source-code from a Google Code repository (hg) using wget or curl? Else using Python? Thanks for all suggestions FYI: I'm on a shared WebFaction host, so can't install hg ...
5
votes
1answer
10k views

CURL File with post parameter

How to make a request for uploading a file to facebook using graph api in curl I am making this request but getting Error: curl https://graph.facebook.com/<id>/photos -F "source=@me.jpg" -d ...
11
votes
1answer
7k views

Why can't cURL properly verify a certificate on Windows?

When I try to use Curl on windows, to retrieve an https url, I get the dreaded "connection error (60)." The exact error message is: curl: (60) SSL certificate problem, verify that the CA cert ...
0
votes
1answer
3k views

Curl Couldn't resolve host for HTTPS

I am trying to use curl on a Red Hat Enterprise Linux Server release 6.1. When I try to use URL with HTTPS it gives an error message "Couldn't resolve host". But for the same URL when tried with HTTP ...
0
votes
2answers
593 views

test of ICMP block

In my bash scripts I have been using something like: until fping -u google.com; do echo "$0[$$] Network/DNS down?? $(date)" 1>&2 && sleep $(($RANDOM%(1 + ++trynum * 1) +1)).222; done ...
0
votes
1answer
1k views

Why can't I fetch this sites contents with curl?

I'm trying to download the contents of https://aarhustech.itslearning.com/elogin/default.aspx with curl, but I get this error: $ curl https://aarhustech.itslearning.com/elogin/default.aspx curl: (35) ...
0
votes
1answer
396 views

apt-get install new packeges, avoid others

Look at this entry: root@vivid-75:~# apt-get --no-install-recommends install curl Reading package lists... Done Building dependency tree Reading state information... Done The following extra ...
0
votes
1answer
3k views

How can I use curl with a SSL URL to get a 200 OK?

I'm trying to use curl to get the HTTP status of a GET request: curl --insecure --silent --show-error --connect-timeout 1 -I https://host:8443/health Note: I'm using the --insecure flag in this ...
0
votes
1answer
451 views

How to make curl reconnect after 60"?

Sometimes, curl freezes: $ curl -O 'http://www.site.com/index.html' www.site.com/index.html --> index.html % Total % Received % Xferd Average Speed Time Time Time Current ...
0
votes
1answer
626 views

Apache and PHP with cURL

I am running Sun server with Solaris 10/SPARC. I had/have Apache 1.37 with PHP 5.2.1 running as a dynamic extension, and want to enable cURL support. I have tried to re-compile PHP with the ...
0
votes
1answer
2k views

curl failed setting cipher list

I'm trying to make curl use GOST2001-GOST89-GOST89 cipher which is available and usable by OpenSSL but keep getting failed setting cipher list error despite the fact that curl sees gost engine and can ...
3
votes
1answer
2k views

VPN routing issue in linux- how to make applications that can only bind to IP addresses work?

Right now I have a PPTP VPN set up on my remote ubuntu VPS. I would like to have certain applications use the VPN to access the internet (rtorrent, irssi, among others). Currently, it is set up like ...
0
votes
2answers
1k views

How to use wget to grab copy of Google Code site documents?

I have a Google Code project which has a lot of wiki'ed documentation. I would like to create a copy of this documentation for offline browsing. I would like to use wget or a similar utility. I have ...
0
votes
2answers
920 views

command line curl - file naming example.com/[1-10].txt => 01.txt through 10.txt

Is it possible to set the variable portion of the file name output to be a specific length and fill the extra spaces with zeros. I want things to be ordered for other users, so I would like curl ...
0
votes
1answer
350 views

Allow Incoming Responses Apache. On Ubuntu 11.10 - Curl

I'm trying to get a Curl Response from an outside server, however I noticed I cant neither PING the server in question nor connect to it. I tried disabling the iptables firewall but I had no success. ...
2
votes
1answer
2k views

Cannot connect to website - SSL handshaking fails

So I cannot connect to certain websites. Just a few, most are OK. The one I really care about is paypal.com. I have done the usual things. Let's see: Cleared browsing history Tried different ...
1
vote
1answer
3k views

How to stop running download process by CURL?

I forgot to use > filename after curl operation. linux console is giving contiguously garbage data(hopefully downloading and displaying the content on console). How can we stop this process (ctrl-c ...
2
votes
1answer
6k views

Downloading only images using curl or wget?

UPDATED: I've found using this Bash script fixes the problem of having GIF files with a .jpg extension. I'm attempting to download images from a forum who's url uses the following format: ...
1
vote
1answer
645 views

How to use delicious API from command line

I'm trying to add a delicious bookmark from a shell script, but keep getting the mysterious "something went wrong" error. I've checked the official spec and several examples and everything looks ...
1
vote
1answer
410 views

curl (7) permission denied on cgi script

Can't understand which permissions could be denied. Running cgi script from command line works fine. http_proxy="" curl http://url/ -o file Produces curl(7): permission denied (same issues with ...
1
vote
2answers
2k views

Can I encrypt a username password combo before using it with cURL?

I'm making a shell script that reached out to a web service with basic auth. I don't want to have to put in my username and password every time the script runs, but I also don't want to have the combo ...
1
vote
1answer
1k views

Submitting a form using curl

Forgive me for the simple question... I haven't used curl before. I'm trying to automatically download some data from this form: http://ida.water.usgs.gov/ida/available_records.cfm?sn=12040500 which ...
2
votes
0answers
573 views

Restore default CA Certificates on Mac OSX 10.7

I've been screwing around with my Mac, trying to install self-signed certificate as trusted for our WAN site (for which we can't justify paying for a real signed certificate) I've broken something ...
1
vote
1answer
1k views

How do I install hiphop-php in CentOS 6?

I've been trying to install hiphop-php on CentOS 6 with no luck. I found an .rpm for it but it fails dependency checks on boost which fails to install because it depends on curl which fails to ...
3
votes
1answer
293 views

Can cURL use a keyring for client certificates?

I'm using cURL to fetch from an HTTPS location, which requires a client certificate. I have my .curlrc set up to tell cURL where it is, but I don't want to leave my unencrypted private key just lying ...
3
votes
2answers
4k views

How to see HTTP request header by curl?

I can see HTTP header response by curl -I command. Is there any option to see the HTTP request header sent? # curl -I www.google.com HTTP/1.1 200 OK Date: Sat, 31 Dec 2011 00:55:53 GMT Expires: -1 ...
0
votes
1answer
13k views

How to install CURL on Debian Squeeze?

I just can't get CURL to install on Debian Squeeze... keeps giving me this: libtool: compile: gcc -DHAVE_CONFIG_H -I../include/curl -I../include -I../include -I../lib -I../lib -g0 -O2 ...
0
votes
2answers
2k views

How can I curl the output of another command

I want to pass curl the output from awk ./jspider.sh http://www.mypage.com | grep 'resource' | awk '{print $4}' | curl OUTPUT_FROM_AWK | grep myString How can I achieve this?!
1
vote
1answer
467 views

Scrape a site with curl/wget appending a parameter to each site

I need to scrape a website for all sites and append an Paramater, e.g. www.theSite.com/allpages?myPara=us Now what I got is wget --recursive to get all sites, but that doesn't append the parameter ...
1
vote
1answer
2k views

When using curl to http post a xml file to a url, how to set the parameter?

I want to use curl to http post a file to a url like: curl -d myfile.xml http://www.example.dom/post/file/here Can I set the parameter I want the file to be posted with? Like if it was a ...
21
votes
2answers
58k views

How to send POST with body data, headers, and HTTP params over the cURL command line?

I found a lot of examples on how to use simple POST commands in cURL, but I didn't find examples on how to send full HTTP POST commands, which contain: Headers (Basic Authentication) HTTP Params ...
0
votes
1answer
312 views

Only output curl result dependent on mime-type

Is it possible to make curl only save the output of curl -o if the file is of a certain mime-type? For example, I might use the following command to download a bunch of images off a server... curl ...
0
votes
1answer
571 views

No response from CURL at Windows Command Prompt

I've tried following Run cURL commands from Windows console but typing in any curl command returns nothing. See below - I just get back the command prompt. c:\curl>curl curl: try 'curl ...
0
votes
0answers
821 views

cURL does not work on verbose mode

I have installed curl on a new ubuntu installation with apt-get and it behaves really weird. When I do curl http://www.google.co.uk everything work ok, but when I do curl -v ...
34
votes
5answers
26k views

PowerShell equivalent of curl

Is there an equivalent of curl in PowerShell? Does it have some similar built-in capability or is there a 3rd party cmdlet?
1
vote
1answer
1k views

compiling curl with HipHop for PHP Patch [closed]

I'm following this guide: https://github.com/facebook/hiphop-php/wiki/Building-and-installing-on-ubuntu-10.10 It want's me to download curl source, doing the patch and compile this. well, but i get ...
1
vote
2answers
1k views

curl can't connect; wget can

trying to connect to my script on a remote server with curl but: col-pos-spd# curl -v http://***:80/monitor/reply.php * About to connect() to *** port 80 (#0) * Trying 83.xxx.xxx.2... Connection ...