curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

1
vote
1answer
98 views

What is the curl equivalent to wget -x?

I want to use curl to save multiple files with the same local paths as would be produced by wget -x (ie, cleaned up versions of the full urls). But I can't find the right option in the curl docs. Is ...
0
votes
0answers
140 views

What is the curl command line syntax to do a post request in windows

I know this question was answered often, but mine is some more special. I have the following Code: <form action="/test.html" method="post" enctype="multipart/form-data"> <a ...
0
votes
2answers
871 views

libcurl or cURL not working OSX Mavericks

Before I start let me say I have uninstalled, reinstalled, compiled from source, used macports and generally tried for the last 6 hours to get curl working on my Mac OSX Mavericks but I still haven't ...
1
vote
1answer
90 views

Curl won't work over FTP

I am trying to install Wine on my Macbook (OS X 10.9.3). First I tried using Homebrew which lists a set of dependencies and then goes to fetch the first. ==> Installing dependencies for wine: ...
0
votes
0answers
94 views

Curl to download large CSV

I am using curl through Cygwin on Windows 7 to download content from Apache Solr in CSV format. This is my command which is running in a for loop curl -o ret-key-data-$i.csv $URL As the content is ...
0
votes
0answers
22 views

Curl works, curlfrpfs doesn't

Following commands work properly: $ curl ftp://user:*****@sma20/ 05-04-14 14:45 872 config.xml 04-11-14 07:26 <DIR> DATA $ curl ...
0
votes
0answers
155 views

Trouble installing Homebrew “curl: (2) Failed initialization”

When I run the command to install homebrew on Mac OSX 10.7, namely ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)", I get curl: (2) Failed initialization. Searching ...
0
votes
1answer
123 views

cURL very long delay before starting download from ftp

I'm trying to write a script to download a file, convert it, then execute a program on the converted file. For some reason curl takes over 1 minute to start the file transfer. Command: curl ...
1
vote
0answers
22 views

Bug or syntax change on Debian's “curl” 7.36? Mess when trying to save sequential files with brackets/hash

(Perhaps specifically 7.36.0-1, which is shown as the most recent available on my package manager). I think there was such a bug some time ago, but perhaps it was back in curl 7.33, I can't ...
1
vote
2answers
439 views

How to stop shell script if curl failed

I have script that used curl when i pass wrong parameters to script curl failed but script continue executing. I have tried use curl -f/--fail parameter but problem does not solved. What is the best ...
10
votes
2answers
3k views

how to use curl to verify if a site's certificate has been revoked?

To check if the certificate for google.com has been revoked, I tried the following command: curl https://www.google.com --cacert GeoTrust_Global_CA.pem --crlfile gtglobal.pem -v , but I got the ...
0
votes
0answers
284 views

Why is SSL broken for one domain on one computer?

I've been having a strange issue, where I cannot connect to https://www.humblebundle.com. Plain HTTP works fine, as does https://humblebundle.com. These all redirect to the above address. I haven't ...
0
votes
0answers
62 views

wget sometimes fails, curl succeeds

I am continuously downloading some packets via wget. Once in a while, the wget fails. I have used retries as 2 and timeout as 10 seconds. When the failure occurs, I try the same URL again, but with ...
0
votes
0answers
245 views

curl: (35) error:0200100D:system library:fopen:Permission denied

In continuation with this question... I get curl error as above only as an unprivileged user on my system. However, on searching I found the above link which asks me to check permissions for ...
0
votes
0answers
34 views

Gzipped file arrives unzipped

I've recently set up an ArchLinux system at work and added the ArchHaskell repo. This is far from the first time I do this. I have a number of systems where it's worked just fine, but this last time ...
1
vote
1answer
180 views

posting data to a web server using curl

I am trying to post data to a web sever using curl -d and curl -X POST. The command is getting succeeded but nothing is there in the file. Tried the following: [root@oooatis santoshi]# curl ...
1
vote
0answers
38 views

SOAP-request realization in DOS 6.22 16-bit

I have a task to develop some application providing SOAP-requests realization for DOS. I have already developed a similar application under Linux, using C++/STL and curl and openssl libraries. Is ...
2
votes
1answer
1k views

How can I download private repo from github?

What I want to do is to download private repository archive from GitHub, extract it, remove archive file and copy some directories that are inside downloaded project. I tried to use wget but I cannot ...
1
vote
1answer
247 views

How do I comment out a line in my .curlrc?

I frequently have to switch between two different auth token request headers, that I keep in my .curlrc as such: -H "X-Auth-Token: 12345678-1234" It would make my life easier, though, if I could ...
0
votes
2answers
323 views

curl shell command always returns error status 0 - even when fails on input errors

Example: trying to upload a non existing file - curl -d @no-real-file http://google.com | echo $? - returns 0. How can I make curl fail on input errors?
2
votes
1answer
278 views

How can I get a list of all messages in my gmail account in linux command line?

How can I get a list of all messages in my gmail account in linux command line? Ideally I would want to see something similar to: from:sender1@domain.com to:me@gmail.com Date:11/22/33 subj:Foo1 ...
0
votes
0answers
107 views

Error uncompressing file when piping curl to gunzip

I need to download a gzipped xml feed to parse in my Ruby app. It may well be quite large, 30Mb compressed, 300Mb uncompressed. So I do this: curl -s http://example.com/file.xml.gz | gunzip and read ...
0
votes
0answers
318 views

Choose the filename when using POST with curl?

I'd like to POST a file upload with curl, and choose the filename at request time. Here's the curl line I have right now, which works to upload the file: curl --form "upload_file=@content.txt" ...
0
votes
0answers
105 views

Pinging local machine with external IP

On a Gentoo server, I am trying to wget or curl a page from its url: wget http://example.com/test.html (I can get result for this url from my browser) All I receive is Timeout. With cUrl I get ...
0
votes
0answers
19 views

Can One Extract a Hyphenated Filename File From a Server Using Curl?

The images I'm dealing with have file names like part-1.jpg, object-a.jpg, Aspect-0.jpg. What I'm wondering is what I put in place of ???: curl -O http://server/[???].jpg
0
votes
0answers
548 views

Libcurl not usable on mac os x

I'm running the configure command to install a particular software package, but it fails when it reaches it's libcurl requirement. This is the output it gives: checking for the version of ...
1
vote
1answer
108 views

Accessing a remote machine with or without ssh tunneling

I just want to educate myself on an interesting issue here. When I curl a URL I get this error curl "http://myserver:8888/payload" curl: (7) couldn't connect to host However when I setup a tunnel ...
0
votes
0answers
85 views

cURL POST on cygwin64 blocks for minutes

This curl command blocks for two and a half minutes and then finally executes. There is no activity on the Tomcat server during this time. curl -i -H "Content-type: application/json" -X POST ...
0
votes
0answers
336 views

Apahce proxy throws error connecting to a third party

Connection to a certain thirdparty service via php curl throws this error: CURL error: Received HTTP code 403 from proxy after CONNECT The apahe error log on the proxy throws this: Connect to ...
0
votes
0answers
224 views

curl with ftp URL works for SSL, but ftps URL doesn't: unknown protocol

I compiled and installed ProFTPD on my Mac for testing purposes and generated self-signed SSL certificates because I want to test ftps. When I do: curl --ftp-ssl-reqd --insecure ...
1
vote
0answers
228 views

pipe unlimited curl stdout into chunked files

I've got an HTTP-endpoint that serves an unlimited stream of events. Now I'd like to record that stream using curl, but splitting the recorded files at the same time - preferably based on the line ...
1
vote
1answer
307 views

calling json-rpc via curl

I'am running Bitcoin-qt 0.8.6.1-beta on Qt 4.8.3 on windows 7 64. below is my .conf content in %appdata%/bitcoin folder rpcuser=bitcoinrpc rpcpassword=ahything rpcallowip=127.0.0.1 ...
0
votes
1answer
643 views

Mac OSX 10.9 resume folder copy to NAS

I have a really big folder (174GB) that I wanted to move to a NAS using Ethernet. I started the transfer and when it was neary finished the computer stopped (I accidentally disabled caffeine) and the ...
1
vote
0answers
57 views

How do I log in to “The Cloud” wifi hotspots using curl?

In England at least, there's this accursèd thing called 'The Cloud', which increasingly provides all public wifi hotspots. Has anyone already knocked up a curl command/bash script/etc to log in to it ...
0
votes
1answer
127 views

File operations on FTPS server with cURL

I need to perform some file transfer operations on some remote FTPS server that uses certificate authentication from a Windows 2008 scripting environment, so that's why I'm stuck with cURL pretty ...
0
votes
1answer
75 views

A Script That is Aware of Remote File Change

curl -z pci.ids --remote-time --silent -f -L -O http://pciids.sourceforge.net/v2.2/pci.ids This Code Checks For The Remote File Change and Download If Remote File Has Been Modified but Is There a ...
0
votes
1answer
439 views

Command line utility to convert html to text

What would be the best command line utility to convert html to text? I want to curl a query to google, convert the output html to text and read converted text on my terminal. I am using rhel6.
0
votes
1answer
689 views

cURL: multiple POST requests while reusing the TCP connection

If multiple URLs are passed to cURL, it reuses the TCP connection when possible. For example: curl -o 1.jpg http://example.com/1.jpg -o 2.jpg http://example.com/2.jpg I need to do the same, but ...
2
votes
1answer
53 views

Shell commands executed from Vim behave differently

I have mapped a key in Vim to run curl-based tests on my API with :map ,t :w\|:!quizzical tests.sh <CR>. The tests run as expected when I press the key; however, the tests which take longer to ...
4
votes
2answers
11k views

how to get and use certificates with curl

1) Entering the following command curl -k email@mail.com:password https://www.site.com Returns the following error <HTML><HEAD><meta http-equiv="content-type" ...
1
vote
1answer
498 views

Omnisharp for Emacs doesn't work

I tried recently to work with C# with Emacs24. I found https://github.com/sp3ctum/omnisharp-emacs I installed it with MELPA. Then I run it. Nothing happened. I wonder what went wrong. I think I don't ...
-1
votes
1answer
230 views

Curl with ipv6 not working by default

I am trying to curl ipv6 adresses, but without success. If i try lets say: curl https://google.com I wait for a timeout and get network unreachable If i force ipv4, everything is fine. ...
2
votes
0answers
382 views

loop curl could not resolve host

I've had working script, which suddenly doesn't works. It contains something like this: while read url; do curl $url done < links links file: http://duckduckgo.com http://google.com And I ...
0
votes
2answers
429 views

SFTP - Recursively fetch new files

I have a remove file system structure like this: + /measure + / 2013-09 + / 2013-09-04 + / fooinstrument + result03343445845.csv + ...
1
vote
4answers
762 views

How does ping figure out the IP address of a website?

I'm trying to understand better the different network-query tools such as nc, curl, whois, dig, nslookup and at the same time a bit more about the architecture of internet requests, servers, and the ...
0
votes
1answer
290 views

Error with curl

I'm having this error with curl: curl https://www.google.com -v * About to connect() to www.google.com port 443 (#0) * Trying 2607:f8b0:400e:c01::67... * Failed to connect to ...
3
votes
1answer
1k views

Batch download of URLs from command line multithreaded

I have 100,000 URLs of small files to download. Would like to use 10 threads and pipelining is a must. I concatenate the result to one file. Current approach is: cat URLS | xargs -P5 -- curl ...
3
votes
1answer
294 views

Why is there a very long 6 second delay in start of Curl

I did a full install of Cygwin on my Windows 7 x64 machine. After each machine reboot, Curl can take around 6 seconds to start (even if no command line options were specified). On subsequent uses of ...
6
votes
1answer
421 views

How can I use a command line tool like wget to log into an OpenID site?

Specifically, I would like to be able to download certain pages from my user profile on the various SE sites. I would, however, like to do this automatically (using a cron job), from the command line ...
0
votes
0answers
232 views

Make curl work on ssh connection

On my windows xp machine, I can connect to a remote host via putty and psftp. However I fail to establish a connection with curl. When I enter curl -v --key priv_key.ppk --pubkey pub_key.pub ...