curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

3
votes
0answers
370 views

pipe unlimited curl stdout into chunked files

I've got an HTTP-endpoint that serves an unlimited stream of events. Now I'd like to record that stream using curl, but splitting the recorded files at the same time - preferably based on the line ...
2
votes
0answers
190 views

Posting XML wrapped in SOAP posted using Curl results in invalid XML character was found exceptions

Sending an XML file using curl results in: Invalid XML character (Unicode : 0x<integer>) Attempt to solve the issue According to this SO answer the issue will be solved if the XML version is ...
2
votes
0answers
215 views

Why doesn’t curl install and run on my 64-bit Windows 8 system but the 32-bit version works fine?

I almost gave up installing cURL on my 64-bit Windows 8 system. I tried almost every single .zip, .exe, etc… on this site, but none of them worked. I got this message every single time: The ...
2
votes
0answers
33 views

How to add a 'greek' character in the path using cURL?

I have a Makefile where one of many tasks is to upload a file using cURL, one of these files, has a greek character in the name and when I try to upload using cURL, I receive this error: curl: (43) A ...
2
votes
0answers
513 views

loop curl could not resolve host

I've had working script, which suddenly doesn't works. It contains something like this: while read url; do curl $url done < links links file: http://duckduckgo.com http://google.com And I ...
2
votes
0answers
181 views

Curl --insecure error in Cygwin (works in Linux)

Curl is returning an error in Cygwin. How do I eliminate this error? curl --insecure -X POST -d '<run><log ...
2
votes
0answers
808 views

Libcurl-devel on Mac OS X

I am trying to build an application that depends on libcurl version 7.21 or newer on Mac OS X 10.5.8. I tried downloading and installing the latest version from libcurl's website, but the application ...
2
votes
0answers
191 views

curl works but ping/urllib2 don't, in a virtual machine

I have a Crunchbang 10 VM running on VirtualBox on a Windows 7 Home host. I have a python script that uses urllib2 that was failing, and discovered that: ordinary browsing and curl work normally on ...
2
votes
0answers
467 views

PHP : curl is and Not working in ubuntu10.04 and Nginx

I am running Nginx server on ubuntu 10.04 I am using the php version 5.3.2 .I installed php5-curl using apt-get install php5-curl. Unfortunately the curl package is not displaying in phpinfo() . ...
2
votes
0answers
4k views

curl http_code of 000

I have a shell script that I use to monitor loading times and response codes on my live server cluster. It runs a total of 250 iterations every 5 minutes, distributed across 10 servers and 6 sites. It ...
2
votes
0answers
841 views

Restore default CA Certificates on Mac OSX 10.7

I've been screwing around with my Mac, trying to install self-signed certificate as trusted for our WAN site (for which we can't justify paying for a real signed certificate) I've broken something ...
1
vote
0answers
9 views

Curl in bash code - produces wrong status

This code seems to ouput a http status of 000 - which seems to indicate something didn't connect properly but when I do this curl outside of the bash script it works fine and produces a 200 so ...
1
vote
0answers
51 views

How do you download a zipped build from a jenkins job using “jenkinsapi” module in Python?

Has anyone tried download a build using jenkinsapi module in Python?
1
vote
0answers
54 views

Run bash command from link in browser

Is there a way to run an arbitrary command on OSX by clicking a link in the browser? As you know, there are certain types of hyperlinks that launch certain applications with a specified action, for ...
1
vote
0answers
127 views

FTP: wildcard downloading

I am trying to download a file with name ending with *.csv The problem is complicated by the fact that this *.csv contains in directory of unknown name: eg. Sometimes it would be 2015->Date ...
1
vote
0answers
454 views

How to use .pfx in certificate store in CURL for Windows 7

I want to use cURL to access a remote server, and I need to identify myself using a private key or the remote server gives a 401 "Unauthorized". I'm on Windows 7 (work machine, not my choice) and my ...
1
vote
0answers
234 views

I destroyed my Mac's PHP setup: SSL certification fails

Long story short, I deleted the PHP symlink on my Mac, and replaced it with a symlink from XAMPP. Now it seems that there's a problem with my SSL configuration or what seems to me, a lack thereof. ...
1
vote
0answers
13 views

Sporadic loss of access to local server

For some reason, I lose access to a local server called lux-app04 seemingly at random, but only in some applications. As you can see in the image below, Postman (top) can connect to it, but curl ...
1
vote
0answers
113 views

curl won't shut up (--silent does nothing)

I'm writing a quick shell script, and I cannot for the life of me get curl to suppress the progress bar. The relevant line looks like this: curl http://website -o $temp >/dev/null 2>&1 ...
1
vote
0answers
32 views

Bug or syntax change on Debian's “curl” 7.36? Mess when trying to save sequential files with brackets/hash

(Perhaps specifically 7.36.0-1, which is shown as the most recent available on my package manager). I think there was such a bug some time ago, but perhaps it was back in curl 7.33, I can't ...
1
vote
0answers
56 views

SOAP-request realization in DOS 6.22 16-bit

I have a task to develop some application providing SOAP-requests realization for DOS. I have already developed a similar application under Linux, using C++/STL and curl and openssl libraries. Is ...
1
vote
0answers
100 views

How do I log in to “The Cloud” wifi hotspots using curl?

In England at least, there's this accursèd thing called 'The Cloud', which increasingly provides all public wifi hotspots. Has anyone already knocked up a curl command/bash script/etc to log in to it ...
1
vote
0answers
259 views

Curl not working on OS X

I recently upgraded XCode and while I'm not sure if that caused the problem, I can't curl out of my computer anymore. This seems to break rvm and homebrew but git still works. I upgraded curl and ...
1
vote
0answers
155 views

Permissions tar extraction to nfs share from a Perl script

As part of an installation procedure of a piece of software, I use: curl -s <url-to-targz> | tar -p -x -z -C / inside a Perl script (I use qx($command) or system($command), either will ...
1
vote
0answers
142 views

No output returned when curl ing any URL in Red Hat 6 but the same url works in lynx browser

I am trying to install some packages in Python using easy install. But i get a 'Download Error' for any package i try to install. So i tried to test if internet is accessible at all from the machine ...
1
vote
0answers
212 views

the right options to traverse/download the pages/directories of a subdomain

Let's suppose exist a site with the following directories (subdomain) index.php |-sub1 |-index.php |-sub1sub1 |-index.php |-other.php |-sub1sub1sub1 |-sub2 |-index.php |- ...
1
vote
0answers
141 views

Download web server structure with empty files

I want to make a mirror of a Web server, but downloading the actual files will take too long. So I thought of having just the directory and file structure, and when I need the actual contents of the ...
1
vote
0answers
638 views

VirtualBox Ubuntu guest on OS X Lion host: Curl won't work

I'm trying to install some gems on an Ubuntu VM on OS X Lion using VirtualBox. I've narrowed my problem down to curl which is not responding at all. It seems to be going through some proxy local host ...
1
vote
0answers
214 views

how to use to proxy in the intranet?

i have an intranet site which is inside only our company and feeds information. i download an weather API from http://www.foreca.com/Philippines/ and hook to my site. the API has a search engine and ...
0
votes
0answers
8 views

Command to fetch a list of urls with connection keep-alive but fail on the first error (404/500)

I am trying to reproduce a potential bug in the Jetty web server with regard to serving static files. I need to repeatedly download a list of urls with "Connection: keep-alive" until one of them ...
0
votes
0answers
23 views

OSX terminal : log on on a website

I would like to login/ out on a Web site using the terminal, I currently using a appleScript I wrote but was wondering can you use terminal command line instead (not running my script from terminal) ...
0
votes
0answers
16 views

Is there a difference between curl --tlsv1 and --tlsv1.0?

The Curl man page lists -1 --tlsv1 and separately lists --tlsv1.0 along with v1.1 and v1.2. The descriptions are: -1, --tlsv1 (SSL) Forces curl to use TLS version 1 when negotiating ...
0
votes
0answers
7 views

Indexing file resources from a website

How can I get a list of all (image) resources from a website? Is there a better wait than using wget and parsing the log stream in order to get the list of all images? Worst is, I'm pretty use I ...
0
votes
0answers
9 views

Telling cUrl not to normalize a url

When using curl to call some url, it simplifies the path in the url (for example from /a/../b to /b). How can I tell cUrl not to do so? $ curl -V curl 7.35.0 (x86_64-pc-linux-gnu) libcurl/7.35.0 ...
0
votes
0answers
29 views

Using curl with a https proxy

I have setup a squid proxy so that it listens on a HTTPS port instead of the usual HTTP port, in order to have encrypted communication between the client and the proxy itself. While I have been able ...
0
votes
0answers
152 views

Notice: Use of undefined constant CURLOPT_CAINFO

I try to use TwitterOAuth which requires cURL. First of all, I think it is important to point that I use Win 8 64 bits with WampServer 3.0 and PHP 5.6.15 When I run the scripts (given by TwitterOAuth) ...
0
votes
0answers
27 views

Issues installing development tools to Ubuntu: curl: (35) Unknown SSL protocol error

I am attempting to download both Node.js with Linuxbrew, and RVM to an Ubuntu VM. Linuxbrew installs fine, but when I try to install Node.js with it, or RVM, I get this error: curl: (35) Unknown SSL ...
0
votes
0answers
22 views

curl -X POST not working properly?

Here's what I'm running as a test and it is working: curl -X POST http://zone-h.org/notify When I run that, I get the HTML of the page returned. However, when I run this, it isn't returning the ...
0
votes
0answers
39 views

Slow download over libraries on certain Windows systems

I am using custom libraries for file download (cURL and wget) on Windows client over HTTP from my server. On most of systems the download speed is as it should be - maximum determined by network ...
0
votes
0answers
34 views

curl to test secure web proxy

I have used Chrome to successfully connect to the secure web proxy(squid https_port) Chrome doc , but I want to know any command line tool(such as curl) will connect to it too. Curl's -x option ...
0
votes
0answers
34 views

Gmail Automation via Bash (To refresh POP Fetch from other accounts)

Gmail has this option to fetch mails from other accounts. I used this feature to create a central mail box. Works good. Gmail periodically polls other mail box over POP. But the frequency is 1 hour + ...
0
votes
0answers
45 views

WGET not working but CURL works on the same address/host?

WGET give me this error: HTTP request sent, awaiting response... 403 Forbidden 2015-11-23 12:07:06 ERROR 403: Forbidden. If I try the same URL (it’s an internal network Apache server, firewall ...
0
votes
0answers
17 views

windows curl --compressed : unrecognized command

I found the SAME problem: (si I know it can be duplicated) Curl Compressed Command not Recognized in Windows However no one of these solutions works for me (win7 / 64bits) When I use without the ...
0
votes
0answers
43 views

How to interact with submit input type with no name in php using curl

<select name="_limit"> <option value='50' >50</option> <option value='200' >200</option> <option value='300' >300</option> <option value='500' ...
0
votes
0answers
42 views

Curl Could not resolve host

I'm trying to get a webpage after submitting a username and password but i'm receiving an error. This is on Ubuntu server The command i'm using is curl -g -d ...
0
votes
0answers
26 views

How can I approximate impact of using proxies on cURL request time using CURLINFO_*_TIME?

Below is a small dataset from which I'm trying to answer two questions: How much time did the proxy take to connect to the API server? How much time did the API request take to return? The basic ...
0
votes
0answers
99 views

Make a curl POST request from scraped data

I scraped a website using the simple DOM php parser : <?php include 'domparse.php'; $html = file_get_html('http://oceanofgames.com/rebel-galaxy-free-download/'); ...
0
votes
0answers
35 views

Redirection not working for the first time

There are output of curl command. Sometimes my site doesn’t redirect properly, showing error page: no data received from the server. Is there any way to remove this error? ...
0
votes
0answers
37 views

How to receive sse (server sent events) and execute bash commands then

How do i receive server sent events (sse) from a webserver and execute a bash command with the received data in debian (raspbian)? Is that possible with curl as i can already receive sse. Thanks ...
0
votes
0answers
36 views

Amazon S3 upload error: An exception occurred while uploading parts to a multipart upload

I am trying to upload a 30 GB file to Amazon S3 using the AWS PHP SDK. require('../vendor/autoload.php'); $client = new Aws\S3\S3Client([ 'version' => 'latest', 'region' => 'us-east-1' ]); ...