curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

1
vote
1answer
9 views

Using variable in a CURL command

Am trying insert a document into my cloudant as below. @echo OFF SPEECH_TEXT="Call me Ishmael. Some years ago-never mind how long precisely-" curl -X POST -k 'https://<user ...
0
votes
0answers
8 views

Command to fetch a list of urls with connection keep-alive but fail on the first error (404/500)

I am trying to reproduce a potential bug in the Jetty web server with regard to serving static files. I need to repeatedly download a list of urls with "Connection: keep-alive" until one of them ...
0
votes
1answer
14 views

How to open cURL in Firefox or other web browser?

I have Firebug installed in Firefox. In the Net panel there are all network requests listed. When I right-click and choose Copy as cURL, how do I use this cURL? How can I repeat my POST request data ...
0
votes
0answers
23 views

OSX terminal : log on on a website

I would like to login/ out on a Web site using the terminal, I currently using a appleScript I wrote but was wondering can you use terminal command line instead (not running my script from terminal) ...
0
votes
0answers
16 views

Is there a difference between curl --tlsv1 and --tlsv1.0?

The Curl man page lists -1 --tlsv1 and separately lists --tlsv1.0 along with v1.1 and v1.2. The descriptions are: -1, --tlsv1 (SSL) Forces curl to use TLS version 1 when negotiating ...
0
votes
1answer
45 views

TLS curl POST resulting in SSLv3 error

I am trying to make a POST request via curl to a server that only supports TLS 1.2, TLS 1.1, and TLS 1.0. However, even when I try to specify the need for TLS instead of the default SSL v. 3 (adding ...
0
votes
0answers
7 views

Indexing file resources from a website

How can I get a list of all (image) resources from a website? Is there a better wait than using wget and parsing the log stream in order to get the list of all images? Worst is, I'm pretty use I ...
0
votes
0answers
9 views

Telling cUrl not to normalize a url

When using curl to call some url, it simplifies the path in the url (for example from /a/../b to /b). How can I tell cUrl not to do so? $ curl -V curl 7.35.0 (x86_64-pc-linux-gnu) libcurl/7.35.0 ...
0
votes
0answers
20 views

Using curl with a https proxy

I have setup a squid proxy so that it listens on a HTTPS port instead of the usual HTTP port, in order to have encrypted communication between the client and the proxy itself. While I have been able ...
0
votes
0answers
126 views

Notice: Use of undefined constant CURLOPT_CAINFO

I try to use TwitterOAuth which requires cURL. First of all, I think it is important to point that I use Win 8 64 bits with WampServer 3.0 and PHP 5.6.15 When I run the scripts (given by TwitterOAuth) ...
0
votes
0answers
26 views

Issues installing development tools to Ubuntu: curl: (35) Unknown SSL protocol error

I am attempting to download both Node.js with Linuxbrew, and RVM to an Ubuntu VM. Linuxbrew installs fine, but when I try to install Node.js with it, or RVM, I get this error: curl: (35) Unknown SSL ...
0
votes
0answers
21 views

curl -X POST not working properly?

Here's what I'm running as a test and it is working: curl -X POST http://zone-h.org/notify When I run that, I get the HTML of the page returned. However, when I run this, it isn't returning the ...
0
votes
1answer
28 views

Curl POST syntax work under Windows?

I am trying to issue the following command under Windows 10: D:\>curl -i -X POST -H "Content-Type:application/json" -d '{ "firstName" : "Frodo", "lastName" : "Baggins" }' ...
0
votes
1answer
117 views

PHP CURL SSL CA cert (path? access rights?)

Curl through php is constantly complaining about Problem with the SSL CA cert (path? access rights?). This all happend after trying to fix another problem with curl ssl with pem cert chain not ...
1
vote
1answer
315 views

curl 'SSL connect error'

basic steps taken: # echo -n | openssl s_client -showcerts -connect example.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/example.cert # curl -v --cacert /tmp/example.cert ...
0
votes
0answers
36 views

Slow download over libraries on certain Windows systems

I am using custom libraries for file download (cURL and wget) on Windows client over HTTP from my server. On most of systems the download speed is as it should be - maximum determined by network ...
1
vote
0answers
9 views

Curl in bash code - produces wrong status

This code seems to ouput a http status of 000 - which seems to indicate something didn't connect properly but when I do this curl outside of the bash script it works fine and produces a 200 so ...
0
votes
0answers
32 views

curl to test secure web proxy

I have used Chrome to successfully connect to the secure web proxy(squid https_port) Chrome doc , but I want to know any command line tool(such as curl) will connect to it too. Curl's -x option ...
0
votes
0answers
34 views

Gmail Automation via Bash (To refresh POP Fetch from other accounts)

Gmail has this option to fetch mails from other accounts. I used this feature to create a central mail box. Works good. Gmail periodically polls other mail box over POP. But the frequency is 1 hour + ...
-2
votes
1answer
34 views

Filter curl header output in verbose mode

Using curl -v ..., i.e. the verbose mode that prints out the input and output headers. However, this info is not piped and I can not grep-out lines which I dont need. How can you filter curl header ...
0
votes
0answers
45 views

WGET not working but CURL works on the same address/host?

WGET give me this error: HTTP request sent, awaiting response... 403 Forbidden 2015-11-23 12:07:06 ERROR 403: Forbidden. If I try the same URL (it’s an internal network Apache server, firewall ...
0
votes
2answers
88 views

Using cURL to post data

I try to post to localhost with some parameters in the URL. Everything just works fine in Fiddler. But cURL doesn't allow me to do it, with the error message below. curl --data "Task=bake" ...
0
votes
0answers
17 views

windows curl --compressed : unrecognized command

I found the SAME problem: (si I know it can be duplicated) Curl Compressed Command not Recognized in Windows However no one of these solutions works for me (win7 / 64bits) When I use without the ...
0
votes
1answer
43 views

cURL in shell vs. knockd

I want to use curl to control features of a IPcam. This works fine in a shell (/bin/sh) but fails when called from knockd. I am trying this on a QNAP NAS (Busybox Linux). The script I am using is: ...
0
votes
0answers
42 views

How to interact with submit input type with no name in php using curl

<select name="_limit"> <option value='50' >50</option> <option value='200' >200</option> <option value='300' >300</option> <option value='500' ...
0
votes
0answers
41 views

Curl Could not resolve host

I'm trying to get a webpage after submitting a username and password but i'm receiving an error. This is on Ubuntu server The command i'm using is curl -g -d ...
0
votes
0answers
26 views

How can I approximate impact of using proxies on cURL request time using CURLINFO_*_TIME?

Below is a small dataset from which I'm trying to answer two questions: How much time did the proxy take to connect to the API server? How much time did the API request take to return? The basic ...
0
votes
0answers
96 views

Make a curl POST request from scraped data

I scraped a website using the simple DOM php parser : <?php include 'domparse.php'; $html = file_get_html('http://oceanofgames.com/rebel-galaxy-free-download/'); ...
0
votes
1answer
121 views

CURL in bash script works with . (dot space), not ./ (dot slash)

I have a bash script which sends a couple of notifications to Slack via CURL and spawns an expect script. When I run the script via . scriptName.sh it works fine, but when I run it by typing ...
0
votes
0answers
33 views

Redirection not working for the first time

There are output of curl command. Sometimes my site doesn’t redirect properly, showing error page: no data received from the server. Is there any way to remove this error? ...
1
vote
2answers
56 views

I'm trying to open query.yahooapis.com but curl is trying to use IPV6 even if I only have IPV4 enabled

When I try to call query.yahooapis.com with curl it's trying to use the ipv6 address but IPV6 is disabled on my computer $ curl -v 'http://query.yahooapis.com/' * Hostname was NOT found in DNS cache ...
0
votes
1answer
199 views

Installing and using cURL in Linux and use it in Eclipse as well?

I come from a Windows environment - I used cURL in Eclipse (C). I want to do the same in a Linux environment (Ubuntu). I followed the installation guide on how to install cURL, but how do you use cURL ...
0
votes
0answers
36 views

How to receive sse (server sent events) and execute bash commands then

How do i receive server sent events (sse) from a webserver and execute a bash command with the received data in debian (raspbian)? Is that possible with curl as i can already receive sse. Thanks ...
0
votes
0answers
36 views

Amazon S3 upload error: An exception occurred while uploading parts to a multipart upload

I am trying to upload a 30 GB file to Amazon S3 using the AWS PHP SDK. require('../vendor/autoload.php'); $client = new Aws\S3\S3Client([ 'version' => 'latest', 'region' => 'us-east-1' ]); ...
0
votes
0answers
210 views

Sending POST with esp8266 Module

I am trying to send a POST request from my esp8266 wifi module and I am getting errors. Such as: "Your browser sent a request that this server could not understand" I am trying to send: POST ...
0
votes
1answer
80 views

Delivering a payload to a URL, using cURL

I need help figuring out the cURL request required to send the following data to a URL. {"requestid":"555555", "partnermatchid":"10000-000-0000-0000-000", ...
0
votes
0answers
99 views

wget or cURL for dynamic recursive URLs (generated through directories and adding specific same text at the end)

I'm using a wget command with some options in order to refresh the cache module from all URLs of a Prestashop. This is the command: wget -r -D www.domain.com -l3 -P temp -x -R "*cart*","*order*","*My ...
1
vote
1answer
57 views

How can I run the equivalent “curl” command that performs similarly to this “wget” command?

I have a wget command: wget -r --no-parent -nH -nd -q --show-progress --progress=bar:force ftp://ftp.ncdc.noaa.gov/pub/data/swdi/database-csv/v2/ But it only works in newer versions of wget. ...
1
vote
2answers
60 views

Download a list of files and use domain name as filename

I got a bunch of files to download which works great with wget -i list_of_urls but my problem is, that wget uses the filename of the downloaded file. Is there a way (or a different tool) which is ...
0
votes
1answer
27 views

How to extract ip address from no-ip account webpage in the shell?

I've set dynamic dns for my home network(which is behind NAT) using www.noip.com. I use ddclient to send updates of my ip, but sometimes it takes a while to update the ip. Anyways it's not the point. ...
4
votes
2answers
242 views

Get curl to output non-2xx to stderr, but 2xxs to a file

I would like some curl command, possibly with some bash hackery, that: Outputs any 2xx responses to a file Outputs any non-2xx responses to stderr Exits with a nonzero status code when a non-2xx ...
1
vote
1answer
48 views

How to run a remote bat file on a local computer

I need to run a batch file, that is stored on a server, locally on a windows 8.1 machine. I can run a shell script in OS X with a curl: cURL "file path" | sh is it possible to cURL in powershell in ...
0
votes
1answer
219 views

how to make curl re-use existing connection

curl --verbose google.com If I execute this command, after curl does its job, it says "Connection #0 to host google.com left intact". But it doesn't use same connection if I execute same command ...
1
vote
0answers
51 views

How do you download a zipped build from a jenkins job using “jenkinsapi” module in Python?

Has anyone tried download a build using jenkinsapi module in Python?
0
votes
1answer
99 views

What is the curl equivalent of wget -qO (wget -q -O)?

I need to run the following command in Red Hat Linux: wget -q -O http://somepage/somefile.cgi| bash But I don't have wget installed. I could install it, but I want to know how to do this with curl. ...
4
votes
1answer
376 views

What units does curl use for bandwidth?

curl on the command-line displays progress like this: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent ...
0
votes
1answer
125 views

How to upload all the files from local to host using Curl?

i have written a shell script to upload all files with different names in the specific path from local to host, but i have facing some issue and the files are not getting uploaded #!/bin/sh for file ...
0
votes
0answers
362 views

Curl: How to send custom headers while using proxy

I want to send custom headers to the host while using proxies. Take a look at the examples below. Explaining the situation with examples was easier. This works fine as expected: curl ...
2
votes
2answers
116 views

Fast way to download many small webpages?

I need to download lots of small pages as fast as possible. Average size for one page is 3kb. There are millions of pages. First, I tried curl but it was way too slow, probably because it was waiting ...
1
vote
1answer
529 views

What is the correct way to submit content-type header in curl?

I’m using Mac 10.9.5 with bash shell. I’m trying to submit a request via curl, but I keep getting errors when I try and set the content-type header. Below I try davea$ curl -v -o -H "Content-Type: ...