curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

0
votes
1answer
13 views

command option confusion in curl on Linux

Wondering what is the function of @ sign? Here is an example command, curl -w "@curl-format.txt" -o /dev/null -s "http://wordpress.com/" Here are more related backgrounds, http://stackoverflow.com/...
1
vote
2answers
5k views

CURL Error version OPENSSL_1.0.1 'not found …'

I have the last few days, a concern with CURL. I get this error: curl: / lib/i386-linux-gnu/libssl.so.1.0.0: version `OPENSSL_1.0.1 'not found (required by / usr/lib/i386-linux-gnu / libcurl.so.4) ...
0
votes
0answers
26 views

Schedule on crontab to reboot router

I would like your much aprreciated help. I have a ISP router (gateway) that doesen't have a schedule reboot on the web interface. I have a raspbian and I would like to schedule on crontab to reboot ...
220
votes
10answers
241k views

Getting curl to output HTTP status code?

I'm using curl at the command line on Linux to issue HTTP requests. The response bodies are printed to standard out, which is fine, but I can't see from the man page how to get curl to print the ...
0
votes
0answers
6 views

jq filtering curl- 'Cannot index number with string'

I am trying to filter a curl request with jq to get specific fields and put them in variables. Initially, I was using a basic curl request such as this: myResponse=$(curl -u myUsername:myPassword -...
0
votes
1answer
5k views

curl failed setting cipher list

I'm trying to make curl use GOST2001-GOST89-GOST89 cipher which is available and usable by OpenSSL but keep getting failed setting cipher list error despite the fact that curl sees gost engine and can ...
0
votes
1answer
1k views

No response from CURL at Windows Command Prompt

I've tried following Run cURL commands from Windows console but typing in any curl command returns nothing. See below - I just get back the command prompt. c:\curl>curl curl: try 'curl --...
2
votes
2answers
3k views

libcurl or cURL not working OSX Mavericks

Before I start let me say I have uninstalled, reinstalled, compiled from source, used macports and generally tried for the last 6 hours to get curl working on my Mac OSX Mavericks but I still haven't ...
0
votes
1answer
16 views

Check MD5 of a file header hosted on a webserver

I am writing a bash script to periodically download a setup file using cURL. The problem is the following, the file is around 110 MB and always have the same filename. I can download the file, check ...
1
vote
1answer
37 views

Could curl print latency in terms of milliseconds?

Wondering if there is an option for curl to print latency in milliseconds? An option that works on either Linux or Mac OSX would be good.
2
votes
1answer
24 views

openssl s_client returns DNS error

When trying to connect to a site using openssl, I get the following error. $> openssl s_client -connect www.citibank.com:443 getaddrinfo: No address associated with hostname connect:errno=2 But ...
0
votes
0answers
10 views

Use curl to download an LDAP attribute value only

I am using the following curl command to download a value from an LDAP directory: curl ldap://example.com/CN=ACME-Test-CRL,O=ACME?certificateRevocationList;binary -o crl.crl The problem is it ...
1
vote
0answers
26 views

can't connect to https API using cURL?

I have API endpoint URL, certificate file ( cert.p12 ) & password. Here is the steps I try to connect to this API. ( IN Windows 7 ) First I type the URL in chrome browser and it didn't give me ...
1
vote
1answer
19 views

how to use curl or wget to download a text file off a website?

i am trying to access eoddata.com to download a list of stock prices at the end of everyday without having to do it myself manually. the full url i am trying to access would be something like: http://...
1
vote
0answers
24 views

Centos cURL connection refused on Its own IP

I have two services on one server (with different ports to access them). One of my services is trying to connect to the other one with the IP of server and its specific port. But I got some errors ...
0
votes
1answer
7k views

Curl returns “Connection Refused” when trying to access XAMPP via localhost

I am running XAMPP (Apache, MySQL and PHP) on my Windows Vista computer. I have curl installed on my machine. When I try to curl my localhost in the command prompt, I get the following: curl -v -i ...
0
votes
0answers
42 views

curl: (56) Recv failure: Connection reset by peer

I am trying to download one file from my local server using curl.i am using command curl -L -O --retry 999 --retry-max-time 0 -C - http://192.168.1.74/gio-v2-9515.dd -u ajay:aj123 For experiment ...
2
votes
3answers
5k views

curl http_code of 000

I have a shell script that I use to monitor loading times and response codes on my live server cluster. It runs a total of 250 iterations every 5 minutes, distributed across 10 servers and 6 sites. It ...
1
vote
0answers
16 views

how to upload files to https server with certificates using curl or any other commands

I am Trying to upload to my HTTPS server using curl .I am using curl for that ,the code is curl -v --cacert /tmp/WIN-CLS.vdi.com.pem -T /etc/hosts \ --user administrator:pass_123 https://xxx.xxx....
0
votes
0answers
57 views

Can someone explain the differences of file_get_contents vs guzzle vs curl?

I've been getting data from api websites by using file_get_contents or curl, but I never knew whats the difference from them. They seem to have the same results, but now there is Guzzle for php. What ...
3
votes
1answer
2k views

Batch download of URLs from command line multithreaded

I have 100,000 URLs of small files to download. Would like to use 10 threads and pipelining is a must. I concatenate the result to one file. Current approach is: cat URLS | xargs -P5 -- curl >>...
1
vote
1answer
35 views

Curl command to download a file over HTTPS

I need to download a particular file from https://file_path. Here is the curl command I use for that: curl --user user:password "https://file_path?raw" > location I wish to download the real ...
2
votes
1answer
101 views

GIT from source: fatal: Unable to find remote helper for 'https'

Due to me not having root rights, I compiled git from source locally. My distro is OpenSUSE 13.1. Googling the error fatal: Unable to find remote helper for 'https' It was suggested that curl is not ...
0
votes
2answers
37 views

How can I archive someone's Facebook profile page?

I thought about using archive.is or archive.org. I tried to view the page in a private browser, but it asks to log in, so that site will see the same thing. I tried to open the developer console and ...
61
votes
6answers
65k views

PowerShell equivalent of curl

Is there an equivalent of curl in PowerShell? Does it have some similar built-in capability or is there a 3rd party cmdlet?
0
votes
1answer
30 views

Sending gist to github via cURL and issues with new lines inside file

So, let's say I have an index.php file containing this: <?= "Hello" ?> <?= echo "WORLD" ?> And I wanted upload the contents of this file to my gists in github, which I am doing via ...
1
vote
1answer
69 views

Can't update meteor, because it can't download manifest

I've meteor installed via npm and I'd like to update it, but I can't. I've tried: $ meteor update Failed to download manifest. but it fails with: Failed to download manifest. So then I've ...
0
votes
0answers
25 views

curl lookup time is 5.5 seconds when accessing address in /etc/hosts

In my /etc/hosts file I have a custom entry since the name doesn't actually exist like this: 5.5.5.5 someinternalserver.thing.net #thats a fake IP for this post Using curl like so: curl -o /dev/...
0
votes
1answer
56 views

How can you “press” a button of a webpage from the command line to restart a cable modem?

I'm trying to reboot my cable modem programmatically. I used to be able to do this by using wget: wget http://192.168.100.1/reset.htm?reset_modem=Restart+Cable+Modem However I changed my cable ...
1
vote
1answer
7k views

How can I send an HTTP GET request from the command line under WinPE?

I need to send an HTTP GET request under Windows Preinstallation Environment (WinPE). My first thought was curl.exe. It works well under Windows, but not WinPE. I committed curl.exe to boot.wim, but ...
0
votes
1answer
22 views

cURL command units of Dload speed?

When I run curl I sometimes see a progress meter like this: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total ...
0
votes
1answer
30 views

Psexec with curl or wget

I recently found out about psexec and was wondering if this program could help me run wget or curl commands on a remote computer. is this even possible?
26
votes
8answers
58k views

Wget/cURL alternative native to Windows?

Is there a Wget or cURL type command line utility native to Windows Vista? How does it work?
6
votes
1answer
1k views

How did Google get on my Mac?

I am running a MacBook Pro, and have never installed Chrome, Google Earth, or anything blatantly Google. Just installed Little Snitch (are there no good free firewalls for Mac?) and see that CURL is ...
4
votes
1answer
320 views

Automatic login to Vodafone Community WiFi (Fon) via Unix script

First I have to say that my question is similar to this one I am using a Vodafone Community Fon AP and it often disconnects. In OS X, which I'm using, there is a tool called Captive Network Assistant ...
0
votes
0answers
51 views

Bashscript using curl to crawl a website that needs a login

I am looking to crawl a website that has kind of a newsfeed with elements hat have a timestamp. There's only a certain amount of elements in the feed and a "Previous" button at the end of the feed. It ...
0
votes
1answer
68 views

Telling cUrl not to normalize a url

When using curl to call some url, it simplifies the path in the url (for example from /a/../b to /b). How can I tell cUrl not to do so? $ curl -V curl 7.35.0 (x86_64-pc-linux-gnu) libcurl/7.35.0 ...
0
votes
3answers
87 views

Sending a POST request via curl has issues with double quotes

So, in-order to send a POST request to gists (github), you can do something like this as seen in https://gist.github.com/caspyi... curl --user "user" -X POST --data '{"description":"Created via API","...
0
votes
1answer
160 views

curl command not found on OSX El Cap

My understanding is that curl should be installed. I had tried using curl from the Unix command line and got the message it could not be found. I'm wondering If something I did in efforts to clean up ...
1
vote
2answers
159 views

Curl response hangs

I have a site set up in a virtual box. I am able to access it just fine from a browser on my machine, but I have trouble accessing it via CURL while SSH'ed in to the box. When I try, curl hangs ...
0
votes
0answers
12 views

mirror site behind login

I would simply like to mirror all the pages (millions) on a site behind a login. This is for personal use. I have a login for the site in question but I want to scrape it into a database. I ...
-1
votes
1answer
34 views

How to use CURL tool?

I just got introduced to CURL today and I went to their website to download the tool and all I got is .cgi file called "metalink" and I don't know how to use it or how to run it, can someone help ...
1
vote
0answers
173 views

curl suddenly unable to get local issuer certificate

I've been using Docker 1.7.1 with base Ubuntu 14.04 image and all of a sudden this curl: curl -L -o sbt.tgz https://dl.bintray.com/sbt/native-packages/sbt/0.13.9/sbt-0.13.9.tgz is failing me with ...
0
votes
1answer
28 views

How to use VERB in url

I am trying to create and API which uses VERB PUT,GET,DELETE I can use it in curl by this curl -X DELETE http://localhost:8080/recipes/hi But how can i use similarly in url for eg This works ...
0
votes
1answer
59 views

Dictionary attack in Bash with cURL

I'm fairly new to Bash, but this is a script I've been working on for the past few days. It's meant to run a dictionary attack on an HTTP-POST login for a website, by reading the time module and ...
27
votes
3answers
15k views

Curl local host names on Mac OS X Yosemite

I just upgraded from Mavericks to Yosemite, and now curl can't see loopback host names. Set up a simple http server to test: $ python -m SimpleHTTPServer Serving HTTP on 0.0.0.0 port 8000 ... Now ...
1
vote
0answers
645 views

How to use .pfx in certificate store in CURL for Windows 7

I want to use cURL to access a remote server, and I need to identify myself using a private key or the remote server gives a 401 "Unauthorized". I'm on Windows 7 (work machine, not my choice) and my ...
1
vote
0answers
21 views

“curl” ignores proxy environment variable when using an ip address?

I am using curl to access some web services through a proxy (an instance of polipo running on port 8123, chained to an ssh SOCKS5 proxy). If I provide proxy information using the http_proxy ...
2
votes
1answer
3k views

Upload a file with a comma in its name with curl

How can I upload a file named yes, this filename has a comma.txt with the cli curl? You would normally do this to upload a file with curl: curl --progress-bar -F "fileUpload=@filename.txt" However ...