curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

0
votes
1answer
16 views

cURL command units of Dload speed?

When I run curl I sometimes see a progress meter like this: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total ...
0
votes
1answer
17 views

Psexec with curl or wget

I recently found out about psexec and was wondering if this program could help me run wget or curl commands on a remote computer. is this even possible?
24
votes
8answers
55k views

Wget/cURL alternative native to Windows?

Is there a Wget or cURL type command line utility native to Windows Vista? How does it work?
-1
votes
0answers
23 views

monitoring websites using curl [on hold]

I am looking to monitor about 40 (Linux/Apache) urls hosted in a DMZ. I am thinking of using curl from a Linux box in the DMZ to do this and pass the data to a monitoring tool. My plan is to use a ...
1
vote
2answers
4k views

CURL Error version OPENSSL_1.0.1 'not found …'

I have the last few days, a concern with CURL. I get this error: curl: / lib/i386-linux-gnu/libssl.so.1.0.0: version `OPENSSL_1.0.1 'not found (required by / usr/lib/i386-linux-gnu / libcurl.so.4) ...
0
votes
1answer
7k views

Curl returns “Connection Refused” when trying to access XAMPP via localhost

I am running XAMPP (Apache, MySQL and PHP) on my Windows Vista computer. I have curl installed on my machine. When I try to curl my localhost in the command prompt, I get the following: curl -v -i ...
6
votes
1answer
1k views

How did Google get on my Mac?

I am running a MacBook Pro, and have never installed Chrome, Google Earth, or anything blatantly Google. Just installed Little Snitch (are there no good free firewalls for Mac?) and see that CURL is ...
4
votes
1answer
201 views

Automatic login to Vodafone Community WiFi (Fon) via Unix script

First I have to say that my question is similar to this one I am using a Vodafone Community Fon AP and it often disconnects. In OS X, which I'm using, there is a tool called Captive Network Assistant ...
0
votes
0answers
13 views

Bashscript using curl to crawl a website that needs a login

I am looking to crawl a website that has kind of a newsfeed with elements hat have a timestamp. There's only a certain amount of elements in the feed and a "Previous" button at the end of the feed. It ...
0
votes
1answer
45 views

Telling cUrl not to normalize a url

When using curl to call some url, it simplifies the path in the url (for example from /a/../b to /b). How can I tell cUrl not to do so? $ curl -V curl 7.35.0 (x86_64-pc-linux-gnu) libcurl/7.35.0 ...
0
votes
3answers
29 views

Sending a POST request via curl has issues with double quotes

So, in-order to send a POST request to gists (github), you can do something like this as seen in https://gist.github.com/caspyi... curl --user "user" -X POST --data '{"description":"Created via ...
0
votes
1answer
45 views

curl command not found on OSX El Cap

My understanding is that curl should be installed. I had tried using curl from the Unix command line and got the message it could not be found. I'm wondering If something I did in efforts to clean up ...
0
votes
1answer
4k views

curl failed setting cipher list

I'm trying to make curl use GOST2001-GOST89-GOST89 cipher which is available and usable by OpenSSL but keep getting failed setting cipher list error despite the fact that curl sees gost engine and can ...
1
vote
2answers
95 views

Curl response hangs

I have a site set up in a virtual box. I am able to access it just fine from a browser on my machine, but I have trouble accessing it via CURL while SSH'ed in to the box. When I try, curl hangs ...
2
votes
1answer
4k views

curl http_code of 000

I have a shell script that I use to monitor loading times and response codes on my live server cluster. It runs a total of 250 iterations every 5 minutes, distributed across 10 servers and 6 sites. It ...
200
votes
9answers
220k views

Getting curl to output HTTP status code?

I'm using curl at the command line on Linux to issue HTTP requests. The response bodies are printed to standard out, which is fine, but I can't see from the man page how to get curl to print the ...
0
votes
0answers
8 views

mirror site behind login

I would simply like to mirror all the pages (millions) on a site behind a login. This is for personal use. I have a login for the site in question but I want to scrape it into a database. I ...
-1
votes
1answer
29 views

How to use CURL tool?

I just got introduced to CURL today and I went to their website to download the tool and all I got is .cgi file called "metalink" and I don't know how to use it or how to run it, can someone help ...
1
vote
0answers
53 views

curl suddenly unable to get local issuer certificate

I've been using Docker 1.7.1 with base Ubuntu 14.04 image and all of a sudden this curl: curl -L -o sbt.tgz https://dl.bintray.com/sbt/native-packages/sbt/0.13.9/sbt-0.13.9.tgz is failing me with ...
0
votes
1answer
23 views

How to use VERB in url

I am trying to create and API which uses VERB PUT,GET,DELETE I can use it in curl by this curl -X DELETE http://localhost:8080/recipes/hi But how can i use similarly in url for eg This works ...
0
votes
1answer
30 views

Dictionary attack in Bash with cURL

I'm fairly new to Bash, but this is a script I've been working on for the past few days. It's meant to run a dictionary attack on an HTTP-POST login for a website, by reading the time module and ...
27
votes
3answers
14k views

Curl local host names on Mac OS X Yosemite

I just upgraded from Mavericks to Yosemite, and now curl can't see loopback host names. Set up a simple http server to test: $ python -m SimpleHTTPServer Serving HTTP on 0.0.0.0 port 8000 ... Now ...
1
vote
0answers
563 views

How to use .pfx in certificate store in CURL for Windows 7

I want to use cURL to access a remote server, and I need to identify myself using a private key or the remote server gives a 401 "Unauthorized". I'm on Windows 7 (work machine, not my choice) and my ...
1
vote
0answers
14 views

“curl” ignores proxy environment variable when using an ip address?

I am using curl to access some web services through a proxy (an instance of polipo running on port 8123, chained to an ssh SOCKS5 proxy). If I provide proxy information using the http_proxy ...
1
vote
1answer
3k views

Upload a file with a comma in its name with curl

How can I upload a file named yes, this filename has a comma.txt with the cli curl? You would normally do this to upload a file with curl: curl --progress-bar -F "fileUpload=@filename.txt" However ...
0
votes
1answer
10 views

Neither wget nor curl downloads this wikipedia api request

I can perfectly download this link and save it as a text file in a web browser environment. But I can't download and save that wikipedia api request using neither wget nor curl. I use wget -O file.txt ...
0
votes
2answers
56 views

How to POST file contents using cURL?

I want to do an HTTP POST of the contents (as a string) of a local file located at path/to/my-file.txt to a URL endpoint at http://example.com/. For example, I might want to do the following: ...
0
votes
0answers
163 views

Make a curl POST request from scraped data

I scraped a website using the simple DOM php parser : <?php include 'domparse.php'; $html = file_get_html('http://oceanofgames.com/rebel-galaxy-free-download/'); ...
0
votes
1answer
19 views

libcurl version 7.21 to work on Redhat 6

I am having an issue with libcurl version 7.19 and socks proxy on redhat 6. I believe it was fixed in libcurl version 7.21 how do i upgrade libcurl on redhat 6 to 7.21 or greater?
0
votes
2answers
68 views

cURL connecting to localhost (127.0.0.1) instead of destination IP

On Mac OS X I have an issue where calling: curl -v www.google.com results in: * Rebuilt URL to: www.google.com/ * Trying 62.253.72.153... * Connected to www.google.com (127.0.0.1) port 80 (#0) ...
0
votes
3answers
485 views

Is it possible to get the download file name of an mp3 from server using bash?

I am trying to get the final download file name of an mp3 from a server, specifically audiotool.com e.g. http://api.audiotool.com/track/haunt_opaque/mixdown.mp3 When you navigate to this url in a ...
2
votes
0answers
138 views

Retrieving from ftp dirs with wildcards

What is the most efficient way to retrieve all relevant data from ftp if the address of said data can be specified with wildcards as below: ...
1
vote
0answers
204 views

SSL error when connecting to git from Mac OS X 10.11.3 (El Capitan)

When I try to clone / pull / fetch from a git repository via https from a Mac System I keep getting an error: error: RPC failed; result=56, HTTP code = 200 fatal: The remote end hung up unexpectedly ...
8
votes
2answers
659 views

How can I use a command line tool like Wget to log into an OpenID site?

Specifically, I would like to be able to download certain pages from my user profile on the various Stack Exchange sites. I would, however, like to do this automatically (using a cron job), from ...
3
votes
2answers
103 views

curl check if file is newer and instead of downloading - execute a bash (or python) script

I am having a bit of an issue. I have a file, hosted on a remote server (http://mywebsite/file.zip). I also have a few embedded linux boxes (running openelec OS). The boxes have fairly limited ...
61
votes
7answers
153k views

Run cURL commands from Windows console

Is there a way to install cURL in windows in order to run cURL commands from the command prompt?
1
vote
1answer
35 views

Fetch source file from GitHub using command line tools?

This question is related to a question offsite at FATAL: MemorySanitizer can not mmap the shadow memory. The person who responded said: Do simple tests (e.g. this one) work for you? The link ...
-1
votes
2answers
24 views

Where is my file saved locally?

I've downloaded a file--I see the contents of my text file in the terminal window as it is downloaded in curl, but where is it saved on my local machine: curl -v -O myFile.txt -u username:password ...
0
votes
0answers
29 views

Finding a text string on a website and all its links

Can you help me understand process of finding a certain text string on a website and all its immediate links? I started delving into wget and curl with help of grep but was unable to come up with ...
1
vote
2answers
2k views

How to convert HTML to text?

How it is possible to convert HTML to text file in Linux? For example I want to curl a query to Google, then convert the output html to text and read converted text on my terminal. I am using RHEL6.
1
vote
1answer
37 views

Is there a difference between curl --tlsv1 and --tlsv1.0?

The Curl man page lists -1 --tlsv1 and separately lists --tlsv1.0 along with v1.1 and v1.2. The descriptions are: -1, --tlsv1 (SSL) Forces curl to use TLS version 1 when negotiating ...
0
votes
1answer
43 views

How to echo only 1st line from 'curl' command output?

I'm trying to get only first line of the curl command output. (Sorry if this is confusing) Let's say, for an instance, i run simply: # curl http://localhost <!-- This is the hidden line i want to ...
0
votes
1answer
22 views

How to Check value for an input element after fetching contents of page using cUrl

I am trying to download a page using cUrl. I have saved the page to a file and I then grep on it to extract the value of the text field. The text field appears multiple times and I just need one ...
39
votes
4answers
40k views

How to wget a file with correct name when redirected?

So after some time of searching on Google and Super User (and scanning man pages) I was unable to find an answer to something that (I think) should be simple: If you go here: ...
2
votes
1answer
20 views

Using variable in a CURL command

Am trying insert a document into my cloudant as below. @echo OFF SPEECH_TEXT="Call me Ishmael. Some years ago-never mind how long precisely-" curl -X POST -k 'https://<user ...
0
votes
0answers
17 views

Command to fetch a list of urls with connection keep-alive but fail on the first error (404/500)

I am trying to reproduce a potential bug in the Jetty web server with regard to serving static files. I need to repeatedly download a list of urls with "Connection: keep-alive" until one of them ...
9
votes
2answers
24k views

How to get and use certificates with curl

1) Entering the following command curl -k email@mail.com:password https://www.example.com Returns the following error <HTML><HEAD><meta http-equiv="content-type" ...
0
votes
1answer
68 views

How to open cURL in Firefox or other web browser?

I have Firebug installed in Firefox. In the Net panel there are all network requests listed. When I right-click and choose Copy as cURL, how do I use this cURL? How can I repeat my POST request data ...
0
votes
0answers
27 views

OSX terminal : log on on a website

I would like to login/ out on a Web site using the terminal, I currently using a appleScript I wrote but was wondering can you use terminal command line instead (not running my script from terminal) ...
1
vote
3answers
10k views

Upload folder with cURL and FTP using Batch file on Windows?

I am trying to upload a whole folder, including it's sub-folders and files to a folder on my server from a Windows batch (.bat) file. Below is the command I have for the Upload part... curl -T ...