curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

0
votes
1answer
7 views

How to download the latest version of youtube-dl programmatically?

The latest version of youtube-dl is at: https://yt-dl.org/downloads/latest/ which redirects to (as of today): https://yt-dl.org/downloads/2015.07.04/ I'd like to set up a cron job to download the ...
0
votes
2answers
36 views

Curl post form and get result

I found this website http://www.fonefinder.net/findome.php , that locates phone provider by number (ex: http://www.fonefinder.net/findome.php?npa=562&nxx=866&thoublock=1666) shows VERIZON ...
0
votes
1answer
8 views

How can I instruct curl to reuse credentials after it followed a redirect?

I know that curl can follow redirects (the -L argument), and I know that I can provide credentials to a request (-u username:password), but I want to use both all the way down the redirect chain. How ...
5
votes
1answer
74 views

Firebug and cURL

I have been using Firebug and cURL for quite a while. Firebug is amazing to capture HTTPS POST request as opposed to a network analyser as it is aware of SSL/TLS negotiation. And cURL is amazing to ...
2
votes
1answer
24 views

Archiving websites like Internet Archive

Most of the data for my research needs are met by our Government websites. Problem is they take down websites or certain pages or break the links. But my research demands the access to information ...
0
votes
0answers
25 views

curl could not open remote file permission denied using sftp

I'm trying to download a file from a new enduser sftp using existing code that works for other end user sftps, using curl. This is my basic command: curl -o [internalFileLocation] -u [user:pw] ...
0
votes
1answer
18 views

Using a curl scheduled task in windows

I am using the following command in a scheduled task in windows to update my IP to the free Dynamic DNS service called EntryDNS: curl.exe -k -X PUT -d "" "Their url API string here" The problem I am ...
1
vote
0answers
23 views

Run bash command from link in browser

Is there a way to run an arbitrary command on OSX by clicking a link in the browser? As you know, there are certain types of hyperlinks that launch certain applications with a specified action, for ...
2
votes
1answer
28 views

Site displays with Curl but not within the browser

I've set up a dev-enviroment on Debian 7 and installed the following services: Apache Bind9 I have one site that is reachable through "asix.debian.local". I've set up the zone and added NS and A ...
1
vote
0answers
9 views

PHP cURL Request for Web Service Returning Error [migrated]

I continue to receive the error: "Failed to de-serialize entity" when making the REST request below. I'm assuming it has to do with the formatting of my json, but can't for the life of me figure it ...
0
votes
1answer
8 views

curl ignore --data starting with @ sign: don't read from file

In slack you can script slackbot to post messages to a channel like this: curl --data "$msg" $'https://<yourteam>.slack.com/services/hooks/slackbot?token=<yourtoken>&channel=#random' ...
3
votes
1answer
108 views

Why does “curl” command work normally while “ping” not?

I find a strange phenomenon in my corporate network: I can execute curl google.com command successfully: [root@localhost ~]# curl google.com <HTML><HEAD><meta ...
0
votes
0answers
43 views

Why do wget and curl fail to download page source?

Via a shell script, I would like to download the source for a web page with air quality data on aqicn.org. Here is the URL for the page with Beijing air quality data: http://aqicn.org/city/beijing/m/ ...
21
votes
2answers
3k views

Is “curl -u username:password http://example.com” secure?

Is curl -u username:password http://example.com secure? If not, can you give a brief explanation of how someone could obtain your password?
0
votes
3answers
41 views

Why can't the guest of VirtualBox connect to network?

I have installed CentOS 7 on virtualbox, and network setting is NAT. The guest can't connect to network: systemctl -l status network output likes this: ip addr output likes this: Could anyone ...
0
votes
0answers
33 views

Pipe remote jpg webcam image to ffmpeg as video

I have a cheap ip webcam. All it provides is a single http url to a static jpg file. Such as http:///image.jpg. Repeatedly requesting the same url will provide subsequent frames from the ip camera. ...
1
vote
1answer
28 views

curl capturing http status and timing the request

I have a loop in wich I check urls from file one-by-one and want to get the status header and the execution time of every request as result. I have two curl commands: This one outputs the header ...
0
votes
2answers
85 views

use curl to download and yum to install (NTLM proxy issues)

I need to use a wi-fi under a corporate environment with my Fedora 21. It uses NTLM. With curl i've been able to reach sites (yahoo) setting the proxy, the type of proxy, the user and the password, ...
0
votes
1answer
46 views

Using curl to simulate Internet Explorer's auto logon

I have a webpage on my intranet in which Internet Explorer can automatically login using whatever user is logged in the network. How can I use curl (or: what are the HTTP headers I need) in order to ...
2
votes
0answers
73 views

Posting XML wrapped in SOAP posted using Curl results in invalid XML character was found exceptions

Sending an XML file using curl results in: Invalid XML character (Unicode : 0x<integer>) Attempt to solve the issue According to this SO answer the issue will be solved if the XML version is ...
0
votes
0answers
19 views

Return response body with curl always, but if non 2XX, don't exit with 0

How can I have curl print the response body, but if the http status code is not 2XX don't exit with a success 0 status? Using the flag --fail does not work, because it does NOT print the response ...
1
vote
1answer
36 views

debugging a curl request / what would be the curl request to test whether a given username / password is working against a speciific smtp server

I have a customer using a specific app that uses secure smtp to connect for outgoing email. I have a suspicion that it somehow got changed (whether by them or by the hosting provider is irrelevant). I ...
1
vote
0answers
25 views

Access Websites & Modify Web Contents [closed]

so I'm given a website, and it requires me to use a token to get in. Is there any way I can do this on a normal web browser? I've been using curl -X GET in the Linux terminal. Also, how can I modify ...
-1
votes
2answers
152 views

How to use,run cURL tool with Android Studio on windows environment? [duplicate]

What is better way to use cURL tool with Android Studio 1.1.x on Windows? because there is no proper information in official site for client tools on windows environment. I tried to download from ...
2
votes
0answers
74 views

Why doesn’t curl install and run on my 64-bit Windows 8 system but the 32-bit version works fine?

I almost gave up installing cURL on my 64-bit Windows 8 system. I tried almost every single .zip, .exe, etc… on this site, but none of them worked. I got this message every single time: The ...
0
votes
1answer
140 views

understanding sending post request (with curl)

alright, I know related questions have been answered : What is the cURL command-line syntax to do a POST request? http://stackoverflow.com/questions/14978411/http-post-and-get-using-curl-in-linux ...
0
votes
1answer
35 views

Send HTTP request with parameters file

My question is very simple : is it possible to send a HTTP request from the *nix command line specifying a file, containing all the parameters, as an argument? In REST APIs documentation, we often ...
0
votes
2answers
90 views

Log date and external ip address with curl and batch (Windows)

I am having weird issues where my external IP is changed VERY often as in every 2, 3 minutes. I have a dynamic dns service in place that I update via task scheduler with curl called in a batch file. ...
1
vote
0answers
51 views

FTP: wildcard downloading

I am trying to download a file with name ending with *.csv The problem is complicated by the fact that this *.csv contains in directory of unknown name: eg. Sometimes it would be 2015->Date ...
1
vote
2answers
421 views

How can I use curl with ::1 for ipv6 based loopback?

I tried a few ways but they seem to be confusing curl. root@testt:~# curl localhost:8080 <h1>Hello world!</h1> <p>Boom Bam Splat</p> root@testt:~# curl ::1:8080 curl: (3) ...
0
votes
0answers
195 views

How to use .pfx in certificate store in CURL for Windows 7

I want to use cURL to access a remote server, and I need to identify myself using a private key or the remote server gives a 401 "Unauthorized". I'm on Windows 7 (work machine, not my choice) and my ...
1
vote
1answer
52 views

Bulk SFTP download of files specified by URL preserving directories

I have a list of several hundred files I need to pull via SFTP. They are located in various directories and I need them to be in the same directory setup on my local machine. It is important the files ...
0
votes
1answer
41 views

How accurate is CURL's 'current rate' in the progress meter?

Is the second-by-second output in curl accurate "enough"? What is the period that the "current rate" is calculated over? I am currently using the second-by-second status output in Curl as a general ...
0
votes
1answer
35 views

How can you lock computer while using curl to download files?

I’m using a Python script in the Terminal in Mac OS X 10.10.2 to download many files from a webpage using curl. However sometimes I might want to lock my computer and walk away but this terminates my ...
0
votes
1answer
62 views

Escape semicolon curl with -u option

I can't wrap my head around how to escape semicolon in username while making a GET request. curl -u testclient:sdtest1:pass -XGET -H "Content-type:application/json" \ http://localhost:8081/hello ...
1
vote
0answers
162 views

I destroyed my Mac's PHP setup: SSL certification fails

Long story short, I deleted the PHP symlink on my Mac, and replaced it with a symlink from XAMPP. Now it seems that there's a problem with my SSL configuration or what seems to me, a lack thereof. ...
2
votes
0answers
71 views

How to curl or wget live tennis scores from a Google search page [closed]

Google has some nice live tennis scores displayed on the search page. If I search for "Halep", for example, I get the tournament she's currently playing, together with all the result and live score. ...
0
votes
0answers
13 views

Curl should be quit but it returns ever something

I tried to login to a website using curl in a bash-script. It is possible to login to the site quiently using this: curl -d "username=$username&passwd=$passwd" -c $cookiefile ...
0
votes
0answers
20 views

How to check what kind of files web page consists

Is there a way to check what files web page consists. For example I want to download .txt files from the web page, but I dont want all .txt files, some of them, and before I want to check are there ...
0
votes
1answer
56 views

Is there a way to download all .txt files from the page via curl or wget command

My question is in a title: Is there a way to download all .txt files from the page via curl or wget command? Thanks
1
vote
1answer
224 views

Exit bash script when curl gets a non 200 HTTP status

I have a bash script setup to perform a few curl requests for currency in EUR INR JPY do curl -i --data '{"currency": "'$currency'"}' -H "Accept: application/json" -H "Content-Type: ...
0
votes
0answers
43 views

Need help constructing cURL config file

I have a (rather long) cURL command that I'm currently running as follows: curl.exe -U : --proxy-ntlm --proxy myproxyserver --cookie-jar cookies.txt -H "Accept-Encoding: gzip,deflate,sdch" -H ...
0
votes
0answers
59 views

Downloading file from (your download will start shortly) page

How can i download a file with curl (or any other alternative) that make user wait until download start for example enter link description here i tried curl -O -J -L ...
2
votes
1answer
396 views

dyld: Library not loaded: /usr/lib/libcurl.4.dylib curl SSL error

I'm not sure what happened to my system but I am having an issue with dyld/curl, affecting my overall computer performance. I am on Yosemite. I have a feeling it's the Conda openssl. How do I just ...
0
votes
0answers
189 views

Why is cURL slower than wget

Test page: https://www.beobank.be/nl/Home.aspx This is the result using wget (or a real browser): # time wget https://www.beobank.be/nl/Home.aspx -O /dev/null --2015-01-26 12:05:46-- ...
0
votes
0answers
143 views

nginx not serving php files with curl in fedora 21

phpinfo and php with echo are working fine but with curl gives 403 error. error in chrome: http://localhost/send3.php Failed to load resource: the server responded with a status of 403 (Forbidden) ...
0
votes
0answers
35 views

Prevent WAMP from loading external requests

I need to prevent wamp from loading external links in order to have a better performance I'm using PrestaShop and sometimes it looks for updates and currency rates etc which slows down loading the ...
0
votes
1answer
92 views

cURL: HTTP file download

I have tried to download a file using curl "https://www.dropbox.com/s/tf7e7vndqd9hv05/Ace_The_IELTS.pdf?dl=0" -o "file.pdf" but it downloads a file which is of few bytes. When I open the link in the ...
0
votes
1answer
60 views

how to download pictures with wget

i would like to download pictures from http://myhomepage.com/plots/20150111-03/4711/4712.png every 3 hours. "20150111-03" is variable, rest is stable. That means every 3 hours the download looks like: ...
2
votes
1answer
255 views

Error executing sftp with curl

I'm using IBM's Data Explorer to import certain data, but am getting this error: Protocol sftp not supported or disabled in libcurl. So from Red Hat I manually ran the command: curl -user ...