curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

0
votes
0answers
64 views

Slow download over libraries on certain Windows systems

I am using custom libraries for file download (cURL and wget) on Windows client over HTTP from my server. On most of systems the download speed is as it should be - maximum determined by network ...
1
vote
0answers
14 views

Curl in bash code - produces wrong status

This code seems to ouput a http status of 000 - which seems to indicate something didn't connect properly but when I do this curl outside of the bash script it works fine and produces a 200 so ...
0
votes
0answers
70 views

curl to test secure web proxy

I have used Chrome to successfully connect to the secure web proxy(squid https_port) Chrome doc , but I want to know any command line tool(such as curl) will connect to it too. Curl's -x option ...
0
votes
0answers
80 views

Gmail Automation via Bash (To refresh POP Fetch from other accounts)

Gmail has this option to fetch mails from other accounts. I used this feature to create a central mail box. Works good. Gmail periodically polls other mail box over POP. But the frequency is 1 hour + ...
-2
votes
1answer
119 views

Filter curl header output in verbose mode

Using curl -v ..., i.e. the verbose mode that prints out the input and output headers. However, this info is not piped and I can not grep-out lines which I dont need. How can you filter curl header ...
0
votes
0answers
79 views

WGET not working but CURL works on the same address/host?

WGET give me this error: HTTP request sent, awaiting response... 403 Forbidden 2015-11-23 12:07:06 ERROR 403: Forbidden. If I try the same URL (it’s an internal network Apache server, firewall ...
0
votes
2answers
431 views

Using cURL to post data

I try to post to localhost with some parameters in the URL. Everything just works fine in Fiddler. But cURL doesn't allow me to do it, with the error message below. curl --data "Task=bake" ...
0
votes
0answers
29 views

windows curl --compressed : unrecognized command

I found the SAME problem: (si I know it can be duplicated) Curl Compressed Command not Recognized in Windows However no one of these solutions works for me (win7 / 64bits) When I use without the ...
0
votes
1answer
69 views

cURL in shell vs. knockd

I want to use curl to control features of a IPcam. This works fine in a shell (/bin/sh) but fails when called from knockd. I am trying this on a QNAP NAS (Busybox Linux). The script I am using is: ...
0
votes
0answers
70 views

How to interact with submit input type with no name in php using curl

<select name="_limit"> <option value='50' >50</option> <option value='200' >200</option> <option value='300' >300</option> <option value='500' ...
0
votes
0answers
78 views

Curl Could not resolve host

I'm trying to get a webpage after submitting a username and password but i'm receiving an error. This is on Ubuntu server The command i'm using is curl -g -d ...
0
votes
0answers
35 views

How can I approximate impact of using proxies on cURL request time using CURLINFO_*_TIME?

Below is a small dataset from which I'm trying to answer two questions: How much time did the proxy take to connect to the API server? How much time did the API request take to return? The basic ...
0
votes
0answers
180 views

Make a curl POST request from scraped data

I scraped a website using the simple DOM php parser : <?php include 'domparse.php'; $html = file_get_html('http://oceanofgames.com/rebel-galaxy-free-download/'); ...
0
votes
1answer
196 views

CURL in bash script works with . (dot space), not ./ (dot slash)

I have a bash script which sends a couple of notifications to Slack via CURL and spawns an expect script. When I run the script via . scriptName.sh it works fine, but when I run it by typing ...
0
votes
0answers
48 views

Redirection not working for the first time

There are output of curl command. Sometimes my site doesn’t redirect properly, showing error page: no data received from the server. Is there any way to remove this error? ...
1
vote
2answers
79 views

I'm trying to open query.yahooapis.com but curl is trying to use IPV6 even if I only have IPV4 enabled

When I try to call query.yahooapis.com with curl it's trying to use the ipv6 address but IPV6 is disabled on my computer $ curl -v 'http://query.yahooapis.com/' * Hostname was NOT found in DNS cache ...
0
votes
1answer
271 views

Installing and using cURL in Linux and use it in Eclipse as well?

I come from a Windows environment - I used cURL in Eclipse (C). I want to do the same in a Linux environment (Ubuntu). I followed the installation guide on how to install cURL, but how do you use cURL ...
0
votes
0answers
92 views

How to receive sse (server sent events) and execute bash commands then

How do i receive server sent events (sse) from a webserver and execute a bash command with the received data in debian (raspbian)? Is that possible with curl as i can already receive sse. Thanks ...
0
votes
0answers
61 views

Amazon S3 upload error: An exception occurred while uploading parts to a multipart upload

I am trying to upload a 30 GB file to Amazon S3 using the AWS PHP SDK. require('../vendor/autoload.php'); $client = new Aws\S3\S3Client([ 'version' => 'latest', 'region' => 'us-east-1' ]); ...
0
votes
0answers
243 views

Sending POST with esp8266 Module

I am trying to send a POST request from my esp8266 wifi module and I am getting errors. Such as: "Your browser sent a request that this server could not understand" I am trying to send: POST ...
0
votes
1answer
119 views

Delivering a payload to a URL, using cURL

I need help figuring out the cURL request required to send the following data to a URL. {"requestid":"555555", "partnermatchid":"10000-000-0000-0000-000", ...
0
votes
0answers
173 views

wget or cURL for dynamic recursive URLs (generated through directories and adding specific same text at the end)

I'm using a wget command with some options in order to refresh the cache module from all URLs of a Prestashop. This is the command: wget -r -D www.domain.com -l3 -P temp -x -R "*cart*","*order*","*My ...
1
vote
1answer
71 views

How can I run the equivalent “curl” command that performs similarly to this “wget” command?

I have a wget command: wget -r --no-parent -nH -nd -q --show-progress --progress=bar:force ftp://ftp.ncdc.noaa.gov/pub/data/swdi/database-csv/v2/ But it only works in newer versions of wget. ...
1
vote
2answers
77 views

Download a list of files and use domain name as filename

I got a bunch of files to download which works great with wget -i list_of_urls but my problem is, that wget uses the filename of the downloaded file. Is there a way (or a different tool) which is ...
0
votes
1answer
58 views

How to extract ip address from no-ip account webpage in the shell?

I've set dynamic dns for my home network(which is behind NAT) using www.noip.com. I use ddclient to send updates of my ip, but sometimes it takes a while to update the ip. Anyways it's not the point. ...
4
votes
2answers
255 views

Get curl to output non-2xx to stderr, but 2xxs to a file

I would like some curl command, possibly with some bash hackery, that: Outputs any 2xx responses to a file Outputs any non-2xx responses to stderr Exits with a nonzero status code when a non-2xx ...
1
vote
1answer
73 views

How to run a remote bat file on a local computer

I need to run a batch file, that is stored on a server, locally on a windows 8.1 machine. I can run a shell script in OS X with a curl: cURL "file path" | sh is it possible to cURL in powershell in ...
0
votes
1answer
485 views

how to make curl re-use existing connection

curl --verbose google.com If I execute this command, after curl does its job, it says "Connection #0 to host google.com left intact". But it doesn't use same connection if I execute same command ...
1
vote
0answers
61 views

How do you download a zipped build from a jenkins job using “jenkinsapi” module in Python?

Has anyone tried download a build using jenkinsapi module in Python?
0
votes
1answer
143 views

What is the curl equivalent of wget -qO (wget -q -O)?

I need to run the following command in Red Hat Linux: wget -q -O http://somepage/somefile.cgi| bash But I don't have wget installed. I could install it, but I want to know how to do this with curl. ...
4
votes
1answer
876 views

What units does curl use for bandwidth?

curl on the command-line displays progress like this: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent ...
0
votes
1answer
189 views

How to upload all the files from local to host using Curl?

i have written a shell script to upload all files with different names in the specific path from local to host, but i have facing some issue and the files are not getting uploaded #!/bin/sh for file ...
0
votes
0answers
556 views

Curl: How to send custom headers while using proxy

I want to send custom headers to the host while using proxies. Take a look at the examples below. Explaining the situation with examples was easier. This works fine as expected: curl ...
2
votes
2answers
130 views

Fast way to download many small webpages?

I need to download lots of small pages as fast as possible. Average size for one page is 3kb. There are millions of pages. First, I tried curl but it was way too slow, probably because it was waiting ...
1
vote
1answer
964 views

What is the correct way to submit content-type header in curl?

I’m using Mac 10.9.5 with bash shell. I’m trying to submit a request via curl, but I keep getting errors when I try and set the content-type header. Below I try davea$ curl -v -o -H "Content-Type: ...
0
votes
1answer
154 views

How to download the latest version of youtube-dl programmatically?

The latest version of youtube-dl is at: https://yt-dl.org/downloads/latest/ which redirects to (as of today): https://yt-dl.org/downloads/2015.07.04/ I'd like to set up a cron job to download the ...
0
votes
2answers
195 views

Curl post form and get result

I found this website http://www.fonefinder.net/findome.php , that locates phone provider by number (ex: http://www.fonefinder.net/findome.php?npa=562&nxx=866&thoublock=1666) shows VERIZON ...
2
votes
2answers
220 views

How can I instruct curl to reuse credentials after it followed a redirect?

I know that curl can follow redirects (the -L argument), and I know that I can provide credentials to a request (-u username:password), but I want to use both all the way down the redirect chain. How ...
5
votes
2answers
252 views

Firebug and cURL

I have been using Firebug and cURL for quite a while. Firebug is amazing to capture HTTPS POST request as opposed to a network analyser as it is aware of SSL/TLS negotiation. And cURL is amazing to ...
0
votes
0answers
303 views

curl could not open remote file permission denied using sftp

I'm trying to download a file from a new enduser sftp using existing code that works for other end user sftps, using curl. This is my basic command: curl -o [internalFileLocation] -u [user:pw] ...
0
votes
1answer
399 views

Using a curl scheduled task in windows

I am using the following command in a scheduled task in windows to update my IP to the free Dynamic DNS service called EntryDNS: curl.exe -k -X PUT -d "" "Their url API string here" The problem I am ...
1
vote
0answers
61 views

Run bash command from link in browser

Is there a way to run an arbitrary command on OSX by clicking a link in the browser? As you know, there are certain types of hyperlinks that launch certain applications with a specified action, for ...
2
votes
1answer
516 views

Site displays with Curl but not within the browser

I've set up a dev-enviroment on Debian 7 and installed the following services: Apache Bind9 I have one site that is reachable through "asix.debian.local". I've set up the zone and added NS and A ...
0
votes
1answer
108 views

curl ignore --data starting with @ sign: don't read from file

In slack you can script slackbot to post messages to a channel like this: curl --data "$msg" $'https://<yourteam>.slack.com/services/hooks/slackbot?token=<yourtoken>&channel=#random' ...
3
votes
1answer
686 views

Why does “curl” command work normally while “ping” not?

I find a strange phenomenon in my corporate network: I can execute curl google.com command successfully: [root@localhost ~]# curl google.com <HTML><HEAD><meta ...
0
votes
0answers
204 views

Why do wget and curl fail to download page source?

Via a shell script, I would like to download the source for a web page with air quality data on aqicn.org. Here is the URL for the page with Beijing air quality data: http://aqicn.org/city/beijing/m/ ...
22
votes
3answers
12k views

Is “curl -u username:password http://example.com” secure?

Is curl -u username:password http://example.com secure? If not, can you give a brief explanation of how someone could obtain your password?
0
votes
3answers
91 views

Why can't the guest of VirtualBox connect to network?

I have installed CentOS 7 on virtualbox, and network setting is NAT. The guest can't connect to network: systemctl -l status network output likes this: ip addr output likes this: Could anyone ...
0
votes
0answers
141 views

Pipe remote jpg webcam image to ffmpeg as video

I have a cheap ip webcam. All it provides is a single http url to a static jpg file. Such as http:///image.jpg. Repeatedly requesting the same url will provide subsequent frames from the ip camera. ...
1
vote
1answer
232 views

curl capturing http status and timing the request

I have a loop in wich I check urls from file one-by-one and want to get the status header and the execution time of every request as result. I have two curl commands: This one outputs the header ...