curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

0
votes
0answers
9 views

wget keeps consuming space on C drive when run from another drive

I have wget download a 200gb file with the -c setting, run from cmd. Yet it keeps making a ghost copy to C drive, a copy I cannot even locate. Have to abort each time it completely eats up my SSD or ...
1
vote
1answer
17 views

curl shows a greater-than sign instead of uploading a given file to a given server

I want to upload a file to a WebDAV server within Linux command line. The standard recommendation for this is to use curl in the following way: curl -u mail_address:password -T filename.ext https://...
0
votes
2answers
27 views

Why does curl and wget take so long to resolve locally mapped DNS on OSX?

I work with a team to develop a web service and client, each of us, of course, working on our own machines. To keep things simple and more consistent, we use similar domains in the code and update /...
0
votes
1answer
28 views

what is the meaning of “body” in a POST request?

I am new to PHP. I have a stupid question and need your explanation. When I make a POST request with the cURL command-line: curl -d "username=admin&password=admin&submit=Login" --dump-header ...
0
votes
1answer
32 views

Compressor detection issue when running curl command from bash script in Docker container

I want to execute curl using bash script inside the Docker container. This is my script file script.sh that I added to the container: curl -X POST http://localhost:9200/tests/test -d usr/share/...
0
votes
0answers
9 views

use curl to download media files

I would like to download a music from a HTTPS website. Now they don't offer a specific download button, so I imagined using curl in Terminal. Sth like that: curl -O https://urlhere This normally ...
0
votes
1answer
10 views

is calling remote bash script via https which then call another bash script with http considers secure?

It might be a silly question but I will go for it anyway… due to some backward compatibility I only adjusted part of the calling script chain. there is a reason why I call it twice but let's leave it ...
0
votes
0answers
33 views

Schedule on crontab to reboot router

I would like your much aprreciated help. I have a ISP router (gateway) that doesen't have a schedule reboot on the web interface. I have a raspbian and I would like to schedule on crontab to reboot ...
0
votes
0answers
21 views

jq filtering curl- 'Cannot index number with string'

I am trying to filter a curl request with jq to get specific fields and put them in variables. Initially, I was using a basic curl request such as this: myResponse=$(curl -u myUsername:myPassword -...
0
votes
1answer
19 views

Check MD5 of a file header hosted on a webserver

I am writing a bash script to periodically download a setup file using cURL. The problem is the following, the file is around 110 MB and always have the same filename. I can download the file, check ...
0
votes
1answer
15 views

command option confusion in curl on Linux

Wondering what is the function of @ sign? Here is an example command, curl -w "@curl-format.txt" -o /dev/null -s "http://wordpress.com/" Here are more related backgrounds, http://stackoverflow.com/...
1
vote
1answer
46 views

Could curl print latency in terms of milliseconds?

Wondering if there is an option for curl to print latency in milliseconds? An option that works on either Linux or Mac OSX would be good.
2
votes
1answer
27 views

openssl s_client returns DNS error

When trying to connect to a site using openssl, I get the following error. $> openssl s_client -connect www.citibank.com:443 getaddrinfo: No address associated with hostname connect:errno=2 But ...
0
votes
0answers
14 views

Use curl to download an LDAP attribute value only

I am using the following curl command to download a value from an LDAP directory: curl ldap://example.com/CN=ACME-Test-CRL,O=ACME?certificateRevocationList;binary -o crl.crl The problem is it ...
1
vote
0answers
29 views

can't connect to https API using cURL?

I have API endpoint URL, certificate file ( cert.p12 ) & password. Here is the steps I try to connect to this API. ( IN Windows 7 ) First I type the URL in chrome browser and it didn't give me ...
1
vote
1answer
23 views

how to use curl or wget to download a text file off a website?

i am trying to access eoddata.com to download a list of stock prices at the end of everyday without having to do it myself manually. the full url i am trying to access would be something like: http://...
1
vote
0answers
36 views

Centos cURL connection refused on Its own IP

I have two services on one server (with different ports to access them). One of my services is trying to connect to the other one with the IP of server and its specific port. But I got some errors ...
0
votes
0answers
109 views

curl: (56) Recv failure: Connection reset by peer

I am trying to download one file from my local server using curl.i am using command curl -L -O --retry 999 --retry-max-time 0 -C - http://192.168.1.74/gio-v2-9515.dd -u ajay:aj123 For experiment ...
1
vote
0answers
20 views

how to upload files to https server with certificates using curl or any other commands

I am Trying to upload to my HTTPS server using curl .I am using curl for that ,the code is curl -v --cacert /tmp/WIN-CLS.vdi.com.pem -T /etc/hosts \ --user administrator:pass_123 https://xxx.xxx....
1
vote
0answers
123 views

Can someone explain the differences of file_get_contents vs guzzle vs curl?

I've been getting data from api websites by using file_get_contents or curl, but I never knew whats the difference from them. They seem to have the same results, but now there is Guzzle for php. What ...
0
votes
2answers
49 views

How can I archive someone's Facebook profile page?

I thought about using archive.is or archive.org. I tried to view the page in a private browser, but it asks to log in, so that site will see the same thing. I tried to open the developer console and ...
2
votes
1answer
219 views

GIT from source: fatal: Unable to find remote helper for 'https'

Due to me not having root rights, I compiled git from source locally. My distro is OpenSUSE 13.1. Googling the error fatal: Unable to find remote helper for 'https' It was suggested that curl is not ...
0
votes
1answer
44 views

Sending gist to github via cURL and issues with new lines inside file

So, let's say I have an index.php file containing this: <?= "Hello" ?> <?= echo "WORLD" ?> And I wanted upload the contents of this file to my gists in github, which I am doing via ...
1
vote
1answer
82 views

Can't update meteor, because it can't download manifest

I've meteor installed via npm and I'd like to update it, but I can't. I've tried: $ meteor update Failed to download manifest. but it fails with: Failed to download manifest. So then I've ...
0
votes
0answers
28 views

curl lookup time is 5.5 seconds when accessing address in /etc/hosts

In my /etc/hosts file I have a custom entry since the name doesn't actually exist like this: 5.5.5.5 someinternalserver.thing.net #thats a fake IP for this post Using curl like so: curl -o /dev/...
0
votes
1answer
92 views

How can you “press” a button of a webpage from the command line to restart a cable modem?

I'm trying to reboot my cable modem programmatically. I used to be able to do this by using wget: wget http://192.168.100.1/reset.htm?reset_modem=Restart+Cable+Modem However I changed my cable ...
1
vote
1answer
68 views

Curl command to download a file over HTTPS

I need to download a particular file from https://file_path. Here is the curl command I use for that: curl --user user:password "https://file_path?raw" > location I wish to download the real ...
0
votes
1answer
25 views

cURL command units of Dload speed?

When I run curl I sometimes see a progress meter like this: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total ...
0
votes
1answer
35 views

Psexec with curl or wget

I recently found out about psexec and was wondering if this program could help me run wget or curl commands on a remote computer. is this even possible?
0
votes
0answers
101 views

Bashscript using curl to crawl a website that needs a login

I am looking to crawl a website that has kind of a newsfeed with elements hat have a timestamp. There's only a certain amount of elements in the feed and a "Previous" button at the end of the feed. It ...
0
votes
3answers
142 views

Sending a POST request via curl has issues with double quotes

So, in-order to send a POST request to gists (github), you can do something like this as seen in https://gist.github.com/caspyi... curl --user "user" -X POST --data '{"description":"Created via API","...
0
votes
1answer
269 views

curl command not found on OSX El Cap

My understanding is that curl should be installed. I had tried using curl from the Unix command line and got the message it could not be found. I'm wondering If something I did in efforts to clean up ...
1
vote
2answers
257 views

Curl response hangs

I have a site set up in a virtual box. I am able to access it just fine from a browser on my machine, but I have trouble accessing it via CURL while SSH'ed in to the box. When I try, curl hangs ...
0
votes
0answers
15 views

mirror site behind login

I would simply like to mirror all the pages (millions) on a site behind a login. This is for personal use. I have a login for the site in question but I want to scrape it into a database. I ...
-1
votes
1answer
35 views

How to use CURL tool?

I just got introduced to CURL today and I went to their website to download the tool and all I got is .cgi file called "metalink" and I don't know how to use it or how to run it, can someone help ...
0
votes
1answer
30 views

How to use VERB in url

I am trying to create and API which uses VERB PUT,GET,DELETE I can use it in curl by this curl -X DELETE http://localhost:8080/recipes/hi But how can i use similarly in url for eg This works ...
0
votes
1answer
70 views

Dictionary attack in Bash with cURL

I'm fairly new to Bash, but this is a script I've been working on for the past few days. It's meant to run a dictionary attack on an HTTP-POST login for a website, by reading the time module and ...
1
vote
0answers
21 views

“curl” ignores proxy environment variable when using an ip address?

I am using curl to access some web services through a proxy (an instance of polipo running on port 8123, chained to an ssh SOCKS5 proxy). If I provide proxy information using the http_proxy ...
0
votes
1answer
15 views

Neither wget nor curl downloads this wikipedia api request

I can perfectly download this link and save it as a text file in a web browser environment. But I can't download and save that wikipedia api request using neither wget nor curl. I use wget -O file.txt ...
0
votes
2answers
385 views

How to POST file contents using cURL?

I want to do an HTTP POST of the contents (as a string) of a local file located at path/to/my-file.txt to a URL endpoint at http://example.com/. For example, I might want to do the following: ...
0
votes
1answer
22 views

libcurl version 7.21 to work on Redhat 6

I am having an issue with libcurl version 7.19 and socks proxy on redhat 6. I believe it was fixed in libcurl version 7.21 how do i upgrade libcurl on redhat 6 to 7.21 or greater?
4
votes
1answer
370 views

Automatic login to Vodafone Community WiFi (Fon) via Unix script

First I have to say that my question is similar to this one I am using a Vodafone Community Fon AP and it often disconnects. In OS X, which I'm using, there is a tool called Captive Network Assistant ...
2
votes
0answers
140 views

Retrieving from ftp dirs with wildcards

What is the most efficient way to retrieve all relevant data from ftp if the address of said data can be specified with wildcards as below: ftp://ftp.some.site.gov/data/directory/only/*/these/*/*...
2
votes
0answers
543 views

SSL error when connecting to git from Mac OS X 10.11.3 (El Capitan)

When I try to clone / pull / fetch from a git repository via https from a Mac System I keep getting an error: error: RPC failed; result=56, HTTP code = 200 fatal: The remote end hung up unexpectedly ...
3
votes
2answers
213 views

curl check if file is newer and instead of downloading - execute a bash (or python) script

I am having a bit of an issue. I have a file, hosted on a remote server (http://mywebsite/file.zip). I also have a few embedded linux boxes (running openelec OS). The boxes have fairly limited ...
1
vote
1answer
85 views

Fetch source file from GitHub using command line tools?

This question is related to a question offsite at FATAL: MemorySanitizer can not mmap the shadow memory. The person who responded said: Do simple tests (e.g. this one) work for you? The link ...
1
vote
0answers
220 views

curl suddenly unable to get local issuer certificate

I've been using Docker 1.7.1 with base Ubuntu 14.04 image and all of a sudden this curl: curl -L -o sbt.tgz https://dl.bintray.com/sbt/native-packages/sbt/0.13.9/sbt-0.13.9.tgz is failing me with ...
-1
votes
2answers
25 views

Where is my file saved locally?

I've downloaded a file--I see the contents of my text file in the terminal window as it is downloaded in curl, but where is it saved on my local machine: curl -v -O myFile.txt -u username:password "...
0
votes
0answers
34 views

Finding a text string on a website and all its links

Can you help me understand process of finding a certain text string on a website and all its immediate links? I started delving into wget and curl with help of grep but was unable to come up with ...
0
votes
2answers
156 views

cURL connecting to localhost (127.0.0.1) instead of destination IP

On Mac OS X I have an issue where calling: curl -v www.google.com results in: * Rebuilt URL to: www.google.com/ * Trying 62.253.72.153... * Connected to www.google.com (127.0.0.1) port 80 (#0) &...