curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

1
vote
1answer
135 views

Downloading an authenticated large file with command line tools

I need to download a file sized around 1.6GB. I's like to get it download via a command line tool like axel, wget or curl However, how can I download a file which needs to be logged in to the site ...
0
votes
2answers
433 views

What tool does yum use to download rpm packages from remote server?

My system is centos, and now I am installing packages using yum command. Updating for dependencies: apr-util i386 1.3.9-1.el5 lxcenter-updates 203 k gcc-gfortran ...
1
vote
0answers
33 views

Access Websites & Modify Web Contents [closed]

so I'm given a website, and it requires me to use a token to get in. Is there any way I can do this on a normal web browser? I've been using curl -X GET in the Linux terminal. Also, how can I modify ...
0
votes
2answers
1k views

Downloading and extracting a .tar to a specific directory

I want to download a tar of a git repo and extract its contents to another folder. The following line (broken up for clarity) works: curl -L -0 nt-dotfiles.tar.gz ...
2
votes
0answers
209 views

Why doesn’t curl install and run on my 64-bit Windows 8 system but the 32-bit version works fine?

I almost gave up installing cURL on my 64-bit Windows 8 system. I tried almost every single .zip, .exe, etc… on this site, but none of them worked. I got this message every single time: The ...
0
votes
1answer
446 views

understanding sending post request (with curl)

alright, I know related questions have been answered : What is the cURL command-line syntax to do a POST request? http://stackoverflow.com/questions/14978411/http-post-and-get-using-curl-in-linux ...
0
votes
1answer
58 views

Send HTTP request with parameters file

My question is very simple : is it possible to send a HTTP request from the *nix command line specifying a file, containing all the parameters, as an argument? In REST APIs documentation, we often ...
26
votes
2answers
97k views

How to send POST with body, headers, and HTTP params using cURL?

I found a lot of examples on how to use simple POST commands in cURL, but I didn't find examples on how to send full HTTP POST commands, which contain: Headers (Basic Authentication) HTTP Params ...
0
votes
2answers
5k views

How to download a file behind authentication via Wget or cURL [duplicate]

Possible Duplicate: Wget a page that requires logging in I wish to download file from http://mbr.nlm.nih.gov/Download/MetaMapped_Medline/2011/. However, in order to download the file, I ...
0
votes
2answers
252 views

Log date and external ip address with curl and batch (Windows)

I am having weird issues where my external IP is changed VERY often as in every 2, 3 minutes. I have a dynamic dns service in place that I update via task scheduler with curl called in a batch file. ...
1
vote
0answers
126 views

FTP: wildcard downloading

I am trying to download a file with name ending with *.csv The problem is complicated by the fact that this *.csv contains in directory of unknown name: eg. Sometimes it would be 2015->Date ...
1
vote
2answers
628 views

Upload image to postimage.org using curl

I found a script that can upload image to postimage.org using curl. It was posted not long ago, on 2013-09-06, at http://crunchbang.org/forums/viewtopic.php?id=28867. The simplified script looks like ...
0
votes
1answer
114 views

How accurate is CURL's 'current rate' in the progress meter?

Is the second-by-second output in curl accurate "enough"? What is the period that the "current rate" is calculated over? I am currently using the second-by-second status output in Curl as a general ...
7
votes
1answer
621 views

How can I use a command line tool like Wget to log into an OpenID site?

Specifically, I would like to be able to download certain pages from my user profile on the various Stack Exchange sites. I would, however, like to do this automatically (using a cron job), from ...
4
votes
2answers
4k views

How can I use curl with ::1 for ipv6 based loopback?

I tried a few ways but they seem to be confusing curl. root@testt:~# curl localhost:8080 <h1>Hello world!</h1> <p>Boom Bam Splat</p> root@testt:~# curl ::1:8080 curl: (3) ...
0
votes
0answers
15 views

Curl should be quit but it returns ever something

I tried to login to a website using curl in a bash-script. It is possible to login to the site quiently using this: curl -d "username=$username&passwd=$passwd" -c $cookiefile ...
1
vote
0answers
438 views

How to use .pfx in certificate store in CURL for Windows 7

I want to use cURL to access a remote server, and I need to identify myself using a private key or the remote server gives a 401 "Unauthorized". I'm on Windows 7 (work machine, not my choice) and my ...
0
votes
1answer
93 views

How can you lock computer while using curl to download files?

I’m using a Python script in the Terminal in Mac OS X 10.10.2 to download many files from a webpage using curl. However sometimes I might want to lock my computer and walk away but this terminates my ...
2
votes
1answer
5k views

How to split the HTTP error code from the contents in cURL?

Yes, this is related to Getting curl to output HTTP status code? but unfortunately not the same. In a script I would like to run: curl -qSfsw %{http_code} URL where the -f option ensures that the ...
1
vote
0answers
233 views

I destroyed my Mac's PHP setup: SSL certification fails

Long story short, I deleted the PHP symlink on my Mac, and replaced it with a symlink from XAMPP. Now it seems that there's a problem with my SSL configuration or what seems to me, a lack thereof. ...
2
votes
0answers
113 views

How to curl or wget live tennis scores from a Google search page [closed]

Google has some nice live tennis scores displayed on the search page. If I search for "Halep", for example, I get the tournament she's currently playing, together with all the result and live score. ...
0
votes
0answers
21 views

How to check what kind of files web page consists

Is there a way to check what files web page consists. For example I want to download .txt files from the web page, but I dont want all .txt files, some of them, and before I want to check are there ...
0
votes
1answer
150 views

Is there a way to download all .txt files from the page via curl or wget command

My question is in a title: Is there a way to download all .txt files from the page via curl or wget command? Thanks
1
vote
1answer
667 views

Exit bash script when curl gets a non 200 HTTP status

I have a bash script setup to perform a few curl requests for currency in EUR INR JPY do curl -i --data '{"currency": "'$currency'"}' -H "Accept: application/json" -H "Content-Type: ...
0
votes
0answers
78 views

Need help constructing cURL config file

I have a (rather long) cURL command that I'm currently running as follows: curl.exe -U : --proxy-ntlm --proxy myproxyserver --cookie-jar cookies.txt -H "Accept-Encoding: gzip,deflate,sdch" -H ...
0
votes
0answers
145 views

Downloading file from (your download will start shortly) page

How can i download a file with curl (or any other alternative) that make user wait until download start for example enter link description here i tried curl -O -J -L ...
2
votes
1answer
669 views

dyld: Library not loaded: /usr/lib/libcurl.4.dylib curl SSL error

I'm not sure what happened to my system but I am having an issue with dyld/curl, affecting my overall computer performance. I am on Yosemite. I have a feeling it's the Conda openssl. How do I just ...
0
votes
0answers
555 views

Why is cURL slower than wget

Test page: https://www.beobank.be/nl/Home.aspx This is the result using wget (or a real browser): # time wget https://www.beobank.be/nl/Home.aspx -O /dev/null --2015-01-26 12:05:46-- ...
27
votes
3answers
12k views

Curl local host names on Mac OS X Yosemite

I just upgraded from Mavericks to Yosemite, and now curl can't see loopback host names. Set up a simple http server to test: $ python -m SimpleHTTPServer Serving HTTP on 0.0.0.0 port 8000 ... Now ...
0
votes
0answers
65 views

Prevent WAMP from loading external requests

I need to prevent wamp from loading external links in order to have a better performance I'm using PrestaShop and sometimes it looks for updates and currency rates etc which slows down loading the ...
0
votes
1answer
141 views

cURL: HTTP file download

I have tried to download a file using curl "https://www.dropbox.com/s/tf7e7vndqd9hv05/Ace_The_IELTS.pdf?dl=0" -o "file.pdf" but it downloads a file which is of few bytes. When I open the link in the ...
0
votes
1answer
66 views

how to download pictures with wget

i would like to download pictures from http://myhomepage.com/plots/20150111-03/4711/4712.png every 3 hours. "20150111-03" is variable, rest is stable. That means every 3 hours the download looks like: ...
2
votes
1answer
1k views

Error executing sftp with curl

I'm using IBM's Data Explorer to import certain data, but am getting this error: Protocol sftp not supported or disabled in libcurl. So from Red Hat I manually ran the command: curl -user ...
0
votes
0answers
241 views

Cannot curl webserver from outside network

I have a webserver running on one of my home computers. I am able to access it within my network, on another computer, with curl localWebserverIP:8888. However, I am having issues accessing it from ...
0
votes
1answer
184 views

RPM created using checkinstall giving dependency error

Compiled openssl-1.0.1.i using commands: export CFLAGS="-fPIC" ./config shared --openssldir=/usr/share/ssl make depend make all make install here it is not asking for any dependencies I am able to ...
1
vote
3answers
170 views

Bash: executing concatenated variables

I have a Linux bash script, to the effect of: COMMAND_TO_EXECUTE="foo "$1" --option1 --option2 "$2 exec $COMMAND_TO_EXECUTE Problem: the script fails Debugging clue, If I echo ...
1
vote
2answers
2k views

libcurl or cURL not working OSX Mavericks

Before I start let me say I have uninstalled, reinstalled, compiled from source, used macports and generally tried for the last 6 hours to get curl working on my Mac OSX Mavericks but I still haven't ...
0
votes
1answer
51 views

Download file on unix from target url which autodownloads file on page visit

I'm trying to download a file to a unix machine on the command line. Accessing a GUI on this particular machine would be quite difficult. The file is autodownloaded when visiting the url in a web ...
1
vote
1answer
3k views

When using curl to http post a xml file to a url, how to set the parameter?

I want to use curl to http post a file to a url like: curl -d myfile.xml http://www.example.dom/post/file/here Can I set the parameter I want the file to be posted with? Like if it was a ...
5
votes
1answer
12k views

CURL File with post parameter

How to make a request for uploading a file to facebook using graph api in curl I am making this request but getting Error: curl https://graph.facebook.com/<id>/photos -F "source=@me.jpg" -d ...
46
votes
2answers
50k views

How do you display POST data with cURL?

As an example, POSTing to a web server with the -v argument: curl -v http://testserver.com/post -d "firstname=john&lastname=doe" And the output > POST /post HTTP/1.1 > User-Agent: ...
1
vote
2answers
2k views

curl can't connect; wget can

trying to connect to my script on a remote server with curl but: col-pos-spd# curl -v http://***:80/monitor/reply.php * About to connect() to *** port 80 (#0) * Trying 83.xxx.xxx.2... Connection ...
0
votes
1answer
356 views

Where does a file downloaded with CURL go on the file system?

Ive downloaded a file from this site with the following curl command: https://www.modern.ie/en-us/virtualization-tools#downloads # Copy/paste the command below into your terminal to begin ...
0
votes
1answer
380 views

Windows CURL download file- special char in URL

I need to download .xls file with Windows cURL and I have got a problem with URL of that file, because it contains special char "Č". That URL is: http://www.ccs.cz/data/141001_Seznam_ČStxt14196.xls ...
12
votes
6answers
61k views

CURL to download a directory

I am trying to download a full website directory using CURL. The following command does not work: curl -LO http://example.com/ It returns an error: curl: Remote file name has no length!. But when ...
1
vote
1answer
353 views

How to restart a router with curl or wget to pass URL as well as POST parameters?

I have an Edimax router and from wireshark I see that the router restarts with the POST http://192.168.2.1/goform/formReboot and the form elements reset_flag=0&submit-url=%2Fadmin_restart.asp ...
0
votes
0answers
21 views

How to Download HTML Pages of a WebSite, Based on a Time Stamp?

I know this might be a repeated question, and I'm sorry for that, but I failed to find any answer for my problem . I Just want to download the Pages html files, without any other contents, and (If ...
2
votes
0answers
33 views

How to add a 'greek' character in the path using cURL?

I have a Makefile where one of many tasks is to upload a file using cURL, one of these files, has a greek character in the name and when I try to upload using cURL, I receive this error: curl: (43) A ...
0
votes
1answer
132 views

curl bails on download in cygwin but not os x

I'm trying to download a copy of a postgresql db from Amazon S3 in cygwin. But it yields zero's across the board yielding a useless file. This is my curl command: curl `heroku.bat pgbackups:url` -o ...
0
votes
1answer
1k views

cURL “response reading failed” in FTPS file transfer

I am trying to download a directory and it's files using cURL via FTPS. The following command gets me very close - I successfully logged in and set the clear command channel, but then it fails with ...