curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

25
votes
2answers
76k views

How to send POST with body, headers, and HTTP params using cURL?

I found a lot of examples on how to use simple POST commands in cURL, but I didn't find examples on how to send full HTTP POST commands, which contain: Headers (Basic Authentication) HTTP Params ...
0
votes
2answers
4k views

How to download a file behind authentication via Wget or cURL [duplicate]

Possible Duplicate: Wget a page that requires logging in I wish to download file from http://mbr.nlm.nih.gov/Download/MetaMapped_Medline/2011/. However, in order to download the file, I ...
0
votes
2answers
117 views

Log date and external ip address with curl and batch (Windows)

I am having weird issues where my external IP is changed VERY often as in every 2, 3 minutes. I have a dynamic dns service in place that I update via task scheduler with curl called in a batch file. ...
1
vote
0answers
62 views

FTP: wildcard downloading

I am trying to download a file with name ending with *.csv The problem is complicated by the fact that this *.csv contains in directory of unknown name: eg. Sometimes it would be 2015->Date ...
1
vote
2answers
364 views

Upload image to postimage.org using curl

I found a script that can upload image to postimage.org using curl. It was posted not long ago, on 2013-09-06, at http://crunchbang.org/forums/viewtopic.php?id=28867. The simplified script looks like ...
0
votes
1answer
44 views

How accurate is CURL's 'current rate' in the progress meter?

Is the second-by-second output in curl accurate "enough"? What is the period that the "current rate" is calculated over? I am currently using the second-by-second status output in Curl as a general ...
6
votes
1answer
554 views

How can I use a command line tool like Wget to log into an OpenID site?

Specifically, I would like to be able to download certain pages from my user profile on the various Stack Exchange sites. I would, however, like to do this automatically (using a cron job), from ...
1
vote
2answers
676 views

How can I use curl with ::1 for ipv6 based loopback?

I tried a few ways but they seem to be confusing curl. root@testt:~# curl localhost:8080 <h1>Hello world!</h1> <p>Boom Bam Splat</p> root@testt:~# curl ::1:8080 curl: (3) ...
0
votes
0answers
13 views

Curl should be quit but it returns ever something

I tried to login to a website using curl in a bash-script. It is possible to login to the site quiently using this: curl -d "username=$username&passwd=$passwd" -c $cookiefile ...
1
vote
1answer
64 views

Bulk SFTP download of files specified by URL preserving directories

I have a list of several hundred files I need to pull via SFTP. They are located in various directories and I need them to be in the same directory setup on my local machine. It is important the files ...
0
votes
0answers
215 views

How to use .pfx in certificate store in CURL for Windows 7

I want to use cURL to access a remote server, and I need to identify myself using a private key or the remote server gives a 401 "Unauthorized". I'm on Windows 7 (work machine, not my choice) and my ...
0
votes
1answer
48 views

How can you lock computer while using curl to download files?

I’m using a Python script in the Terminal in Mac OS X 10.10.2 to download many files from a webpage using curl. However sometimes I might want to lock my computer and walk away but this terminates my ...
2
votes
1answer
3k views

How to split the HTTP error code from the contents in cURL?

Yes, this is related to Getting curl to output HTTP status code? but unfortunately not the same. In a script I would like to run: curl -qSfsw %{http_code} URL where the -f option ensures that the ...
1
vote
0answers
168 views

I destroyed my Mac's PHP setup: SSL certification fails

Long story short, I deleted the PHP symlink on my Mac, and replaced it with a symlink from XAMPP. Now it seems that there's a problem with my SSL configuration or what seems to me, a lack thereof. ...
2
votes
0answers
77 views

How to curl or wget live tennis scores from a Google search page [closed]

Google has some nice live tennis scores displayed on the search page. If I search for "Halep", for example, I get the tournament she's currently playing, together with all the result and live score. ...
0
votes
0answers
20 views

How to check what kind of files web page consists

Is there a way to check what files web page consists. For example I want to download .txt files from the web page, but I dont want all .txt files, some of them, and before I want to check are there ...
0
votes
1answer
64 views

Is there a way to download all .txt files from the page via curl or wget command

My question is in a title: Is there a way to download all .txt files from the page via curl or wget command? Thanks
1
vote
1answer
258 views

Exit bash script when curl gets a non 200 HTTP status

I have a bash script setup to perform a few curl requests for currency in EUR INR JPY do curl -i --data '{"currency": "'$currency'"}' -H "Accept: application/json" -H "Content-Type: ...
0
votes
0answers
48 views

Need help constructing cURL config file

I have a (rather long) cURL command that I'm currently running as follows: curl.exe -U : --proxy-ntlm --proxy myproxyserver --cookie-jar cookies.txt -H "Accept-Encoding: gzip,deflate,sdch" -H ...
0
votes
0answers
68 views

Downloading file from (your download will start shortly) page

How can i download a file with curl (or any other alternative) that make user wait until download start for example enter link description here i tried curl -O -J -L ...
2
votes
1answer
424 views

dyld: Library not loaded: /usr/lib/libcurl.4.dylib curl SSL error

I'm not sure what happened to my system but I am having an issue with dyld/curl, affecting my overall computer performance. I am on Yosemite. I have a feeling it's the Conda openssl. How do I just ...
0
votes
0answers
230 views

Why is cURL slower than wget

Test page: https://www.beobank.be/nl/Home.aspx This is the result using wget (or a real browser): # time wget https://www.beobank.be/nl/Home.aspx -O /dev/null --2015-01-26 12:05:46-- ...
25
votes
3answers
7k views

Curl local host names on Mac OS X Yosemite

I just upgraded from Mavericks to Yosemite, and now curl can't see loopback host names. Set up a simple http server to test: $ python -m SimpleHTTPServer Serving HTTP on 0.0.0.0 port 8000 ... Now ...
0
votes
0answers
157 views

nginx not serving php files with curl in fedora 21

phpinfo and php with echo are working fine but with curl gives 403 error. error in chrome: http://localhost/send3.php Failed to load resource: the server responded with a status of 403 (Forbidden) ...
0
votes
0answers
36 views

Prevent WAMP from loading external requests

I need to prevent wamp from loading external links in order to have a better performance I'm using PrestaShop and sometimes it looks for updates and currency rates etc which slows down loading the ...
0
votes
1answer
102 views

cURL: HTTP file download

I have tried to download a file using curl "https://www.dropbox.com/s/tf7e7vndqd9hv05/Ace_The_IELTS.pdf?dl=0" -o "file.pdf" but it downloads a file which is of few bytes. When I open the link in the ...
0
votes
1answer
61 views

how to download pictures with wget

i would like to download pictures from http://myhomepage.com/plots/20150111-03/4711/4712.png every 3 hours. "20150111-03" is variable, rest is stable. That means every 3 hours the download looks like: ...
2
votes
1answer
318 views

Error executing sftp with curl

I'm using IBM's Data Explorer to import certain data, but am getting this error: Protocol sftp not supported or disabled in libcurl. So from Red Hat I manually ran the command: curl -user ...
0
votes
0answers
120 views

Cannot curl webserver from outside network

I have a webserver running on one of my home computers. I am able to access it within my network, on another computer, with curl localWebserverIP:8888. However, I am having issues accessing it from ...
0
votes
1answer
91 views

RPM created using checkinstall giving dependency error

Compiled openssl-1.0.1.i using commands: export CFLAGS="-fPIC" ./config shared --openssldir=/usr/share/ssl make depend make all make install here it is not asking for any dependencies I am able to ...
0
votes
3answers
113 views

Bash: executing concatenated variables

I have a Linux bash script, to the effect of: COMMAND_TO_EXECUTE="foo "$1" --option1 --option2 "$2 exec $COMMAND_TO_EXECUTE Problem: the script fails Debugging clue, If I echo ...
1
vote
2answers
2k views

libcurl or cURL not working OSX Mavericks

Before I start let me say I have uninstalled, reinstalled, compiled from source, used macports and generally tried for the last 6 hours to get curl working on my Mac OSX Mavericks but I still haven't ...
0
votes
0answers
30 views

Send and receive file in curl

I have a remote server running and i want to send a text file to that server. I have this script to upload the file: curl -F 'txtfile=@sometxtfile.txt''http://www.example.com/transfers' My ...
0
votes
1answer
40 views

Download file on unix from target url which autodownloads file on page visit

I'm trying to download a file to a unix machine on the command line. Accessing a GUI on this particular machine would be quite difficult. The file is autodownloaded when visiting the url in a web ...
1
vote
2answers
6k views

Upload folder with cURL and FTP using Batch file on Windows?

I am trying to upload a whole folder, including it's sub-folders and files to a folder on my server from a Windows batch (.bat) file. Below is the command I have for the Upload part... curl -T ...
1
vote
1answer
3k views

When using curl to http post a xml file to a url, how to set the parameter?

I want to use curl to http post a file to a url like: curl -d myfile.xml http://www.example.dom/post/file/here Can I set the parameter I want the file to be posted with? Like if it was a ...
0
votes
0answers
90 views

Posting XML through cURL using --data-binary

I have this really typical problem. I have an XML file that I have to post to a server. I was told by the network engineer of that site to use the cURL function. The function that he provided to me ...
5
votes
1answer
11k views

CURL File with post parameter

How to make a request for uploading a file to facebook using graph api in curl I am making this request but getting Error: curl https://graph.facebook.com/<id>/photos -F "source=@me.jpg" -d ...
39
votes
2answers
39k views

How do you display POST data with cURL?

As an example, POSTing to a web server with the -v argument: curl -v http://testserver.com/post -d "firstname=john&lastname=doe" And the output > POST /post HTTP/1.1 > User-Agent: ...
1
vote
2answers
1k views

curl can't connect; wget can

trying to connect to my script on a remote server with curl but: col-pos-spd# curl -v http://***:80/monitor/reply.php * About to connect() to *** port 80 (#0) * Trying 83.xxx.xxx.2... Connection ...
0
votes
1answer
160 views

Where does a file downloaded with CURL go on the file system?

Ive downloaded a file from this site with the following curl command: https://www.modern.ie/en-us/virtualization-tools#downloads # Copy/paste the command below into your terminal to begin ...
0
votes
1answer
309 views

Windows CURL download file- special char in URL

I need to download .xls file with Windows cURL and I have got a problem with URL of that file, because it contains special char "Č". That URL is: http://www.ccs.cz/data/141001_Seznam_ČStxt14196.xls ...
12
votes
6answers
52k views

CURL to download a directory

I am trying to download a full website directory using CURL. The following command does not work: curl -LO http://example.com/ It returns an error: curl: Remote file name has no length!. But when ...
1
vote
1answer
233 views

How to restart a router with curl or wget to pass URL as well as POST parameters?

I have an Edimax router and from wireshark I see that the router restarts with the POST http://192.168.2.1/goform/formReboot and the form elements reset_flag=0&submit-url=%2Fadmin_restart.asp ...
0
votes
0answers
20 views

How to Download HTML Pages of a WebSite, Based on a Time Stamp?

I know this might be a repeated question, and I'm sorry for that, but I failed to find any answer for my problem . I Just want to download the Pages html files, without any other contents, and (If ...
2
votes
0answers
27 views

How to add a 'greek' character in the path using cURL?

I have a Makefile where one of many tasks is to upload a file using cURL, one of these files, has a greek character in the name and when I try to upload using cURL, I receive this error: curl: (43) A ...
0
votes
1answer
106 views

curl bails on download in cygwin but not os x

I'm trying to download a copy of a postgresql db from Amazon S3 in cygwin. But it yields zero's across the board yielding a useless file. This is my curl command: curl `heroku.bat pgbackups:url` -o ...
0
votes
1answer
699 views

cURL “response reading failed” in FTPS file transfer

I am trying to download a directory and it's files using cURL via FTPS. The following command gets me very close - I successfully logged in and set the clear command channel, but then it fails with ...
-1
votes
1answer
138 views

POST to web form using CURL & BASH

I'm trying, without success, to use cURL to post data to the form on this URL: https://nic.ar/buscarDominio.xhtml how can i correctly send the request to the server? I've analysed the post data ...
0
votes
3answers
243 views

Is it possible to get the download file name of an mp3 from server using bash?

I am trying to get the final download file name of an mp3 from a server, specifically audiotool.com e.g. http://api.audiotool.com/track/haunt_opaque/mixdown.mp3 When you navigate to this url in a ...