curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

2
votes
0answers
13 views

How to add a 'greek' character in the path using cURL?

I have a Makefile where one of many tasks is to upload a file using cURL, one of these files, has a greek character in the name and when I try to upload using cURL, I receive this error: curl: (43) A ...
2
votes
0answers
382 views

loop curl could not resolve host

I've had working script, which suddenly doesn't works. It contains something like this: while read url; do curl $url done < links links file: http://duckduckgo.com http://google.com And I ...
2
votes
0answers
144 views

Curl --insecure error in Cygwin (works in Linux)

Curl is returning an error in Cygwin. How do I eliminate this error? curl --insecure -X POST -d '<run><log ...
2
votes
0answers
110 views

curl works but ping/urllib2 don't, in a virtual machine

I have a Crunchbang 10 VM running on VirtualBox on a Windows 7 Home host. I have a python script that uses urllib2 that was failing, and discovered that: ordinary browsing and curl work normally on ...
2
votes
0answers
305 views

PHP : curl is and Not working in ubuntu10.04 and Nginx

I am running Nginx server on ubuntu 10.04 I am using the php version 5.3.2 .I installed php5-curl using apt-get install php5-curl. Unfortunately the curl package is not displaying in phpinfo() . ...
2
votes
0answers
573 views

Restore default CA Certificates on Mac OSX 10.7

I've been screwing around with my Mac, trying to install self-signed certificate as trusted for our WAN site (for which we can't justify paying for a real signed certificate) I've broken something ...
1
vote
0answers
26 views

How to download certain elements of a web page, skipping long tag attributes

(formerly titled: Curl -- how to exclude certain elements) Background: I regularly use command line cURL (on linux) to retrieve web pages. Situation: A new page I have to get is heavily bloated with ...
1
vote
0answers
13 views

Sporadic loss of access to local server

For some reason, I lose access to a local server called lux-app04 seemingly at random, but only in some applications. As you can see in the image below, Postman (top) can connect to it, but curl ...
1
vote
0answers
22 views

Bug or syntax change on Debian's “curl” 7.36? Mess when trying to save sequential files with brackets/hash

(Perhaps specifically 7.36.0-1, which is shown as the most recent available on my package manager). I think there was such a bug some time ago, but perhaps it was back in curl 7.33, I can't ...
1
vote
0answers
38 views

SOAP-request realization in DOS 6.22 16-bit

I have a task to develop some application providing SOAP-requests realization for DOS. I have already developed a similar application under Linux, using C++/STL and curl and openssl libraries. Is ...
1
vote
0answers
228 views

pipe unlimited curl stdout into chunked files

I've got an HTTP-endpoint that serves an unlimited stream of events. Now I'd like to record that stream using curl, but splitting the recorded files at the same time - preferably based on the line ...
1
vote
0answers
57 views

How do I log in to “The Cloud” wifi hotspots using curl?

In England at least, there's this accursèd thing called 'The Cloud', which increasingly provides all public wifi hotspots. Has anyone already knocked up a curl command/bash script/etc to log in to it ...
1
vote
0answers
129 views

Curl not working on OS X

I recently upgraded XCode and while I'm not sure if that caused the problem, I can't curl out of my computer anymore. This seems to break rvm and homebrew but git still works. I upgraded curl and ...
1
vote
0answers
119 views

Permissions tar extraction to nfs share from a Perl script

As part of an installation procedure of a piece of software, I use: curl -s <url-to-targz> | tar -p -x -z -C / inside a Perl script (I use qx($command) or system($command), either will ...
1
vote
0answers
476 views

Libcurl-devel on Mac OS X

I am trying to build an application that depends on libcurl version 7.21 or newer on Mac OS X 10.5.8. I tried downloading and installing the latest version from libcurl's website, but the application ...
1
vote
0answers
122 views

No output returned when curl ing any URL in Red Hat 6 but the same url works in lynx browser

I am trying to install some packages in Python using easy install. But i get a 'Download Error' for any package i try to install. So i tried to test if internet is accessible at all from the machine ...
1
vote
0answers
157 views

the right options to traverse/download the pages/directories of a subdomain

Let's suppose exist a site with the following directories (subdomain) index.php |-sub1 |-index.php |-sub1sub1 |-index.php |-other.php |-sub1sub1sub1 |-sub2 |-index.php |- ...
1
vote
0answers
120 views

Download web server structure with empty files

I want to make a mirror of a Web server, but downloading the actual files will take too long. So I thought of having just the directory and file structure, and when I need the actual contents of the ...
1
vote
0answers
553 views

VirtualBox Ubuntu guest on OS X Lion host: Curl won't work

I'm trying to install some gems on an Ubuntu VM on OS X Lion using VirtualBox. I've narrowed my problem down to curl which is not responding at all. It seems to be going through some proxy local host ...
1
vote
0answers
201 views

how to use to proxy in the intranet?

i have an intranet site which is inside only our company and feeds information. i download an weather API from http://www.foreca.com/Philippines/ and hook to my site. the API has a search engine and ...
0
votes
0answers
48 views

Mac OS X Yosemite and Client SSL Certificates

Ever since I upgraded from Mavericks to Yosemite, my SSL Client Certificates have stopped working in certain places. They still work in web browsers and in wget, but they have stopped working in curl ...
0
votes
0answers
14 views

Send and receive file in curl

I have a remote server running and i want to send a text file to that server. I have this script to upload the file: curl -F 'txtfile=@sometxtfile.txt''http://www.example.com/transfers' My ...
0
votes
0answers
14 views

Posting XML through cURL using --data-binary

I have this really typical problem. I have an XML file that I have to post to a server. I was told by the network engineer of that site to use the cURL function. The function that he provided to me ...
0
votes
0answers
36 views

Download file vis curl but not with wget

I have a link I'm trying to download. Before, we could download with either wget or curl because the website (which runs under Tomcat) didn't require a user name or password in order to download. ...
0
votes
0answers
20 views

How to Download HTML Pages of a WebSite, Based on a Time Stamp?

I know this might be a repeated question, and I'm sorry for that, but I failed to find any answer for my problem . I Just want to download the Pages html files, without any other contents, and (If ...
0
votes
0answers
14 views

Curl to update dyndns record for *.domain.com

I use curl to update my dns whenever my ip changes and it works as expected. url = domain.com user = username pass = password ext = 123.123.123.123 curl -s --ssl --user "$user:$pass" ...
0
votes
0answers
14 views

upload Multiple xml on https form

I have a annoying problem, I must send daily +-100 xml files over a https,an upload form without a layout and can't see which files are on the server. Synch isnt allowed. I've tried everything, the ...
0
votes
0answers
68 views

How to emulate a cookie-based browser with curl?

The title tells pretty much everything. I need curl to both GET and POST as a web browser while accepting cookies.
0
votes
0answers
22 views

How to set curl -o to any desired location in AppleScript?

I have a basic question about curl -o. I just want to make a short AppleScript. does shell script "curl -f 'http://www.SOMEWEBSIT.COM/ANYIMAGE.jpg' -o ~/Desktop/ANYIMAGE.jpg" tell application ...
0
votes
0answers
53 views

Downloading an authenticated large file with command line tools

I need to download a file sized around 1.6GB. I's like to get it download via a command line tool like axel, wget or curl However, how can I download a file which needs to be logged in to the site ...
0
votes
0answers
39 views

curl to resume dropped file copy

I refer to an older question in the forum: Question asked by Simon Fox My laptop froze up in the middle of a (large) file copy from my desktop (over a simple LAN). The copy operation was about ...
0
votes
0answers
173 views

SSL certificate error attempting to download Homebrew?

I've been trying to download and install Homebrew for my Mac (OS 10.9.4) using the install script on the home page, but the download fails with a SSL certificate error (see log below). I understand ...
0
votes
0answers
33 views

How can I get all urls of pattern x from a paged list of search results?

I'm using Windows 7 SP1, and I'm willing to use powershell or wget or whatever will work. I downloaded powershell today (ps 4) and I've got some scripts working, but I'm having trouble with this one. ...
0
votes
0answers
69 views

Curl slow download from other server

I'm trying to download large files to my personal VPS server from public video sharing site . curl -o bigfile.mp4 -A "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20120101 Firefox/29.0" "URL" ...
0
votes
0answers
44 views

curl won't shut up (--silent does nothing)

I'm writing a quick shell script, and I cannot for the life of me get curl to suppress the progress bar. The relevant line looks like this: curl http://website -o $temp >/dev/null 2>&1 ...
0
votes
0answers
134 views

Parallel download for a list of urls and renaming

I have a list of tab-separated urls, and target file names, urls_to_download.txt, for example: first_file.jpg\thttps://www.google.co.il/images/srpr/logo11w.png ...
0
votes
0answers
59 views

import .cer file into ubuntu for curl

I am trying to use fiddler to act as a ssl proxy for debugging. I am trying to install fiddlers root certificate. I am mainly doing it so I can debug my python program. Using python I get a cert ...
0
votes
0answers
171 views

How can I use Wget / Curl / Plowshare to download from 1fichier.com as a premium member?

How can I download files from 1fichier.com (I am a premium member) with the help of Wget / Curl / Plowshare (Plowdown) or other CLI / command line tool / script?
0
votes
0answers
36 views

Curl over interface for port forwarding

I'm trying to set up OpenVPN and Deluge to seed torrents via the VPN. I am running other services on the server, so I only want the torrent info to go over the VPN. The way I've accomplished this so ...
0
votes
0answers
47 views

crawling google links produces 400 error

I had a google scraper some time ago that quit working when they made some security changes. I just had a task come up and realized something must of changed a lot. the best I can get with any tool to ...
0
votes
0answers
140 views

What is the curl command line syntax to do a post request in windows

I know this question was answered often, but mine is some more special. I have the following Code: <form action="/test.html" method="post" enctype="multipart/form-data"> <a ...
0
votes
0answers
93 views

Curl to download large CSV

I am using curl through Cygwin on Windows 7 to download content from Apache Solr in CSV format. This is my command which is running in a for loop curl -o ret-key-data-$i.csv $URL As the content is ...
0
votes
0answers
22 views

Curl works, curlfrpfs doesn't

Following commands work properly: $ curl ftp://user:*****@sma20/ 05-04-14 14:45 872 config.xml 04-11-14 07:26 <DIR> DATA $ curl ...
0
votes
0answers
155 views

Trouble installing Homebrew “curl: (2) Failed initialization”

When I run the command to install homebrew on Mac OSX 10.7, namely ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)", I get curl: (2) Failed initialization. Searching ...
0
votes
0answers
284 views

Why is SSL broken for one domain on one computer?

I've been having a strange issue, where I cannot connect to https://www.humblebundle.com. Plain HTTP works fine, as does https://humblebundle.com. These all redirect to the above address. I haven't ...
0
votes
0answers
62 views

wget sometimes fails, curl succeeds

I am continuously downloading some packets via wget. Once in a while, the wget fails. I have used retries as 2 and timeout as 10 seconds. When the failure occurs, I try the same URL again, but with ...
0
votes
0answers
245 views

curl: (35) error:0200100D:system library:fopen:Permission denied

In continuation with this question... I get curl error as above only as an unprivileged user on my system. However, on searching I found the above link which asks me to check permissions for ...
0
votes
0answers
34 views

Gzipped file arrives unzipped

I've recently set up an ArchLinux system at work and added the ArchHaskell repo. This is far from the first time I do this. I have a number of systems where it's worked just fine, but this last time ...
0
votes
0answers
107 views

Error uncompressing file when piping curl to gunzip

I need to download a gzipped xml feed to parse in my Ruby app. It may well be quite large, 30Mb compressed, 300Mb uncompressed. So I do this: curl -s http://example.com/file.xml.gz | gunzip and read ...
0
votes
0answers
318 views

Choose the filename when using POST with curl?

I'd like to POST a file upload with curl, and choose the filename at request time. Here's the curl line I have right now, which works to upload the file: curl --form "upload_file=@content.txt" ...