curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

0
votes
1answer
16 views

How can I get more information from Curl regarding a FTP 425 “Cannot open data connection” error?

I am trying to upload a file with the following line: curl -3 --ftp-pasv --ftp-ssl-control -v -i -T ./mybigfile.gz ftp://lawrence@mecloud.xxxxxxx.co.uk:21/OnlineBackup/Lawrence/ --insecure -u ...
0
votes
1answer
19 views

Using cURL to retreive public IP for tmux config creates too many connections

I am currently using the command: curl icanhazip.com and the option set -g status-left ... in my tmux.conf to retrieve and display my public IP in status bar. This works great, although I am really ...
0
votes
0answers
14 views

Downloading an authenticated large file with command line tools

I need to download a file sized around 1.6GB. I's like to get it download via a command line tool like axel, wget or curl However, how can I download a file which needs to be logged in to the site ...
1
vote
0answers
12 views

Sporadic loss of access to local server

For some reason, I lose access to a local server called lux-app04 seemingly at random, but only in some applications. As you can see in the image below, Postman (top) can connect to it, but curl ...
0
votes
0answers
12 views

curl to resume dropped file copy

I refer to an older question in the forum: Question asked by Simon Fox My laptop froze up in the middle of a (large) file copy from my desktop (over a simple LAN). The copy operation was about ...
0
votes
2answers
21 views

curl and sni-enabled server

I am running curl against a sni-enabled server with the following command curl --cacert CustomCA.crt -H "Host: example.com" https://1.2.3.4/foo However, I am not getting the right certificate where ...
0
votes
0answers
47 views

SSL certificate error attempting to download Homebrew?

I've been trying to download and install Homebrew for my Mac (OS 10.9.4) using the install script on the home page, but the download fails with a SSL certificate error (see log below). I understand ...
0
votes
0answers
20 views

How can I get all urls of pattern x from a paged list of search results?

I'm using Windows 7 SP1, and I'm willing to use powershell or wget or whatever will work. I downloaded powershell today (ps 4) and I've got some scripts working, but I'm having trouble with this one. ...
0
votes
0answers
16 views

Curl slow download from other server

I'm trying to download large files to my personal VPS server from public video sharing site . curl -o bigfile.mp4 -A "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20120101 Firefox/29.0" "URL" ...
0
votes
0answers
23 views

curl won't shut up (--silent does nothing)

I'm writing a quick shell script, and I cannot for the life of me get curl to suppress the progress bar. The relevant line looks like this: curl http://website -o $temp >/dev/null 2>&1 ...
0
votes
0answers
17 views

Parallel download for a list of urls and renaming

I have a list of tab-separated urls, and target file names, urls_to_download.txt, for example: first_file.jpg\thttps://www.google.co.il/images/srpr/logo11w.png ...
0
votes
2answers
28 views

How to download all file with name emp* using curl from a FTP site

How can I get all files downloaded from a FTP site directory with file name like emp*. Please suggest.
0
votes
1answer
31 views

How to automatically save a list of files with different names?

I have a text file of URLs, each in the form of: http://domain.com/a/b/c/d.jpg I want to download all of these, but save each file under the name: c_d.jpg In other words, for each file, I want to ...
1
vote
1answer
48 views

Upload image to postimage.org using curl

I found a script that can upload image to postimage.org using curl. It was posted not long ago, on 2013-09-06, at http://crunchbang.org/forums/viewtopic.php?id=28867. The simplified script looks like ...
0
votes
0answers
21 views

curl bails on download in cygwin but not os x

I'm trying to download a copy of a postgresql db from Amazon S3 in cygwin. But it yields zero's across the board yielding a useless file. This is my curl command: curl `heroku.bat pgbackups:url` -o ...
0
votes
0answers
20 views

import .cer file into ubuntu for curl

I am trying to use fiddler to act as a ssl proxy for debugging. I am trying to install fiddlers root certificate. I am mainly doing it so I can debug my python program. Using python I get a cert ...
0
votes
3answers
47 views

Is there a linux filter which converts html to simple and pretty formatted text?

I would like to filter html response coming from cURL output. Which filter should I apply? Viewing html content with Midnight Commander is no problem, it shows them as pretty formatted plain text, so ...
1
vote
1answer
115 views

How to upload file with curl when --data-binary is present in tampered data?

I try to upload files to a popular website, I tampered the http POST requests with Chrome developer tools' "copy as curl" feature. When the upload begins I get this curl command: curl ...
0
votes
0answers
56 views

How can I use Wget / Curl / Plowshare to download from 1fichier.com as a premium member?

How can I download files from 1fichier.com (I am a premium member) with the help of Wget / Curl / Plowshare (Plowdown) or other CLI / command line tool / script?
0
votes
0answers
18 views

Curl over interface for port forwarding

I'm trying to set up OpenVPN and Deluge to seed torrents via the VPN. I am running other services on the server, so I only want the torrent info to go over the VPN. The way I've accomplished this so ...
1
vote
1answer
35 views

downloading multiple files using curl, curl repeats address in download

I am running the command curl -O "http://www.*site*.com/[1-9].png" but the download is failing. - the output is as follows [1/10]: http://www.*site*.com/1http://w --> 1.png % Total % ...
0
votes
0answers
33 views

crawling google links produces 400 error

I had a google scraper some time ago that quit working when they made some security changes. I just had a task come up and realized something must of changed a lot. the best I can get with any tool to ...
1
vote
2answers
62 views

file downloaded via firefox and curl have different size

When I download a file from this link by Firefox its size is 74580 B, But when I download it by curl with exactly all of header was sent by Firefox its size is 79891 B (I copied all header from ...
2
votes
0answers
122 views

curl: (7) Failed connect to www.google.com:80;

I have installed Git for Windows and got cURL automatically. C:\>curl --version curl 7.30.0 (i386-pc-win32) libcurl/7.30.0 OpenSSL/0.9.8y zlib/1.2.7 Protocols: dict file ftp ftps gopher http https ...
1
vote
1answer
41 views

What is the curl equivalent to wget -x?

I want to use curl to save multiple files with the same local paths as would be produced by wget -x (ie, cleaned up versions of the full urls). But I can't find the right option in the curl docs. Is ...
0
votes
0answers
55 views

What is the curl command line syntax to do a post request in windows

I know this question was answered often, but mine is some more special. I have the following Code: <form action="/test.html" method="post" enctype="multipart/form-data"> <a ...
0
votes
0answers
310 views

libcurl or cURL not working OSX Mavericks

Before I start let me say I have uninstalled, reinstalled, compiled from source, used macports and generally tried for the last 6 hours to get curl working on my Mac OSX Mavericks but I still haven't ...
1
vote
1answer
58 views

Curl won't work over FTP

I am trying to install Wine on my Macbook (OS X 10.9.3). First I tried using Homebrew which lists a set of dependencies and then goes to fetch the first. ==> Installing dependencies for wine: ...
0
votes
0answers
40 views

Curl to download large CSV

I am using curl through Cygwin on Windows 7 to download content from Apache Solr in CSV format. This is my command which is running in a for loop curl -o ret-key-data-$i.csv $URL As the content is ...
0
votes
0answers
13 views

Curl works, curlfrpfs doesn't

Following commands work properly: $ curl ftp://user:*****@sma20/ 05-04-14 14:45 872 config.xml 04-11-14 07:26 <DIR> DATA $ curl ...
0
votes
0answers
93 views

Trouble installing Homebrew “curl: (2) Failed initialization”

When I run the command to install homebrew on Mac OSX 10.7, namely ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)", I get curl: (2) Failed initialization. Searching ...
0
votes
1answer
72 views

cURL very long delay before starting download from ftp

I'm trying to write a script to download a file, convert it, then execute a program on the converted file. For some reason curl takes over 1 minute to start the file transfer. Command: curl ...
1
vote
0answers
21 views

Bug or syntax change on Debian's “curl” 7.36? Mess when trying to save sequential files with brackets/hash

(Perhaps specifically 7.36.0-1, which is shown as the most recent available on my package manager). I think there was such a bug some time ago, but perhaps it was back in curl 7.33, I can't ...
1
vote
2answers
137 views

How to stop shell script if curl failed

I have script that used curl when i pass wrong parameters to script curl failed but script continue executing. I have tried use curl -f/--fail parameter but problem does not solved. What is the best ...
9
votes
2answers
803 views

how to use curl to verify if a site's certificate has been revoked?

To check if the certificate for google.com has been revoked, I tried the following command: curl https://www.google.com --cacert GeoTrust_Global_CA.pem --crlfile gtglobal.pem -v , but I got the ...
0
votes
0answers
181 views

Why is SSL broken for one domain on one computer?

I've been having a strange issue, where I cannot connect to https://www.humblebundle.com. Plain HTTP works fine, as does https://humblebundle.com. These all redirect to the above address. I haven't ...
0
votes
0answers
43 views

wget sometimes fails, curl succeeds

I am continuously downloading some packets via wget. Once in a while, the wget fails. I have used retries as 2 and timeout as 10 seconds. When the failure occurs, I try the same URL again, but with ...
0
votes
0answers
128 views

curl: (35) error:0200100D:system library:fopen:Permission denied

In continuation with this question... I get curl error as above only as an unprivileged user on my system. However, on searching I found the above link which asks me to check permissions for ...
0
votes
0answers
30 views

Gzipped file arrives unzipped

I've recently set up an ArchLinux system at work and added the ArchHaskell repo. This is far from the first time I do this. I have a number of systems where it's worked just fine, but this last time ...
1
vote
1answer
105 views

posting data to a web server using curl

I am trying to post data to a web sever using curl -d and curl -X POST. The command is getting succeeded but nothing is there in the file. Tried the following: [root@oooatis santoshi]# curl ...
1
vote
0answers
31 views

SOAP-request realization in DOS 6.22 16-bit

I have a task to develop some application providing SOAP-requests realization for DOS. I have already developed a similar application under Linux, using C++/STL and curl and openssl libraries. Is ...
0
votes
0answers
368 views

How can I download private repo from github?

What I want to do is to download private repository archive from GitHub, extract it, remove archive file and copy some directories that are inside downloaded project. I tried to use wget but I cannot ...
1
vote
1answer
116 views

How do I comment out a line in my .curlrc?

I frequently have to switch between two different auth token request headers, that I keep in my .curlrc as such: -H "X-Auth-Token: 12345678-1234" It would make my life easier, though, if I could ...
0
votes
2answers
129 views

curl shell command always returns error status 0 - even when fails on input errors

Example: trying to upload a non existing file - curl -d @no-real-file http://google.com | echo $? - returns 0. How can I make curl fail on input errors?
2
votes
1answer
129 views

How can I get a list of all messages in my gmail account in linux command line?

How can I get a list of all messages in my gmail account in linux command line? Ideally I would want to see something similar to: from:sender1@domain.com to:me@gmail.com Date:11/22/33 subj:Foo1 ...
0
votes
0answers
75 views

Error uncompressing file when piping curl to gunzip

I need to download a gzipped xml feed to parse in my Ruby app. It may well be quite large, 30Mb compressed, 300Mb uncompressed. So I do this: curl -s http://example.com/file.xml.gz | gunzip and read ...
0
votes
0answers
176 views

Choose the filename when using POST with curl?

I'd like to POST a file upload with curl, and choose the filename at request time. Here's the curl line I have right now, which works to upload the file: curl --form "upload_file=@content.txt" ...
0
votes
0answers
80 views

Pinging local machine with external IP

On a Gentoo server, I am trying to wget or curl a page from its url: wget http://example.com/test.html (I can get result for this url from my browser) All I receive is Timeout. With cUrl I get ...
0
votes
0answers
17 views

Can One Extract a Hyphenated Filename File From a Server Using Curl?

The images I'm dealing with have file names like part-1.jpg, object-a.jpg, Aspect-0.jpg. What I'm wondering is what I put in place of ???: curl -O http://server/[???].jpg
0
votes
0answers
359 views

Libcurl not usable on mac os x

I'm running the configure command to install a particular software package, but it fails when it reaches it's libcurl requirement. This is the output it gives: checking for the version of ...