curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

0
votes
1answer
1k views

How can I apply quotes to a URL in a .bat script?

The general form of a curl command in CMD that works is: curl -o latest.dump -L "https://s3.amazonaws.com/hkpgbackups/app28197640@heroku.com/xxxx.dump?AWSAccessKeyId=AKIAJSCBEZJRDOTGNGZQ&Expires=...
0
votes
1answer
976 views

Mac OSX 10.9 resume folder copy to NAS

I have a really big folder (174GB) that I wanted to move to a NAS using Ethernet. I started the transfer and when it was neary finished the computer stopped (I accidentally disabled caffeine) and the ...
3
votes
2answers
1k views

bash: source from URL

In addition to my own computer, I sometimes use an Ubuntu cluster at my school. Rather than manually keep my .bashrc's in sync, I would like to make the school cluster's .bashrc source my personal ....
8
votes
2answers
12k views

POSTing a file's contents with CURL

(The following is backstory, you can jump down to "Question:" if you'd like) There's a webpage for one of my courses that accepts input text (expected to be Python code) and then gives some tokenized ...
24
votes
4answers
32k views

cURL: how to suppress strange output when redirecting?

I'm trying to print just the verbose sections of a cURL request (which are sent to stderr) from the bash shell. But when I redirect stdout like this: curl -v http://somehost/somepage > /dev/null ...
0
votes
1answer
382 views

Curl can't follow “This page should automatically redirect”

I'm attempting to automate a login for the a course webpage I have to check this semester. I know that the POST request I am making is being done correctly because I get to the intermediate redirect ...
0
votes
1answer
468 views

How can I get more information from Curl regarding a FTP 425 “Cannot open data connection” error?

I am trying to upload a file with the following line: curl -3 --ftp-pasv --ftp-ssl-control -v -i -T ./mybigfile.gz ftp://lawrence@mecloud.xxxxxxx.co.uk:21/OnlineBackup/Lawrence/ --insecure -u '...
0
votes
1answer
271 views

Using cURL to retreive public IP for tmux config creates too many connections

I am currently using the command: curl icanhazip.com and the option set -g status-left ... in my tmux.conf to retrieve and display my public IP in status bar. This works great, although I am really ...
1
vote
0answers
13 views

Sporadic loss of access to local server

For some reason, I lose access to a local server called lux-app04 seemingly at random, but only in some applications. As you can see in the image below, Postman (top) can connect to it, but curl (...
8
votes
1answer
21k views

cURL Upload file and send POST data

I have a web server running some PHP that checks for an image (curl -F 'imageName=@myimage') and it also checks the POST data for username=&password=. When the PHP checks _REQUEST I cannot just ...
1
vote
2answers
5k views

curl and sni-enabled server

I am running curl against a sni-enabled server with the following command curl --cacert CustomCA.crt -H "Host: example.com" https://1.2.3.4/foo However, I am not getting the right certificate where ...
1
vote
3answers
9k views

Silence curl with >/dev/null 2>&1 when it's piped

So I have a curl that is piped to a grep and a sed. Where would I apply the >/dev/null 2>&1 ? curl www.site.com | grep stuff | sed "other stuff" At the very end or after the curl?
1
vote
0answers
122 views

curl won't shut up (--silent does nothing)

I'm writing a quick shell script, and I cannot for the life of me get curl to suppress the progress bar. The relevant line looks like this: curl http://website -o $temp >/dev/null 2>&1 ...
0
votes
2answers
372 views

How to download all file with name emp* using curl from a FTP site

How can I get all files downloaded from a FTP site directory with file name like emp*. Please suggest.
0
votes
1answer
150 views

How to automatically save a list of files with different names?

I have a text file of URLs, each in the form of: http://domain.com/a/b/c/d.jpg I want to download all of these, but save each file under the name: c_d.jpg In other words, for each file, I want to ...
2
votes
1answer
3k views

How to upload file with curl when --data-binary is present in tampered data?

I try to upload files to a popular website, I tampered the http POST requests with Chrome developer tools' "copy as curl" feature. When the upload begins I get this curl command: curl 'http://example....
0
votes
3answers
623 views

Is there a linux filter which converts html to simple and pretty formatted text?

I would like to filter html response coming from cURL output. Which filter should I apply? Viewing html content with Midnight Commander is no problem, it shows them as pretty formatted plain text, so ...
0
votes
0answers
596 views

How can I use Wget / Curl / Plowshare to download from 1fichier.com as a premium member?

How can I download files from 1fichier.com (I am a premium member) with the help of Wget / Curl / Plowshare (Plowdown) or other CLI / command line tool / script?
1
vote
1answer
332 views

downloading multiple files using curl, curl repeats address in download

I am running the command curl -O "http://www.*site*.com/[1-9].png" but the download is failing. - the output is as follows [1/10]: http://www.*site*.com/1http://w --> 1.png % Total % ...
0
votes
0answers
66 views

crawling google links produces 400 error

I had a google scraper some time ago that quit working when they made some security changes. I just had a task come up and realized something must of changed a lot. the best I can get with any tool to ...
1
vote
2answers
217 views

file downloaded via firefox and curl have different size

When I download a file from this link by Firefox its size is 74580 B, But when I download it by curl with exactly all of header was sent by Firefox its size is 79891 B (I copied all header from ...
1
vote
1answer
179 views

What is the curl equivalent to wget -x?

I want to use curl to save multiple files with the same local paths as would be produced by wget -x (ie, cleaned up versions of the full urls). But I can't find the right option in the curl docs. Is ...
1
vote
1answer
175 views

Curl won't work over FTP

I am trying to install Wine on my Macbook (OS X 10.9.3). First I tried using Homebrew which lists a set of dependencies and then goes to fetch the first. ==> Installing dependencies for wine: ...
1
vote
0answers
362 views

Curl to download large CSV

I am using curl through Cygwin on Windows 7 to download content from Apache Solr in CSV format. This is my command which is running in a for loop curl -o ret-key-data-$i.csv $URL As the content is ...
0
votes
1answer
298 views

cURL very long delay before starting download from ftp

I'm trying to write a script to download a file, convert it, then execute a program on the converted file. For some reason curl takes over 1 minute to start the file transfer. Command: curl ftp://...
1
vote
0answers
33 views

Bug or syntax change on Debian's “curl” 7.36? Mess when trying to save sequential files with brackets/hash

(Perhaps specifically 7.36.0-1, which is shown as the most recent available on my package manager). I think there was such a bug some time ago, but perhaps it was back in curl 7.33, I can't remember....
1
vote
2answers
3k views

How to stop shell script if curl failed

I have script that used curl when i pass wrong parameters to script curl failed but script continue executing. I have tried use curl -f/--fail parameter but problem does not solved. What is the best ...
0
votes
0answers
457 views

Why is SSL broken for one domain on one computer?

I've been having a strange issue, where I cannot connect to https://www.humblebundle.com. Plain HTTP works fine, as does https://humblebundle.com. These all redirect to the above address. I haven't ...
0
votes
0answers
42 views

Gzipped file arrives unzipped

I've recently set up an ArchLinux system at work and added the ArchHaskell repo. This is far from the first time I do this. I have a number of systems where it's worked just fine, but this last time ...
2
votes
0answers
550 views

loop curl could not resolve host

I've had working script, which suddenly doesn't works. It contains something like this: while read url; do curl $url done < links links file: http://duckduckgo.com http://google.com And I ...
1
vote
1answer
443 views

posting data to a web server using curl

I am trying to post data to a web sever using curl -d and curl -X POST. The command is getting succeeded but nothing is there in the file. Tried the following: [root@oooatis santoshi]# curl --data-...
1
vote
0answers
58 views

SOAP-request realization in DOS 6.22 16-bit

I have a task to develop some application providing SOAP-requests realization for DOS. I have already developed a similar application under Linux, using C++/STL and curl and openssl libraries. Is ...
1
vote
1answer
624 views

calling json-rpc via curl

I'am running Bitcoin-qt 0.8.6.1-beta on Qt 4.8.3 on windows 7 64. below is my .conf content in %appdata%/bitcoin folder rpcuser=bitcoinrpc rpcpassword=ahything rpcallowip=127.0.0.1 rpcallowip=192....
1
vote
1answer
1k views

How do I comment out a line in my .curlrc?

I frequently have to switch between two different auth token request headers, that I keep in my .curlrc as such: -H "X-Auth-Token: 12345678-1234" It would make my life easier, though, if I could ...
0
votes
2answers
706 views

curl shell command always returns error status 0 - even when fails on input errors

Example: trying to upload a non existing file - curl -d @no-real-file http://google.com | echo $? - returns 0. How can I make curl fail on input errors?
3
votes
1answer
1k views

How can I get a list of all messages in my gmail account in linux command line?

How can I get a list of all messages in my gmail account in linux command line? Ideally I would want to see something similar to: from:sender1@domain.com to:me@gmail.com Date:11/22/33 subj:Foo1 from:...
1
vote
2answers
520 views

cURL unable to interpret hostnames, proxy issue?

When using cURL from my ubuntu machine to merely GET from a remote machine, I seem unable to resolve the hostname and just get a Proxy Issue - The proxy has encountered an error (Error 21: Server ...
0
votes
0answers
617 views

Choose the filename when using POST with curl?

I'd like to POST a file upload with curl, and choose the filename at request time. Here's the curl line I have right now, which works to upload the file: curl --form "upload_file=@content.txt" http:/...
0
votes
0answers
257 views

Pinging local machine with external IP

On a Gentoo server, I am trying to wget or curl a page from its url: wget http://example.com/test.html (I can get result for this url from my browser) All I receive is Timeout. With cUrl I get ...
2
votes
2answers
1k views

curl not uploading file to ftp

I am using curl to upload a file to ftp server. The same script is working on another system, but on this one after connecting to ftp it is not doing anything and connection times out. The detailed ...
0
votes
0answers
21 views

Can One Extract a Hyphenated Filename File From a Server Using Curl?

The images I'm dealing with have file names like part-1.jpg, object-a.jpg, Aspect-0.jpg. What I'm wondering is what I put in place of ???: curl -O http://server/[???].jpg
0
votes
0answers
1k views

Libcurl not usable on mac os x

I'm running the configure command to install a particular software package, but it fails when it reaches it's libcurl requirement. This is the output it gives: checking for the version of libcurl.....
1
vote
1answer
309 views

Accessing a remote machine with or without ssh tunneling

I just want to educate myself on an interesting issue here. When I curl a URL I get this error curl "http://myserver:8888/payload" curl: (7) couldn't connect to host However when I setup a tunnel ...
0
votes
1answer
2k views

cURL: multiple POST requests while reusing the TCP connection

If multiple URLs are passed to cURL, it reuses the TCP connection when possible. For example: curl -o 1.jpg http://example.com/1.jpg -o 2.jpg http://example.com/2.jpg I need to do the same, but ...
0
votes
1answer
233 views

File operations on FTPS server with cURL

I need to perform some file transfer operations on some remote FTPS server that uses certificate authentication from a Windows 2008 scripting environment, so that's why I'm stuck with cURL pretty much....
1
vote
0answers
105 views

How do I log in to “The Cloud” wifi hotspots using curl?

In England at least, there's this accursèd thing called 'The Cloud', which increasingly provides all public wifi hotspots. Has anyone already knocked up a curl command/bash script/etc to log in to it ...
0
votes
1answer
199 views

A Script That is Aware of Remote File Change

curl -z pci.ids --remote-time --silent -f -L -O http://pciids.sourceforge.net/v2.2/pci.ids This Code Checks For The Remote File Change and Download If Remote File Has Been Modified but Is There a ...
0
votes
2answers
9k views

curl windows command line [duplicate]

Possible Duplicate: Run cURL commands from Windows console I saw this questions I downloaded the CURL and placed it under C. Still I can't run it from CMD (with or without full path). as well,...
2
votes
1answer
76 views

Shell commands executed from Vim behave differently

I have mapped a key in Vim to run curl-based tests on my API with :map ,t :w\|:!quizzical tests.sh <CR>. The tests run as expected when I press the key; however, the tests which take longer to ...
2
votes
1answer
1k views

Omnisharp for Emacs doesn't work

I tried recently to work with C# with Emacs24. I found https://github.com/sp3ctum/omnisharp-emacs I installed it with MELPA. Then I run it. Nothing happened. I wonder what went wrong. I think I don't ...