curl is a data transfer utility used for a multitude of network=related applications (such as scraping data).

learn more… | top users | synonyms

0
votes
1answer
9 views

What's the exit code for “curl -I” when not HTTP 200?

I want to check what HTTP status code is returned for an HTTP(S) URL. I don't care about content, so I just request head with curl -I $url or curl --head $url But what's the exit code I should ...
0
votes
1answer
28 views

Cannot curl webserver from outside network

I have a webserver running on one of my home computers. I am able to access it within my network, on another computer, with curl localWebserverIP:8888. However, I am having issues accessing it from ...
1
vote
0answers
24 views

How to download certain elements of a web page, skipping long tag attributes

(formerly titled: Curl -- how to exclude certain elements) Background: I regularly use command line cURL (on linux) to retrieve web pages. Situation: A new page I have to get is heavily bloated with ...
0
votes
1answer
10 views

RPM created using checkinstall giving dependency error

Compiled openssl-1.0.1.i using commands: export CFLAGS="-fPIC" ./config shared --openssldir=/usr/share/ssl make depend make all make install here it is not asking for any dependencies I am able to ...
0
votes
3answers
52 views

Bash: executing concatenated variables

I have a Linux bash script, to the effect of: COMMAND_TO_EXECUTE="foo "$1" --option1 --option2 "$2 exec $COMMAND_TO_EXECUTE Problem: the script fails Debugging clue, If I echo ...
0
votes
0answers
28 views

Mac OS X Yosemite and Client SSL Certificates

Ever since I upgraded from Mavericks to Yosemite, my SSL Client Certificates have stopped working in certain places. They still work in web browsers and in wget, but they have stopped working in curl ...
0
votes
0answers
12 views

Send and receive file in curl

I have a remote server running and i want to send a text file to that server. I have this script to upload the file: curl -F 'txtfile=@sometxtfile.txt''http://www.example.com/transfers' My ...
0
votes
1answer
21 views

Download file on unix from target url which autodownloads file on page visit

I'm trying to download a file to a unix machine on the command line. Accessing a GUI on this particular machine would be quite difficult. The file is autodownloaded when visiting the url in a web ...
0
votes
0answers
13 views

Posting XML through cURL using --data-binary

I have this really typical problem. I have an XML file that I have to post to a server. I was told by the network engineer of that site to use the cURL function. The function that he provided to me ...
1
vote
1answer
39 views

Can cURL send requests to sockets?

I can find nothing about this on Google or the cURL man pages. I have a HTTP server running at /var/run/my-server.sock, and I want to test it by sending a simple request using cURL. Can this be done ...
0
votes
1answer
29 views

Where does a file downloaded with CURL go on the file system?

Ive downloaded a file from this site with the following curl command: https://www.modern.ie/en-us/virtualization-tools#downloads # Copy/paste the command below into your terminal to begin ...
8
votes
1answer
1k views

Curl local host names on Mac OS X Yosemite

I just upgraded from Mavericks to Yosemite, and now curl can't see loopback host names. Set up a simple http server to test: $ python -m SimpleHTTPServer Serving HTTP on 0.0.0.0 port 8000 ... Now ...
0
votes
0answers
34 views

Download file vis curl but not with wget

I have a link I'm trying to download. Before, we could download with either wget or curl because the website (which runs under Tomcat) didn't require a user name or password in order to download. ...
0
votes
1answer
88 views

Windows CURL download file- special char in URL

I need to download .xls file with Windows cURL and I have got a problem with URL of that file, because it contains special char "Č". That URL is: http://www.ccs.cz/data/141001_Seznam_ČStxt14196.xls ...
1
vote
1answer
64 views

How to restart a router with curl or wget to pass URL as well as POST parameters?

I have an Edimax router and from wireshark I see that the router restarts with the POST http://192.168.2.1/goform/formReboot and the form elements reset_flag=0&submit-url=%2Fadmin_restart.asp ...
0
votes
0answers
19 views

How to Download HTML Pages of a WebSite, Based on a Time Stamp?

I know this might be a repeated question, and I'm sorry for that, but I failed to find any answer for my problem . I Just want to download the Pages html files, without any other contents, and (If ...
2
votes
0answers
13 views

How to add a 'greek' character in the path using cURL?

I have a Makefile where one of many tasks is to upload a file using cURL, one of these files, has a greek character in the name and when I try to upload using cURL, I receive this error: curl: (43) A ...
0
votes
1answer
194 views

cURL “response reading failed” in FTPS file transfer

I am trying to download a directory and it's files using cURL via FTPS. The following command gets me very close - I successfully logged in and set the clear command channel, but then it fails with ...
-1
votes
1answer
82 views

POST to web form using CURL & BASH

I'm trying, without success, to use cURL to post data to the form on this URL: https://nic.ar/buscarDominio.xhtml how can i correctly send the request to the server? I've analysed the post data ...
0
votes
3answers
81 views

Is it possible to get the download file name of an mp3 from server using bash?

I am trying to get the final download file name of an mp3 from a server, specifically audiotool.com e.g. http://api.audiotool.com/track/haunt_opaque/mixdown.mp3 When you navigate to this url in a ...
0
votes
0answers
14 views

Curl to update dyndns record for *.domain.com

I use curl to update my dns whenever my ip changes and it works as expected. url = domain.com user = username pass = password ext = 123.123.123.123 curl -s --ssl --user "$user:$pass" ...
0
votes
1answer
133 views

How can I apply quotes to a URL in a .bat script?

The general form of a curl command in CMD that works is: curl -o latest.dump -L ...
0
votes
0answers
13 views

upload Multiple xml on https form

I have a annoying problem, I must send daily +-100 xml files over a https,an upload form without a layout and can't see which files are on the server. Synch isnt allowed. I've tried everything, the ...
0
votes
0answers
66 views

How to emulate a cookie-based browser with curl?

The title tells pretty much everything. I need curl to both GET and POST as a web browser while accepting cookies.
0
votes
1answer
115 views

Curl can't follow “This page should automatically redirect”

I'm attempting to automate a login for the a course webpage I have to check this semester. I know that the POST request I am making is being done correctly because I get to the intermediate redirect ...
0
votes
0answers
21 views

How to set curl -o to any desired location in AppleScript?

I have a basic question about curl -o. I just want to make a short AppleScript. does shell script "curl -f 'http://www.SOMEWEBSIT.COM/ANYIMAGE.jpg' -o ~/Desktop/ANYIMAGE.jpg" tell application ...
0
votes
1answer
112 views

How can I get more information from Curl regarding a FTP 425 “Cannot open data connection” error?

I am trying to upload a file with the following line: curl -3 --ftp-pasv --ftp-ssl-control -v -i -T ./mybigfile.gz ftp://lawrence@mecloud.xxxxxxx.co.uk:21/OnlineBackup/Lawrence/ --insecure -u ...
0
votes
1answer
58 views

Using cURL to retreive public IP for tmux config creates too many connections

I am currently using the command: curl icanhazip.com and the option set -g status-left ... in my tmux.conf to retrieve and display my public IP in status bar. This works great, although I am really ...
0
votes
0answers
52 views

Downloading an authenticated large file with command line tools

I need to download a file sized around 1.6GB. I's like to get it download via a command line tool like axel, wget or curl However, how can I download a file which needs to be logged in to the site ...
1
vote
0answers
13 views

Sporadic loss of access to local server

For some reason, I lose access to a local server called lux-app04 seemingly at random, but only in some applications. As you can see in the image below, Postman (top) can connect to it, but curl ...
0
votes
0answers
36 views

curl to resume dropped file copy

I refer to an older question in the forum: Question asked by Simon Fox My laptop froze up in the middle of a (large) file copy from my desktop (over a simple LAN). The copy operation was about ...
0
votes
2answers
516 views

curl and sni-enabled server

I am running curl against a sni-enabled server with the following command curl --cacert CustomCA.crt -H "Host: example.com" https://1.2.3.4/foo However, I am not getting the right certificate where ...
0
votes
0answers
172 views

SSL certificate error attempting to download Homebrew?

I've been trying to download and install Homebrew for my Mac (OS 10.9.4) using the install script on the home page, but the download fails with a SSL certificate error (see log below). I understand ...
0
votes
0answers
31 views

How can I get all urls of pattern x from a paged list of search results?

I'm using Windows 7 SP1, and I'm willing to use powershell or wget or whatever will work. I downloaded powershell today (ps 4) and I've got some scripts working, but I'm having trouble with this one. ...
0
votes
0answers
67 views

Curl slow download from other server

I'm trying to download large files to my personal VPS server from public video sharing site . curl -o bigfile.mp4 -A "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20120101 Firefox/29.0" "URL" ...
0
votes
0answers
42 views

curl won't shut up (--silent does nothing)

I'm writing a quick shell script, and I cannot for the life of me get curl to suppress the progress bar. The relevant line looks like this: curl http://website -o $temp >/dev/null 2>&1 ...
0
votes
0answers
133 views

Parallel download for a list of urls and renaming

I have a list of tab-separated urls, and target file names, urls_to_download.txt, for example: first_file.jpg\thttps://www.google.co.il/images/srpr/logo11w.png ...
0
votes
2answers
84 views

How to download all file with name emp* using curl from a FTP site

How can I get all files downloaded from a FTP site directory with file name like emp*. Please suggest.
0
votes
1answer
53 views

How to automatically save a list of files with different names?

I have a text file of URLs, each in the form of: http://domain.com/a/b/c/d.jpg I want to download all of these, but save each file under the name: c_d.jpg In other words, for each file, I want to ...
1
vote
1answer
184 views

Upload image to postimage.org using curl

I found a script that can upload image to postimage.org using curl. It was posted not long ago, on 2013-09-06, at http://crunchbang.org/forums/viewtopic.php?id=28867. The simplified script looks like ...
0
votes
1answer
57 views

curl bails on download in cygwin but not os x

I'm trying to download a copy of a postgresql db from Amazon S3 in cygwin. But it yields zero's across the board yielding a useless file. This is my curl command: curl `heroku.bat pgbackups:url` -o ...
0
votes
0answers
57 views

import .cer file into ubuntu for curl

I am trying to use fiddler to act as a ssl proxy for debugging. I am trying to install fiddlers root certificate. I am mainly doing it so I can debug my python program. Using python I get a cert ...
0
votes
3answers
103 views

Is there a linux filter which converts html to simple and pretty formatted text?

I would like to filter html response coming from cURL output. Which filter should I apply? Viewing html content with Midnight Commander is no problem, it shows them as pretty formatted plain text, so ...
1
vote
1answer
715 views

How to upload file with curl when --data-binary is present in tampered data?

I try to upload files to a popular website, I tampered the http POST requests with Chrome developer tools' "copy as curl" feature. When the upload begins I get this curl command: curl ...
0
votes
0answers
170 views

How can I use Wget / Curl / Plowshare to download from 1fichier.com as a premium member?

How can I download files from 1fichier.com (I am a premium member) with the help of Wget / Curl / Plowshare (Plowdown) or other CLI / command line tool / script?
0
votes
0answers
36 views

Curl over interface for port forwarding

I'm trying to set up OpenVPN and Deluge to seed torrents via the VPN. I am running other services on the server, so I only want the torrent info to go over the VPN. The way I've accomplished this so ...
1
vote
1answer
110 views

downloading multiple files using curl, curl repeats address in download

I am running the command curl -O "http://www.*site*.com/[1-9].png" but the download is failing. - the output is as follows [1/10]: http://www.*site*.com/1http://w --> 1.png % Total % ...
0
votes
0answers
47 views

crawling google links produces 400 error

I had a google scraper some time ago that quit working when they made some security changes. I just had a task come up and realized something must of changed a lot. the best I can get with any tool to ...
1
vote
2answers
91 views

file downloaded via firefox and curl have different size

When I download a file from this link by Firefox its size is 74580 B, But when I download it by curl with exactly all of header was sent by Firefox its size is 79891 B (I copied all header from ...
3
votes
1answer
339 views

curl: (7) Failed connect to www.google.com:80;

I have installed Git for Windows and got cURL automatically. C:\>curl --version curl 7.30.0 (i386-pc-win32) libcurl/7.30.0 OpenSSL/0.9.8y zlib/1.2.7 Protocols: dict file ftp ftps gopher http https ...