1

Recently I started to use Debian 9 (9.4, from Debian 8.x) and a script involving curl stopped working. I connect to internet through a squid proxy on localhost connected to a parent proxy.

My environment variables are configured like this

root@server:~# printenv | grep -i proxy
HTTP_PROXY=http://127.0.0.1:3128
FTP_PROXY=http://127.0.0.1:3128
https_proxy=https://127.0.0.1:3128
http_proxy=http://127.0.0.1:3128
HTTPS_PROXY=https://127.0.0.1:3128
ftp_proxy=http://127.0.0.1:3128

When I use wget, it works:

root@server:~# wget https://www.google.com.cu
--2018-03-14 09:08:53--  https://www.google.com.cu/
Connecting to 127.0.0.1:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘index.html’

index.html                  [ <=>                          ]  11.12K  --.-KB/s    in 0.001s

2018-03-14 09:08:54 (14.9 MB/s) - ‘index.html’ saved [11389]

when I use curl, this is what I get

root@server:~# curl -v https://www.google.com.cu
* Rebuilt URL to: https://www.google.com.cu/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to (nil) (127.0.0.1) port 3128 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection:     ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
* Curl_http_done: called premature == 0
* Closing connection 0
curl: (35) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol

I know these two commands are not equivalent, this is just to illustrate the HTTPS transfer problem.

I need to use curl because the script uses a web API, so it needs to use POST instead of GET request, and to set some headers and data to the POST request. (api.dropboxapi.com is the target site)

This all used to work on Debian 8 without a hitch, and besides wget WORKS, only curl is failing with the debian version change. All the other HTTPS clients seem unaffected (FF, Chrome, Edge, wget all seems to work as always)

I know curl is a little bit picky about TLS connections, but this is a little extreme. I tested both sites (www.google.com.cu and api.dropboxapi.com) on https://www.ssllabs.com/ssltest/index.html and both comes as an "A" on overall rating)

Is there any workaround, fix, command line option or whatever for making curl work?

Output of curl -V as requested

root@server:~# curl -V
curl 7.52.1 (x86_64-pc-linux-gnu) libcurl/7.52.1 OpenSSL/1.0.2l zlib/1.2.8 libidn2/0.16 libpsl/0.17.0 (+libidn2/0.16) libssh2/1.7.0 nghttp2/1.18.1 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL
  • I suspect it's due to lack of support in your version of curl for the protocols and/or ciphers used by the remote sites. It might be useful to edit your answer to include the output of curl -V. – Anthony Geoghegan Mar 14 '18 at 14:21
0

Many, many thanks to Michael Hampton. It turns out the problem was in the proxy configuration. It should say

https_proxy=http://127.0.0.1:3128
HTTPS_PROXY=http://127.0.0.1:3128

So curl was trying to connect to squid using TLS and failing of course.

Original answer in https://serverfault.com/questions/901626/debian-version-change-affecting-scripts-using-curl-and-https

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.