I need to use cURL to upload some files to an SFTP server, it has to be using cURL because I'll be later using cURL_multi etc to transfer files simultaneously (unless anyone knows a command line library which supports FTP, SFTP using password, SFTP using key, and supports up to 10 simultaneous transfers).

The problem I have is cURL is running painfully slowly. Uploading using the standard SFTP client on Ubuntu 12.04, I'm getting around 70Kb/s (this is fine, that's just the speed of the server), but when using cURL, it's around 35Kb/s, half the speed of SFTP :/

Does anyone know of any ways I can speed up cURL, preferably without having to recompile cURL, but I will if I need to.

Here's the command for SFTP:

/usr/bin/sftp -oProtocol=2 -oIdentityFile=/path/to/key.pri -oPort=5000 -oStrictHostKeyChecking=no username@thedomain.com

then just put filename.wav when it's connected...

and with cURL:

curl -v -u username: --key /path/to/key.pri --pubkey /path/to/key.pub -T /path/to/filename.wav sftp://thedomain.com:5000/folder/TEST/filename.wav --insecure

This is the cURL version info:

curl 7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/ libidn/1.23 libssh2/1.2.8 librtmp/2.3 Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtmp rtsp scp sftp smtp smtps telnet tftp Features: GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

Surely it can't be this slow for everyone?


This is unfortunately a known issue with curl. It seems it can be mitigated by recompiling with a larger CURL_MAX_WRITE_SIZE; see this entry from their mailing list.

Since you're trying to achieve multi upload in the end, you may still succeed in filling your outgoing pipe with SFTP over curl, which would give you the best possible throughput anyway :)

  • The problem we've got is there's an added complexity of using it with PHP... if I have to recompile cURL, then I think I also have to recompile PHP, and I just can't risk that when it's being used on a live production site! – BT643 May 24 '13 at 10:34
  • The way we need to send the files doesn't fully utilise the "multi" nature either... we have to send a block of files at a time, and can only send more when they've finished, sometimes this means we can only send one at a time, and other times we can send 10 at once :( think I'll have to use a different method :( – BT643 May 24 '13 at 10:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.