3

Upload a large file via broadband connection takes time. Compress the file using max. compression ratio before upload also consume time.

Is there any solution that may compress and upload file to remote end at the same time? If it does, the compression time is embedded in uploading time.

For example, compress the file takes 5 minutes and upload the compress file takes 10 minutes. It perform the compress followed by upload task, it takes about 15 minutes.

If using a solution that compress and upload, it may take about 10 minutes only as uploading speed are usually slower than compressing.

  • What are you uploading to (a web server?) and what are you using to upload (a form in a browser?) – Paul Jul 3 '14 at 9:58
  • Server is web server if possible. Client could be home grown. – Chau Chee Yang Jul 3 '14 at 10:10
3

It depends on the protocol you are using to transfer files. Usually it is already done transparently.

HTTP (and HTTPS) protocol supports on-the-fly compression, provided that both server and client implements it. First of all, client will let server knows which type of compression it supports. Typical HTTP request header includes a line like

Accept-Encoding: gzip, deflate

Server will pick a compression algorithm it supports, indicate in the HTTP header (as below) and serve the contents compressed.

Content-Encoding: gzip

Most modern HTTP clients (like Chrome, Firefox, Internet Explorer, Safari etc, in alphabetical order) and servers (like Apache, Microsoft IIS, nginx) when configured properly, are transferring data compressed on-the-fly.

Similarly other protocols may implement compression, e.g. Mode Z in FTP. Specialized protocols such as Remote Differential Compression by Microsoft allows synchronizing local and remote files by transferring the different portion only.


EDIT:

I found that the transparent HTTP compression is for file download only but not the other way round. If you wish to compress data for upload, you may need to write some Javascript, and make use of FileReader and compression library like compressjs.

  • Yes. I should mention On-The-Fly... – Chau Chee Yang Jul 3 '14 at 10:10

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.