I live in a rural location, using high-latency wireless off a local ISP's tower. My speed tests vary day to day, but I can get around 1Mb up/down. The problem is, I work with large files, uploading and downloading (HD videos, development software, etc.). It can be painful to wait sometimes. Plus I do some side contract game development, and it can be very difficult to playtest with other developers (200ms ping is a good day for me).

Now, obviously it's not going to be easy to solve the latency problem without different wireless hardware. But speedwise, I am wondering if I can use some kind of compression technology on a proxy.

For instance, my work computer has full access to a 26Mb down, 10Mb up connection, that is totally unused at night and the weekends. If I could run some kind of compression technology on our server, and use it as a proxy to route to my home computer, I could stand to gain some major speed. I realize that by bogging down a system with compression, I could potentially lose whatever speed gain I had. But the proxy server is a quad core xeon, and the receiving computer is a pretty decent i7 computer, so that shouldn't be a concern.

I found http://toonel.net/ but it seems more geared toward very slow narrowband users, like dial-up. Plus, I would prefer to just be able to point my browser to a proxy server, rather then install software on my client machine.

EDIT

I thought about my question a little more, and realize I am going to need to install software on my client in order to decompress, and possible compress (for uploading). That's not a huge deal.

Your best bet would be to set up a VPN between your home computer and work that uses compression.

Most client/server based VPN solutions use some form of compression (mainly because they are based around PPP which has rudimentary compression). PPTP is probably the easiest to set up as it has native support in Windows.

For extreme compression the best I have found is a Unix / Linux system called vtund which can use a number of different encryption systems, including gzip at compression level 9 ;) Obviously that would require Linux or Unix at both ends of the link.

  • Now how would that work? I would setup a VPN, and just route my client to get it's internet via the VPN "server" address? – user23150 Mar 7 '11 at 15:11
  • @user23150 yes, that's how it would work, if the VPN endpoint is set to give you Internet access. If it's not, you'll just get access to the LAN at the endpoint. This could cause problems with delays as data has to go from you to the VPN endpoint first and then to wherever it's supposed to go. The best way to see if you'll have problems is to try it out, since a lot of things depend on your connection. – AndrejaKo Mar 7 '11 at 17:19
  • xz at lower levels is actually faster than gzip afaik – Ramchandra Apte Oct 24 '13 at 3:58

video: if these are already compressed with Theora or H.264 like most video files, it's probably not worth trying to compress them again.

Web browsing: If you have a web browser written after 1998, most web sites will transparently compress stuff before sending it and your web browser will transparently decompress it. It's probably not worth trying to compress it again.

Software development: Source code and executable binaries can be compressed a lot. However, rather than use a tool that compresses an entire directory and sends it, often it takes even less time to either

  • Use a tool like rsync that analyzes the previous version of files at the destination, and only sends the changes. Or
  • Use a revision control tool that keeps a copy of the previous version locally, and likewise only sends the changes.

Unison is a nice graphical front-end to rsync. I've used Unison on a Windows box to talk to rsync on a Linux file server in another city to synchronize a folder at both ends.

  • @N-ate: "HTTP compression ... virtually all web servers rely upon it to improve data transmission speeds for users." -- en.wikipedia.org/wiki/HTTP_compression – David Cary Apr 21 at 0:57
  • 1
    Sorry. It looks like I misread your comment. I thought you had said that the web browser does the compression. :) – N-ate Apr 22 at 22:13

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.