Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

My employer's Internet connection is extremely slow. It's a $6,000 a month, 9 Mb, bonded T1, shared by over 150 users. It's not untypical to see regular file transfer speeds of 20-100 KBps. And that was after I convinced upper management to upgrade - before it was $3,000 a month for 3Mb. There are no other carriers in the area with better prices or speeds.

When I need to download a large file (which is frequently), I drive 40 miles back home, use my $60 a month 20 Mb FiOS connection, download the file in ten minutes, and drive back in less time than it would take me to just download the file at work (this is efficient for files greater than 500 megabytes, depending on the congestion of the network).

After more than a year doing this, and more and more frequent downloads, I'm at my wits' end.

I am seriously contemplating raising carrier pigeons to fly back and forth between my home and office to transfer the files. Assuming a pigeon is carrying a 16 GB MicroSD card and can fly at a speed of about 50 miles per hour, the equivalent throughput for file transfer using trained pigeon would be about 48 Mbps. Given the price of gas where I live and wear and tear on the car, that would probably be the cheapest option too.

What are some good, viable, alternative methods of transferring large files, rather than using my employer's Internet connection?

share|improve this question

closed as not constructive by Nifle, MaQleod, Mokubai, soandos, Randolph West Aug 17 '12 at 1:18

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

113  
+1 for pigeons. IPoAC is the way to go. –  Mechanical snail Aug 3 '11 at 20:14
20  
More seriously, $6000?! Are you in some remote location? I'm surprised that that's really the only ISP in your area. –  Mechanical snail Aug 3 '11 at 20:16
10  
Have you spoken to your software vendor about getting the latest builds shipped to you overnight on a flash drive? A lot of vendors and resellers are prepared to make special arrangements, you'll probably get billed for it but dependant on distance and build frequency you may well find it comparable to your current costs. –  Robb Aug 3 '11 at 22:22
69  
Your speed estimate is for African or European pigeons? –  Diadistis Aug 4 '11 at 0:15
14  
You can easily parallelize pigeons without line installation costs! –  moala Aug 4 '11 at 9:09

24 Answers 24

Here's a crazy idea.

If you're within 100 km or so of a populated area, use an ISP in town, set up a wireless network, and make a point-to-point link using a pair of woks. More generally, see long range Wi-Fi; apparently some guys have set up a 300 km, 12-to-48 Mb/s link, using 2 satellite dishes with no special amplification.

You could check if more conventional wireless broadband or power-line internet is available in your area.

Edit: Your user profile says you're in California, so wi-fi will probably only work if you're in the plains, or if at least one site is at high altitude (so that you have line-of-sight between the endpoints).

share|improve this answer
62  
This is badass. –  benzado Aug 3 '11 at 22:02
2  
@snail: Huh? What does that (Chuck Norris router virus) have to do with it? –  benzado Aug 3 '11 at 22:13
4  
This is a applicable in the real world and fairly common. Don't forget about the curvature of the earth in an implementation like this though. This is actually where a lot of the cost in this solution lays is the site survey. –  Mike Soule Aug 4 '11 at 4:16
3  
I'll have to go next door to Boeing and ask if I can borrow a couple of their satellite dishes. –  Kyle Lowry Aug 4 '11 at 18:52
1  
@Kyle Lowry: Boeing's next door? Rent a 747 and fill it with disks. It'll be cheaper than your current plan... –  Mechanical snail Aug 4 '11 at 19:24

I'm surprised nobody here mentioned of using rsync. Yes, it still requires Internet access, but considering that the files are weekly software build then there must be lots of unchanged files that do not need to be downloaded and in the files that do change there might be lots of structural similarity where rdiff can do its magic. Your software vendor might even have rsync download server option, or they might be willing to set up one for you; or if they don't then it is relatively easy to setup rsync server at home.

Depending on the changes, rsync can easily reduce a 1GB download to 50MB download.

share|improve this answer
21  
@Kyle ssh to your home computer, download the files and rsync from there. –  Alexandre Jasmin Aug 4 '11 at 10:04
1  
@Kyle rsync shouldn't be complicated to set up in any instalation. –  woliveirajr Aug 4 '11 at 14:48
2  
@wol perhaps it shouldn't be, but setting up an rsync server on Windows is complicated by any measure. –  romkyns Aug 4 '11 at 19:37
4  
Get an old machine/laptop and put linux on it then. Ubuntu server and rsync is super easy. –  Callum Rogers Aug 4 '11 at 21:41
  • Get reimbursed for the gas, or report the mileage on your taxes.

OR

  • Are any "consumer" level internet connections available? They tend to be highly asymmetric in terms of transfer rates, so it might be worthwhile to have an inexpensive line for downloads.
share|improve this answer
3  
+1 We found a consumer option to addd a second link and cost was much lower than our 10Mbps link. Works great for downloads though. –  Dave M Aug 3 '11 at 20:55
5  
Unless you're relying on a WAN and need super high availability, I'd even downgrade back to the 3Mbps on the business-class link and use a few consumer services with a load balancer to keep things moving and to handle connection faults. Net savings in the $thousand each month plus thicker pipe? What's not to like? –  music2myear Aug 3 '11 at 21:32
3  
I've often wondered why more people don't do this. Sure, load balancing makes things much more complicated, but the money saving is almost already someone's salary. Better to just hire another guy to manage your multi-ISP network. Many companies operating using a proxy server for "security" reasons. If you already have this in place, the multi-ISP stuff is easy as pie. –  user606723 Aug 4 '11 at 15:02

Given the relative flight velocity between the two birds I would recommend the obvious upgrade to swallows and coconuts rather than pigeons and MicroSD cards. (How had nobody made that joke yet? Honestly, you can't softball it in like that and not be expected to have someone crank the sucker).

But seriously, an unlimited bandwidth 4g solution is probably the most obvious and convenient method. Correct me if I’m wrong but the only problems with that are:

1) No carrier presently offers such a plan (unless you happened to have an unlimited plan prior to Verizon or AT&T's elimination of those offerings);

2) Your post seems to indicate that you want a sans-internet solution, in which case 4g is not for you.

soandos has some helpful ideas, especially with starting the download via remote access so it is finished and ready for transfer by the time you get home. However, all of this seems remarkably after the though and reactive as opposed to proactive.

If you are aware of such large downloads, why don’t you queue them up at night before nappy time and then just bring them into work with you in the morning? It seems pretty unreasonable to think your company would be so impatient to insist on random +500MB downloads but not be willing to cooperate with you working on the files the following day (i.e. have your manager tell you what you need to be working on for the week/day/month BEFORE you come into work any given day)

That solution may seem kind of old school and certainly not as cool as a 4g wireless tether; but hey, capitalists are known for making due with high efficiency and the resources available. Want a promotion? Save your company $3000 a month in upgraded internet costs (what manager wouldn’t love that) and also help your company get a better grip on its POM (process and operations management), and I can almost give you my Cajun guarantee you will receive high marks.

share|improve this answer
10  
I hear African swallows are cheaper than many other types. –  music2myear Aug 3 '11 at 21:33
4  
There was a well publicized stunt here in South Africa a few years ago where a carrier pigeon was pitted against an ADSL connection provided by the incumbent telco. The pigeon won, by far. It was of course just a publicity stunt, but nevertheless it goes to show that TCP-over-carrier-pigeon is faster here than standard DSL. Link to article: news24.com/SciTech/News/Pigeon-beats-Telkom-20090909 –  Richard Keller Aug 3 '11 at 23:01

Looking at you question, it seems to me that the problem isn't you having to drive home, but the fact that your T1 bonding isn't working correctly (I'm going to skip over the fact that you being reamed on the pricing) or that you have a QoS policy in place that is limiting your HTTP download speed.

100 KBps is 800 kbps which as a max speed on a line shared with 150 uses is actually quite good, for a single T1 (~1.5 mbps). Most providers/equipment only allow you to bond 4 T1's at once giving you a theoretical maximum bandwidth of 6 mbps (1.5*4), although if they sold you a 9 mbps line then you probably are lucky enough to have a provider that allows up to 8 bonds with a theoretical max of 12 mbps (1.5*8).

My suggestion would be to figure out why your bonding isn't working and fix that. (If you need help feel free to ask over at Server Fault).

share|improve this answer
10  
... and if the answer is that the problem is everyone else in the office listening to Pandora, etc., then the solution is to set up QoS—prioritize business-critical traffic. –  derobert Aug 4 '11 at 17:50
1  
that's exactly the answer: some traffic shaping / prioritization. Dead easy to do with squid. You're using a proxy cache, right? –  Javier Aug 7 '11 at 0:56

Have you explored satellite and wireless mesh options - maybe there's one in your area if the local xDSL services are not too hot.

share|improve this answer
8  
+1 for satellite: horrible latency but decent throughput is fine for large files. –  Mechanical snail Aug 3 '11 at 20:23

I don't know what you are hoping for, but you have the two methods of getting the files: Download them, no matter how slow, or transport them in.

All I can say is be clairvoyant, and know what files you might need to download before you go to work, or start downloading common large files you use, like service packs and media, and burn them to disk. Make a list.

Also, if you are using Microsoft products, consider a Software Update Server for Windows Updates, and a proxy server to cache Internet traffic, to minimize your Internet access.

Lastly, make sure your firewall is locked down to block major streaming music sites and Facebook-type bandwidth wasters. And because streaming over HTTP is a bit harder block without a monitoring software, have management implement a strong policy regarding the Internet and bandwidth.

share|improve this answer
3  
+1 for blocking bandwidth-wasting sites - if the work internet connection is struggling, it should be restricted to work-only purposes. –  jwaddell Aug 3 '11 at 23:13

Is 4G service available around your work? That may be an easy way to add a relatively high-bandwidth internet-only connection.

If your bosses are antsy about having an unfiltered internet connection, use a sneakernet isolation system where the computer connected to the wireless is not on the domain and you have to use thumbdrives to get the files from that computer to your own. The costs of such a system could be easily balanced by the alternative of you driving home and back, especially if you start speaking forcefully to your bosses about their assumption you'll use your own economic resources to accomplish what is necessarily their responsibility to make sure you're capable of.

share|improve this answer

Obligatory xkcd:

Every time you email a file to yourself so you can pull it up on your friend's laptop, Tim Berners-Lee sheds a single tear.

share|improve this answer

if it's for business purposes, and to/from specific places (customers, suppliers, etc), I've known companies to buy rugged external drives and use FedEx/UPS to ship data back & forth. You may be able to run some numbers and show your management that's actually cheaper than tying up your computer (and the rest of your co-worker's network).

share|improve this answer
1  
Yeah, but unless the OP has someone else at home to get the file onto some sort of portable media, who's going to hand the drive to the FedEx/UPS at his house? –  afrazier Aug 3 '11 at 20:10
1  
I was meaning to take the trip home completely out of the process. The companies I was referring to, for example, had to transfer large video files to/from customers. The files were large enough that, no matter how fast their network was, shipping a drive from their office to their clients' was always faster. –  StevenV Aug 3 '11 at 20:28

As other have suggested, you have several options to set up a long range connection from an area with better rates to your area. You can do it using satellite dishes or microwave connections or even cable connections, you should really check the cost for those options.

While checking those options, consider this: what if you work together with other companies in the same district to invest in those options? They are probably facing the same issues your as company.
If everything works out you could become the "BRINGER OF INTERNET AT OK PRICES" (a title worse than "the bringer of doom" but a good title nonetheless).

Legal issues for sharing internet connections among multiple companies may apply.

share|improve this answer

Expanding on what @Zypher answered, if the problem is network congestion from the 150 users, and you can't schedule the download for off hours, then you should definitely look into QoS. You could prioritize your download traffic over all other network traffic.

You should look into exactly what is causing the problem. You may be able to fix the problem with some simple reconfiguration of your network.

share|improve this answer

I don't know of the pricing involve, but a microwave link could span the distance between your home (where internet access is cheap and fast) and to where your employer is (slow and expensive.) It's all line-of-sight, but you can get full T-1 speeds easily.

I'm not suggesting connecting your home to your business, but you may be able to set up a microwave link between a ISP in the area in which you live to your business.

At $6,000 a month, you would not have to go very long before reaching break-even. A quick search of "microwave link pricing" yielded this. Google shopping (not definitive, I know) shows the most expensive pricing to be about $28K, which has a break-even in about five months.

Let me know if you want more information, and I can contact some folks I know who maintain a rather extensive microwave link relay (secondary emergency data communications for first responders and such)

share|improve this answer

In a documentary on some SE Asian mudhole, motorcycle riders delivered email by having a wifi-equipped PC in the motorcycle trunk with a mobile whip wifi antenna and driving back and forth between urban and rural areas.

Computers along the route would recognize the availability of the signal to get off a few packets as the cyclist drives by. For short messages, this could be sufficient but big files require some waiting time.

At $10/hour ~ $1600/month, your company could enhance its internet service with a wireless N motorcyclist. For the cost of an extra used motorcycle he can switch cycles so that one is always exchanging data with the work site and one is always enroute.

share|improve this answer
1  
Hehe... the "SE Asian mudhole" you are referring to is Cambodia. They did it here some years ago. –  snap Aug 4 '11 at 14:28

There are really two problems here:

  1. How do you get the file from the Internet to a device you own?
  2. How do you get it to your employer?

It would be faster to start downloading it via remote access, so that it is ready to go when you get home. This still leaves the time it takes as total round trip time though, but seems to be the fastest you can do it without spending more money.

If you are willing to spend some money, there might be a 3G or 4G solution out there that you could use for this. Beware that there are generally caps, slow downs after a certain point, terms of use etc that are meant to stop people from using that much bandwidth.

share|improve this answer
1  
Yeah - I have remote access set up. So I remote into a computer at home, and start the download before I leave work. In most cases, the file is done downloading before I even get past the surface streets and onto the freeway. –  Kyle Lowry Aug 3 '11 at 20:09
1  
@Kyle, So then rsync from home to work. –  TRiG Aug 4 '11 at 10:03
1  
@TRiG - That's such an obvious solution, I never even considered it before. I'll do some tests and see how that works out. fingers crossed –  Kyle Lowry Aug 4 '11 at 18:58

You could download the file to a computer that you use remotely, via Windows Terminal Services, or VNC, or some other screen sharing technology. All you need is the bandwidth to remotely access the computer to do your work.

Multiple people in your office could log into the remote computer at the same time. In other words, figure out how much you could move to a remote server; just because you are in your office doesn't mean your computers need to be.

It could be a co-located machine at an ISP, or maybe even a virtual machine on Amazon's EC2.

share|improve this answer

Perhaps part of the problem isn't available bandwidth, but latency over long distance connections to India. I ran a studio in the US, using a post production facility in Mumbai, and while we had a fat pipe in our facility, and they had nearly as much bandwidth in theirs, FTP connections were horidly slow. After looking at a number of options - and paying through the nose to UPS in the meantime, killing our schedules - we switched from FTP to a file transfer accelerator product - we used FileCatalyst (don't mean to shill for one product), but there are several others and all were significantly faster than FTP on the same pipe, under the same conditions. All are UDP-based, have built in compression, validation, and reliable pause/resume.

share|improve this answer

Are there any areas nearby that have better internet options? Perhaps you could come to some arrangement with a business in that area to download your files for you, and they would be closer (and cheaper) than going home?

share|improve this answer

Plenty of ideas here, but I'll throw in my .02.

Write up an agreement with your employer and see if you can resell your FIOS connection (or make an arrangement to do so as a business customer.) Have fiber connected from your home to your employer.

Or find another business that will do this (share their higher-speed connection to your company).

I work for an organization that is spread out over the county in our state. To connect all the buildings we pay for fiber connections that run to each of the buildings, so we have one Internet connection and share it to all of our buildings bridged via fiber. High speed, and all we pay for is renting the previously dark fiber and the poles on which it's run. Depending on how far away you are, it may be cheaper than what you're paying and you'll get a significantly faster speed, but you'll have to run the numbers and check what it'll cost for you to get a business class connection that can be resold and you'll have to deal with the phone company...not usually a fun proposition. This may be a drastic option (and you'd need to take some precautions to separate this from your work life, since you don't know what the future will hold) but you'll have really fast speeds and you may be able to monitor systems at work from home. If that's a bonus. I don't know.

share|improve this answer

Can you get such a $60 a month 20 Mb FiOS connection at work too? For 6000$ you should get 100 of them.

share|improve this answer
1  
Unfortunately Verizon doesn't service this area. –  Kyle Lowry Aug 4 '11 at 18:56
1  
Just use 3g/4g hotspot device. Or if you have a smart phone, use that (Then write it off as a work related expense). –  Steve Robbins Aug 5 '11 at 23:23

Big question. So many votes! But, I'll chip in as well!

My solution is simply: remote desktop/other remote access application to your home computer. You could work with the files, read the files, and even print the files if you needed to. If you needed to have the files on the network for whatever reason, you could just do it at the end of the day, keep a log of what files need to be transferred and throw them on a portable storage. The only problems in that are:

  1. It may be choppy working remotely working with your computer at home. The image might not be the resolution you are hoping for.
  2. If you need the file on the network right now.

If that works for you, then awesome! Good luck. =)

share|improve this answer

What I would do is get one of those Verizon cards that gets internet through the cell network, and plug it into your LAN.
NS about price, but it's definetly cheaper than $6k

share|improve this answer

You can install a second asymmetric 20 Mbps connection only for you.

Can you use a download manager ?

share|improve this answer

1 . Ask your admin to make tomorrow downloads at night ( when bandwidth is free )

2 . if files are necessary for your job then stop working and ask for dedicated bandwidth

3 . if the bandwidth is shared ( not limited per user ) use download managers with as many possible connections

4 . Send link of this page to your boss

5 . i want FIOS too

share|improve this answer

protected by random Aug 4 '11 at 1:50

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.