Is there a way to prioritize network speeds on networked applications?

Question

Just as there is "high priority, normal priority etc" or "niceness" for cpu time, is there a way to do something like this for a network card/internet? For example If I am downloading a file from my server over wifi and watching a youtube video, is there a way to make the youtube video a higher priority? How would this be done on windows and linux?

Answer 1

First, some backgrounder (come back to this answer after you've read this) -- on Wikipedia.

Now, you might think, "QoS" (Quality of Service) -- that's the ticket! Well, no, probably not. The problem is that QoS really should be "end to end" to be effective, yet many nodes will tend not to honor it, making it kind of useless. Also, it still doesn't solve an underlying problem which is actually causing your video lag, most likely.

Buffer bloat.

You might start reading about buffer bloat by reading up on Jim Gettys' excellent articles on his blog.

The short answer is that large HTTP file downloads that "saturate" the network (use up 100% of available bandwidth at the router level) cause the router hardware to create larger and larger buffers to store all the data. These huge buffers cause higher and higher latency to get a packet through the network, the larger your buffer is.

To see why this is, imagine that you have a network card receiving data at 100 KBps. Now, remember, every packet that comes in gets in line at the end of the queue (at the back of the line, like when getting in the lunch line at school).

If you have a buffer that's 100 KB large, it will take how long for a packet to traverse the buffer? 1 second.

If you have a buffer that's 1 MB large, it will take how long for a packet to traverse the buffer, assuming the throughput of 100 KBps as above? 10 seconds.

The problem, in all likelihood, is that the download from your file server is causing your buffers to "bloat" (grow ridiculously large) due to a poisonously harmful programming technique in almost all networking hardware in recent years, of growing the buffer to avoid packet loss.

The problem is, once the latency gets past a certain point, the TCP (Transmission Control Protocol) starts to break down. You see, TCP depends on packets arriving within a set amount of time, and then receiving an "ACK" (acknowledgment, like a message saying "OK, I got it!") from the other end. When it doesn't get an "ACK" in time, it tries to send the packet again, assuming that it has been lost. So the net effect is that, in trying to prevent packet loss, bloated buffers actually cause packet loss!!!

This packet loss sometimes then causes the firmware to grow the buffers even larger, and the problem deteriorates until the network is completely broken, and connections are dropped. Only then do the buffers shrink again and restore functionality.

You may not be aware of it, but YouTube works by streaming you small amounts of video every few seconds. It'll stream at just enough throughput to give you about 10 to 15 seconds worth of video in ~5 seconds of downloading, and then stop cold, and then start up again, ad nauseum. The problem is, if you have bloated buffers, your video will lag because the Youtube video packets can't get through your bloated buffers in time!

Controlled Delay Active Queue Management to the rescue.

You should really try to deploy Controlled Delay Active Queue Management (CoDel), if at all possible, on your router -- if you can install CeroWRT firmware on it, that would be best. OpenWRT can also be configured to use CoDel. You can also configure CoDel on a Linux computer (laptop/desktop) and route your traffic through there. CoDel is the best currently-known solution to the buffer bloat problem. Unfortunately, no implementation yet exists for Windows. Installation details can be found on bufferbloat.net.

If you try CoDel and it doesn't work, I'm betting it's either your configuration, or I'm a flying unicorn. If you try QoS and it doesn't work, well, that's perfectly normal. ;)