# Why is distributing random pieces of a file faster than a contiguous transfer?

Why is it said that BitTorrent is faster, partly, because it transfers random pieces of a file instead of transferring it from start to finish in a contiguous/linear fashion?

EDIT: But why do they have to be 'random'? Why not a more predictable 'semi-contiguous' transfer in which one peer gives you the first half, and the second peer the other half? This would theoretically reduce overhead.

-

Regarding the edit asking why they have to be random: Let's say that torrents were changed to only download the first block that the downloader doesn't already have, thus guaranteeing a contiguous transfer.

I then share a 100MB file. 10 people connect and each downloads 50MB before I go offline.

These people are then screwed, because they all have the first half of the file, while only I have the second half. Until I make it available again, they can't complete the download.

Instead, with the blocks being transferred in a random order as is actually the case, each of these 10 people has a different set of blocks, so they can trade among themselves in order to complete their downloads without needing me to be around.

Even better, though, because each of them has a different set of blocks, each of them can download from all of the others to fill in the gaps in their copy. Your edit alluded to the idea of starting from a random point in the file and downloading contiguously from that point, but that would increase the amount of overlap from one downloader's copy to the next (if Alice starts at the beginning of the file and Bob starts 10% into it, then there's 40% of the file that they both have and only 10% unique to each of them that they can trade without me reconnecting to make the rest available). Downloading in a random order maximizes the uniqueness of each downloader's set of blocks held, thus maximizing the ability of downloaders to start trading among themselves and maximizing the odds of it being possible for them to assemble a complete copy if no seeds are available.

A single contiguous download may (or may not, depending on conditions) be better for you, but downloading in random order is better for the network as a whole.

-
"a random order maximizes the uniqueness of each set" -- precisely. – eternalmatt Sep 13 '11 at 20:36

The key is that you are getting the file from several sources (peers) at the same time rather than from one source.

If you have a file that's split into 10 parts and takes 100 seconds to download from 1 server then theoretically you can get the file from 10 peers in 10 seconds (plus some overheads) as each peer can deliver it's part in 10 seconds. In practice it will take longer as each peer won't be perfect, and you'll probably hit your bandwidth limit (but you get the idea).

As you have to reassemble the file anyway you don't have to fetch the pieces in order. So if you only have 5 peers with each delivering 2 parts these can be random parts of the file.

-
Simple. Clear. +1 – Xavierjazz Sep 4 '11 at 19:12
"If 6 men can dig 6 holes in 6 minutes, how long does it take 10 men to dig 10 holes?" – Martin Sep 4 '11 at 21:29
@martin depends on the size of the holes (the chunk size) and available equipement (bandwith) :) – ratchet freak Sep 4 '11 at 21:31

It should be said that this depends entirely on the file in question, amount of peers and speed of the non bittorent source.

You will never download faster than the maximum download bandwidth of your connection, and if you are downloading from a place with enough bandwidth, it will be faster than using Bittorrent.

However, as internet speeds are getting faster and faster, we are getting to the point where (smaller) websites and hosts can not keep up.

For example, when I was looking around at colocation prices up in London a few months ago, I was quoted absolutely terrible prices for a 5Mb connection. If I was to take this, it would serve the majority of people well.

However, on the same line, if I was to host a few 500Mb files, and everyone had Cable (50Mb) or fast ADSL2+ (24Mb) lines, you would notice that my server would not be able to serve up speed that fast to you...

... however, if I was to offer you the same file through Bittorrent and there were 200 people, each giving just 30Kb/s, that would equal 5.8Mb/s (and, many people have much faster upload speed than that!).... and now, if I was to run Bittorrent on my server and offer the same file, it would mean that there is a total of 10.8Mb/s downloadable - much more than I could provide through that 5Mb/s pipe on my own.

Bittorrent is very much a numbers game, you have to have enough people uploading with enough bandwidth... Because of extra steps involved such as integrity checking (and the fact that you should upload back), it is hard for it to ever beat direct downloading from a good site with enough bandwidth, but, for many smaller sites - it is brilliant or even larger sites who simply want to save money off their bandwidth bill.

-
not to mention that upload and download bandwidth in standard cable connections are not the same (the A in ADSL) – ratchet freak Sep 4 '11 at 21:34
Weird colo bandwidth rule, it sounds like to me. Here in Colorado it seems most colos offer 100 Mbps connections and charge you by usage each month, and it is up to you to manage and limit your own connection speed if you need to keep the cost down. – Zan Lynx Sep 4 '11 at 23:18
Another comment: I prefer downloading from Bittorrent when possible because of the integrity checking. It is incredibly painful to be forced to redownload an entire DVD ISO because the SHA1 does not match. Bittorrent checks each block so its only a small chunk to replace. – Zan Lynx Sep 4 '11 at 23:20
@Zan Lynx - Ouch! I have only had that problem once in the past 5 ish years... and before then, it was mainly when I had bad memory silently corrupting things, I can understand, but, haven't seen it for many years.... Anyway, I have been meaning to look for colocation and hosting abroad from some time - England is very expensive for pretty much everything :( – William Hilsum Sep 4 '11 at 23:26

Most responses don't seem to be answering your question.

BitTorrent is not faster.

It's actually slower because of the overhead of connecting to multiple sources.

The real difference in practice however, is you are downloading from many sources that can potentially add up to be faster than what a website could give you. Many web servers are under heavy loads or are otherwise speed-limited so you don't suck them dry.

The fact that BitTorrent transfers the file in a seemingly random order has no direct affect on its speed.

In regards to your edit, the pieces are not random! They are prioritized based on rarity: a rare piece needs to become less rare as fast as possible so it is not lost if the source that has it suddenly leaves. It has to do with availability, not speed.

-

It's an availability issue. If you have to download contiguously, then you're dependent on people hosting more of the file than you. Every person with less of the file than you, will be unable to send you any data.

If chunks are randomly distributed, the number of hosts you can download from will be higher, therefore you'll download the file faster.

-

Torrents aren't always faster, but when they are, one reason is network entropy. A connection of any duration (the download of a large file) may degrade over time, as random, uncontrollable factors affect the transmission of packets, such as router overloading and other problems between you and the server. More errors mean more retransmissions of packets, and this situation tends to worsen over time. Torrents make many smaller connections, across many routing pathways, a strategy that can, under the right circumstances reduce errors and lost or damaged packets, and seems to be faster, especially as the number of seeds and peers increase.

-

Say you send two people to the grocery store to pick up a list of items. You give them each the exact same list. However, one of them must pick up the items in the exact order they appear on the list. The other may pick up the items in any order they please. Which do you think will finish first?

-
What if the items were all arranged in order? (Defragmented?) – Mateen Ulhaq Sep 5 '11 at 6:18
Arranged in what order? In alphabetical order? Unless they just happened to be arranged exactly the way the supermarket was arranged, it wouldn't help. In the analogy, the "supermarket" is the block lists of various uploaders. (If everyone started from the beginning, then two people who each had 33% of the file couldn't help each other, since they'd have exactly the same blocks. Yuck.) – David Schwartz Sep 5 '11 at 6:41