Seeding thousands of torrents

Question

Is there any difference for CPU loading between seeding one big torrent (over 20Gb) and 2000 of 20mb torrents?

Answer 1

Provided your client can handle those many torrents, I would not expect the CPU usage to differ by that much, because most of the work involved is disk and network I/O. Practically, you might face difficulties in maintaining so many (thousands) network connections to other clients.

Answer 2

If you can seed is as one torrent, then it's better to do just that than creating 2000 torrents.

With 2000 torrents, you will have a lot of overhead happening:

• Your processor needs to compute more, it needs to do the extra stuff 2000 times instead of once.
  It will also need to switch between threads a lot more, again causing overhead.

• Your memory will have a lot of overhead, all the overhead data for the file maps sums up a lot.

• Your disk I/O will have a lot of overhead, caching things in memory will be less effective.

• Your network I/O will have a lot of overhead, a lot more connections than needed are prepared.
  Your computer will probably not be able to handle all these extra connections,
  or you can set less connections which would make it be ineffective.

So, either make one torrent or split it up in some torrents, but don't create 2000 them...

Please don't forget that the user downloading your torrent can select what files he wants!
They will be happier with a small amount of torrents so that they have enough connections
so that the torrent downloads faster!

Oh, another hint: If your CPU and disk are fast enough you might want to compress things first... ;-)

Answer 3

If you try to seed 2000 torrents all at once, you'll want to watch to limits in your TCP stack causing connections to fail - most kernels of any sort have some sort of limit on the number of concurrent TCP connections, if you want to try seeding 2000 torrents, you'll want to make sure you know how to adjust said limits on your system.

Answer 4

For each torrent that is seeding, your computer has to maintain

• multiple network connections - typically 5-50 in my experience
• a map of the pieces of the torrent
• a record of how far through the torrent each connected client is
• possibly other stuff ...

Doing all of that takes a little memory for a handful of torrents. For 1000 it's going to be a huge amount of memory.

Another key difference is in the total number of pieces to keep track of. The 20GB torrent will have much larger pieces than a 20MB torrent. Lets say the 20GB torrent has 1000 pieces, and the 20MB torrent has 100 pieces. Then the total number of pieces will be 1000 for the 20GB torrent, and 100,000 for the 20MB torrents. Your torrent server will use a certain amount of memory for each piece, so for this part the 20MB torrents will use 100x the memory.

If you are looking to distribute one set of things where most people are going to want all of it, then if you break it up each torrent will have to maintain network connections to every client, rather than just having one network connection to each client because there is only one torrent.

Answer 5

You'd probably have more connections to deal with with a large number of torrents, which would naturally cause more load on the system doing the seeding.