0

How might I find out the maximum number of threads I can use? I'm new to programming, so I'm trying to figure out what I need to know. I have 950 GB of files to download from Amazon S3. I was thinking I could use two threads that each download half of that amount, running in parallel.

I'm hoping to speed up that bulk data download from Amazon S3 by multithreading my application, but it would be good to first know if my computer even supports multithreading.

Computer specs:

  • Model: MacBook (13-inch, Mid 2010, A1342)
  • Processor: 2.4 GHz Intel Core Duo
  • Memory: 8 GB 1067 MHz DDR3
  • Software: MacOS Sierra 10.12.6

The processor is very old and has been discontinued, so it doesn't support Intel hyperthreading. I'm not sure if it's any good with multithreading. But if it can support multithreading, how many threads can it support, and would there be an impact on performance if I use too many threads?

Please correct me if I am wrong, but I read that a non-hyperthreading core can support only 1 thread, so I'm thinking maybe my processor can support up to 2 threads, then any more threads after that would have negligible effect.

  • 1
    Sounds like you need to read up on Multitasking. It’s very important to tell I/O-bound work from CPU-bound work. – Daniel B Jun 10 '18 at 18:10
  • What exactly do you mean when you say, “I'm hoping to speed up some bulk data download from Amazon S3…” What is this? Are you downloading files? Or something else? Amazon S3 is just a file storage setup. I’m not too sure how multithreading would help a bandwidth-bound task like this. – JakeGould Jun 10 '18 at 18:17
  • I'm new to programming, so I'm trying to figure out what I need to know. I have 950 GB of files to download from Amazon S3. I was thinking I could use two threads that each download half of that amount, running in parallel. @JakeGould – briennakh Jun 10 '18 at 18:19
  • 1
    @briennakh As stated in this answer, downloading files is not a CPU intensive task by any measure. You can launch multiple processes that can attempt to download more than one file in parallel, but it won’t be a CPU bound task. It’s all dependent on bandwidth and your network connection. – JakeGould Jun 10 '18 at 18:23
  • Ahhh, ok. Thank you, I appreciate your time to answer my noob question. The reason why I had thought this would work is because of this recommendation: boto3.readthedocs.io/en/latest/guide/… – briennakh Jun 10 '18 at 18:23
2

There are several false assumptions in your question so let me clarify here:

  1. Multithreading is possible even on single-core CPUs. The execution just splits in time so you have 1 thread active in any given moment, however the overall threads execution is parallel.
  2. Multithreading is not equivalent to speed-up. If naively designed, multithreading app is always slower than single-threaded app by tens or even hundreds of times.
  3. Downloading from outer networks in 99% of cases is not depending to multithreading. It may be worth to take a look to multi-range HTTP downloading like it was implemented in apps like NetAnts, ReGet and others from dial-up modem era. However, there is no warranty that multi-range downloading could be faster that regular one, due various network policies and involved server(s).

So, all after all. You definitely have to collect metrics in your app: What is slowest operation? And then re-design it to get more speed.

1

How might I find out the maximum number of threads I can use?

Intel Core Duo has 2 threads due to it having a 2 core single thread processor. Each core has the capability of a single thread.

But if it can support multithreading, how many threads can it support, and would there be an impact on performance if I use too many threads?

It supports 2 threads.

so I'm thinking maybe my processor can support up to 2 threads, then any more threads after that would have negligible effect.

It’s incapable of handling more than 2 threads. Your multi-threaded application would only have two threads since you can only generate 2 threads due to the hardware.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.