I'm using Spyder IDE for Data Analysis using Python. My dataset is pretty large and hence I wish to give it maximum priority.
I've set the priority to realtime however it is only using 13-15% of CPU. How can I give 100% CPU usage to it? I'm using Dell Insiron 15Z ultrabook with 2 RAMs of 4 GB each.
If nothing else is running and its using only a small percentage of the CPU, then that's telling you that it's not CPU-bound: something else is limiting it.
My first guess would be filesystem access is limiting it so it spends a lot of its time waiting to read the data.
I see from your screenshot you are using Task Manager to inspect the usage of this process. A better way to analyze this situation would be with Resource Monitor:
Open Resource Monitor by clicking the Start button. In the search box, type Resource Monitor, and then, in the list of results, click Resource Monitor. Administrator permission required If you're prompted for an administrator password or confirmation, type the password or provide confirmation.
From there you can select the process you want to monitor and click to the different tabs to see the Disk, Network, or CPU activity for the specific process you have selected.
This allows you to see get a better idea of the resources used by your process and you may be able to determine if it is the disk or network causing the slow down.
Most probably it's the waiting for I/O that's holding you. Network access is slower, than accessing local storage, SSDs are faster than HDDs, RAM is faster than SSD.
If the dataset is really large and produces large result - you may want to read from one drive and write to another, so that there's less seeking.
Other thing to look for - is context switches, for example your program may have excessive logging and after each operation it has to wait to switch to OS to pass some string to terminal emulator, which renders some string of text that you're not going to read anyway.
