When I look at task manager I cannot find any way to show me which cores are being used by which applications. Is there any way to see that? Or is there some util out there which can show me that kind of usage over time?
The reason you cannot see what core is being used by which application is because this can potentially change hundreds of times a second. It is information that is almost certainly invalid the moment it is printed.
The information is also not worth logging because the cost of logging that kind of data is exorbitant compared to any benefit you may get from it. Memory costs for that sort of logging would be huge over even a short amount of time, and there is no real benefit besides curiosity, it has no lasting value.
While Windows may give preference to not switching an application to another core while it is running if that application goes to sleep (however briefly) then it will be unscheduled and rescheduled on another core when it wakes up again.
You can limit a program to a particular core using the
Set task affinity option when you right click on it in task manager, but that is not the same as what you are asking.