MultiThreading - making use of 100% of cpu?

Question

Visual Studio 2008
.net frmaework ver 3.5
windows server 2008
48 amd core processors
32 GB RAm

I am having problems running my multithreaded application. the application only takes advantage of between 30-40% of my processors capability. I ran visual studio profiler to try to identify what the problem is. I found there was a lot of contention from mscorwks.dll. This seems to be responsable for garbage collection. Each thread is designed to pretty much work in isolation only reporting progress as it goes. i am inheriting from BackroundWorker for use on my worker threads. i have tried all combinations of GCSettings.LatencyMode, And i tried using server instead of workstation mode, and i tried running each worker thread as a single threaded apartment(STA), but these efforts didn't make much difference.

My temporary solution has been to open up multiple instances of my application(running 4 threads each), so they run in seperate processes. This works quite well and takes advantage of the servers full processing power. Anyone got any ideas of how i can acheive the same thing on with one application instance?

Sam Hobbs · Answer

I would be satisfied with 90% utilization of 48 processors. Are you able to determine if more work is done over-all if you use 4 processes instead of 1? I admit that I don't know enough to know what the efficiencies and innefficiencies are but the overhead required by multiple processes might be greater than the benefits.

Darren Scott · Answer

Hi thanks for your reply. Currently i am running 12 processes each with 4 threads(12 *4). I chose that number because i have 48 cores. I am now making use of around 90% of the processors. So it something about running all the threads in one process that is causing problems. When I try to run the 48 threads in one process there seems to be a rediculous amount of pausing threads while waiting for other threads. I am not sure why this is happening im pretty sure all the code is thread safe. the profiler suggests the contention is being caused by mscorwks.dll. I checked this out online and it seems to be responsable for garbage collection. its strange that everything runs ok when i split up the threads into more processes. can't seem to figure out why I can't do the same in one process. any thoughts much appreciated?

Sam Hobbs · Answer

I am confused. You say  there was a lot of contention from mscorwks.dll  ....  garbage collection  yet you say  only takes advantage of between 30-40% of my processors capability . Garbage collection uses the processor so it is not relevant to the amount of time that the processor is not used. I don't know how to use the profiler to do it but you need to determine what the processor is waiting for such as disk IO. If each thread is accessing various areas of the disk then the drive might need to do many seeks which is costly. Another possibility is virtual storage thrashing; main memory is cheap enough that most systems have ample main memory but if your application is using more main memory than the system has real memory available for use then that could be a costly bottleneck. You need to look for things like that.