In this article we will look into what is meant by parallel programming; how it works internally in the CPU.
Parallel Programming - Simple Definition
Most Laptops and Desktops have multi-core processors in them. Although we have multiple processors in the machine, our normal code will run in only a single core of the processor and not utilize multiple cores of the processor in the manner they are designed to be used. Now the developers can take advantage of this robustness in multiple ways in their code, particularly when working on complex algorithms or while doing multiple tasks at the same time.
Microsoft's Visual Studio 2010 & .NET Framework 4 has provided a way to achieve this by using the Parallel Library and Parallel LINQ in it.
Normal programming will make use of only one processor without utilizing the other processors even when they are idle. But when you implement the Parallel library or Parallel LINQ, it makes use of the other processors while executing the code.
Quick Sort - Sequential Programming
Here I have taken quick sort as an example to explain how the Sequential Programming utilize the CPU and how much time it takes to complete the tasks.
Input Given
[213250, 732668, 385850, 711263 ........ 269207, 327728, 457644, 654506], 40000000 Elements
Sequential Quick Sort Result
Total Time Taken - 00:00:22.24
Output
[0, 0, 0, 0 ... 999999, 999999, 999999, 999999], 40000000 elements
Explanation
The CPU usage screenshot in my system is shown below. It's a Dual Core system, so it has two processors. In this example, the sequential programming is utilizing only one CPU at a time and the CPU usage shows only 52%. You can see the graph showing the two processor's use, one processor graph is up whereas the other processor is at the bottom of the graph. Hence it takes 22 seconds to complete the task.
Quick Sort - Parallel Programming
Here I have taken quick sort as an example to explain how the Parallel Programming utilize the CPU and how much time it takes to complete the tasks.
Input Given
[213250, 732668, 385850, 711263 ........ 269207, 327728, 457644, 654506], 40000000 Elements
Parallel Quick Sort Result
Total Time Taken - 00:00:12.73
Output
[0, 0, 0, 0 ... 999999, 999999, 999999, 999999], 40000000 elements
Explanation
The CPU usage screenshot in my system is shown below. It's a Dual Core system, so it has two processors. In this example, the parallel programming uses both CPUs at a time and the CPU usage shows 100%. You can see the graph showing the use of two processors, both the processor's graph is up to the top level. Now if you compare the time taken by two programming, the parallel programming completes the task nearly in half the time taken by sequential programming.
Points to be Noted
- Sequential Programming - CPU Usage is Less, Time taken is More
- Parallel Programming - CPU Usage is More, Time taken is Less
If the concurrent users are less, there won't be any issues in CPU usage, it can manage. But if the concurrent users are more, surely there will be an issue in CPU usage, either it requires more memory to perform all tasks at once, or there should be some other way to solve this issue.
So we can't simply change the sequential programming into parallel programming. We need to do several levels of load test and other tests to find out when to go for parallel and when to go for sequential.
In many cases, parallel programming runs slower than the sequential programming. Let's see the potential pitfalls of parallel programming in my next article.