Your CPU meter shows a problem. One core is running at 100 percent, but all the other cores are idle. Your application is CPU-bound, but you are using only a fraction of the computing power of your multicore system. Is there a way to get better performance?
The answer, in a nutshell, is parallel programming. Where you once would have written the kind of sequential code that is familiar to all programmers, you now find that this no longer meets your performance goals. To use your system’s CPU resources efficiently, you need to split your application into pieces that can run at the same time. Of course, this is easier said than done. Parallel programming has a reputation for being the domain of experts and a minefield of subtle, hard-to-reproduce software defects. Everyone seems to have a favorite story about a parallel program that did not behave as expected because of a mysterious bug.
These stories should inspire a healthy respect for the difficulty of the problems you will face in writing your own parallel programs. Fortunately, help has arrived. The Parallel Patterns Library (PPL) and the Asynchronous Agents Library introduce a new programming model for parallelism that significantly simplifies the job. Behind the scenes are sophisticated algorithms that dynamically distribute computations on multicore architectures. In addition, Microsoft® Visual Studio® 2010 developmentsystem includes debugging and analysis tools to support the new parallel programming model.
Proven design patterns are another source of help. This guide introduces you to the most important and frequently used patterns of parallel programming and provides executable code samples for them, using PPL. When thinking about where to begin, a good place to start is to review the patterns in this book. See if your problem has any attributes that match the six patterns presented in the following chapters. If it does, delve more deeply into the relevant pattern or patterns and study the sample code.
Chapter 1 Introduction
The Importance of Potential Parallelism
Decomposition, Coordination, and Scalable Sharing
Selecting the Right Pattern
A Word about Terminology
The Limits of Parallelism
A Few Tips
Exercises
For More Information
Chapter 2 Parallel Loops
The Basics
An Example
Variations
Anti-Patterns
Related Patterns
Exercises
Further Reading
Chapter 3 Parallel Tasks
The Basics
An Example
Variations
Anti-Patterns
Design Notes
Exercises
Further Reading
Chapter 4 Parallel Aggregation
The Basics
An Example
Variations
Design Notes
Related Patterns
Exercises
Further Reading
Chapter 5 Futures
The Basics
Example: The Adatum Financial Dashboard
Variations
Design Notes
Related Patterns
Exercises
Chapter 6 Dynamic Task Parallelism
The Basics
An Example
Variations
Exercises
Further Reading
Chapter 7 Pipelines
Types of Messaging Blocks
The Basics
An Example
Variations
Anti-Patterns
Design Notes
Related Patterns
Exercises
Further Reading
Appendix The Task Scheduler and Resource Manager
Resource Manager
Kinds of Tasks
Task Schedulers
Anti-Patterns
References
Appendix Debugging and Profiling Parallel Applications
Colin Campbell is a coauthor of Model-Based Software Testing and Analysis in C# and he has written several academic papers on mathematically rigorous approaches to software analysis. He is a founder and principal at Modeled Computation LLC, in Seattle.
Ade Miller is currently a Principal Program Manager for Dryad and DryadLINQ a set of technologies that support data-intensive computing applications running on a Windows HPC Server 2008 R2 cluster. Previously he was the Development Lead for Microsoft’s patterns & practices group (p&p) where he managed p&p’s agile development teams. His primary interests are parallel computing and in engineering leadership to improve the way people develop software.
He is one of the authors of Parallel Programming with Microsoft .NET and Parallel Programming with Microsoft Visual C++. Ade also writes and speaksabout parallel computing and his experiences with agile software development at Microsoft and elsewhere.
Prior to leading the p&p development team Ade lead the development of the p&p Web Services Software Factory. He was a developer and then a Development Lead on Visual Studio Tools for Office 2005 and 2008. Before joining Microsoft Ade worked on a variety of interesting projects including a web start-up, embedded languages, and High Performance Computing (HPC). Ade received his BSc and PhD in Physics from the University of Southampton, UK.
