1. Designing parallel programs is harder than designing
sequential programs.
2. We start our design by using the best sequential program
we know and then we must solve the following additional problems
a. Application level modelingb. Discretization of the Problem (finite difference, finite element, grid simulations, linear algebra, cellular automata, etc.)
- Partial Differential Equations: heat transfer, atmosperic models, soil models, radiation transport
- systems of linear equations: circuit simulation,
- nonlinear of equations: natural gas pipelines
- particle interaction: gravitational pull and molecular dynamics
- recurrence relations: optimization, biological sequence analysis
- etc.
c. Parallel Program Design
d. Construction of Correct Concurrent Programs
- Partitioning to find maximal parallelism in model
- Finding paths of Communication between objects in parallel model
- involves defining synchronization constraints: global invariants
- Agglomerating objects to increase granularity for best performance
- Distributing (Mapping) computation among multiple processors for best performance
e. Verification of Concurrent Software
- using patterns and parallel programming paradigms (if they exist)
- global invariant constraints
- compilation procedures to implement global constraints
- building a good test harness
- eliminating deadlock and livelock
- checking all global states for correctness properties