Testing Concurrent Software
- Testing
Concurrent Software is Difficult, but not impossible
- Use
many of the techniques for testing sequential code
- However,
the number of states in a concurrent program grows exponentially with the
number of threads in the program.
- Concurrent
code is non-deterministic and test cases may not be repeatable
- need
longer test runs and more complex test cases
- test
on multiple platforms
- Most
testing of concurrent programs is for:
- Safety
nothing bad ever happens
- synchronization
errors (critical regions)
- atomicity
failures (assumptions about caches)
- Liveness eventually something good will happen
- deadlock
no thread can continue
- livelock one or more threads never continue
- missed
signals
- Design
for Testability
- separate
concurrency from business logic (build model)
- isolate
concurrency by extracting concurrent abstractions bounded buffers,
semaphores, thread pools
- use java.util.concurrent
- testing
a single abstraction (unit testing) is easier than integration testing
- Testing
Performance
- Throughput
the rate at which tasks can be completed
- Responsiveness
how quickly can a system complete a task
- Scalability
if you add additional processors, how does the increase in computing
resources relate to faster results?
- Creating
a Test Plan
- unit
tests, load tests, performance tests, systems tests
- manual
code review by experts, but expensive to do frequently
- annotations
that document concurrency design
- Concurrent
Testing Scenarios
- Unit
testing functionality
- basic
tests of safety and liveness
- Unit
testing functionality under concurrent stress
- looking
for timing-related interactions
- Component
performance testing
- evaluate
performance or scalability of a concurrent abstraction under varying
load
- System
stress testing
- test
a large application to see if it works and how it performs
- Focus
on Testing Components
- one or more Java monitors accessed
by multiple threads
- need a driver to execute the component
(# threads?, controlling interaction)
- Unit
Testing
- some
tests can be sequential
- blocking
while waiting for an event
- non-blocking
- most
testing needs at least 2 threads
- some
behaviors require multiple threads
- exchanger
- cyclic
barrier
- lock
- blocking
queue
- Example:
Bounded Buffer Problem
- Testing
blocking and non-blocking
- With one then multiple threads
- Deadlock? All threads are waiting on an event
that will never occur (dining philosophers)
- Livelock? There are possible interleavings of
thread execution but no progress is made
- Starvation? A thread fails to make progress
- Testing
Safety
- mutual
exclusion
- functional
testing: input/output
- Testing
Resource Management
- Generating
More Interleavings