Testing Concurrent Software

1. Testing Concurrent Software is Difficult, but not impossible
2. Use many of the techniques for testing sequential code
1. However, the number of states in a concurrent program grows exponentially with the number of threads in the program.

2. Concurrent code is non-deterministic and test cases may not be repeatable
3. need longer test runs and more complex test cases
4. test on multiple platforms
2. Most testing of concurrent programs is for:
2. Safety – nothing bad ever happens
1. synchronization errors (critical regions)
2. atomicity failures (assumptions about caches)
3. Liveness – eventually something good will happen
1. deadlock – no thread can continue
2. livelock – one or more threads never continue
3. missed signals

4. Design for Testability
1. separate concurrency from business logic (build model)
2. isolate concurrency by extracting concurrent abstractions – bounded buffers, semaphores, thread pools
3. use java.util.concurrent
4. testing a single abstraction (unit testing) is easier than integration testing
5. Testing Performance

1. Throughput – the rate at which tasks can be completed
2. Responsiveness – how quickly can a system complete a task
3. Scalability – if you add additional processors, how does the increase in computing resources relate to faster results?
5. Creating a Test Plan

1. unit tests, load tests, performance tests, systems tests
2. manual code review by experts, but expensive to do frequently
3. annotations that document concurrency design
6. Concurrent Testing Scenarios

1. Unit testing functionality
1. basic tests of safety and liveness

2. Unit testing functionality under concurrent stress
1. looking for timing-related interactions

3. Component performance testing
1. evaluate performance or scalability of a concurrent abstraction under varying load

4. System stress testing
1. test a large application to see if it works and how it performs

8. Focus on Testing Components
1. one or more Java monitors accessed by multiple threads
2. need a driver to execute the component (# threads?, controlling interaction)
9. Unit Testing
1. some tests can be sequential
1. blocking while waiting for an event
2. non-blocking

2. most testing needs at least 2 threads
3. some  behaviors require multiple threads
1. exchanger
2. cyclic barrier
3. lock
4. blocking queue

9. Example: Bounded Buffer Problem
10. Testing blocking and non-blocking
1. With one then multiple threads
2. Deadlock? All threads are “waiting” on an event that will never occur (dining philosophers)
3. Livelock? There are possible interleavings of thread execution but no progress is made
4. Starvation? A thread fails to make progress
11. Testing Safety
1. mutual exclusion
2. functional testing: input/output
12. Testing Resource Management
13. Generating More Interleavings