Verification, Validation, and Testing of Concurrent Software

1. Verification: demonstration of consistency, completeness, and correctness of the software at each stage of and between states of the software life-cycle

a. Types of verification – manual inspection, testing, formal methods, model-checking

b. Is the software built correctly?

2. Validation: at the end of the software life-cycle, the software must be evaluated to ensure the compliance with respect to the customer needs and requirements

a. Must verify the software artifacts produced at every stage of the life-cycle

b. Did we build the correct software? (to solve the problem)

3. Testing: evaluating the execution of the software on sample data

a. Does the sample data test give you confidence the program is correct?

b. Is the program time and space efficient?

c. Used at all stages of development

4. Model-checking:

a. a model is an abstract representation of the entire program; it contains only that part of the code that you want to check for correctness

b. a model-checker guarantees that every state of the program has been checked for correctness

i. all invariants hold

ii. all post- and pre-conditions are true at their specified points in the program

5. Debugging: isolating where a specific error has occurred, fixing it, and re-testing it.

a. Used at all stages in life-cycle

b. Desk-checking (walk-thrus) help to understand the computational model of the program

c. Interactive tools are useful in isolating the specific statement at which the error occurred

d. and assertions are extremely useful in isolating the trace of where the error-chain started

6. Every phase of the software life-cycle requires verification techniques to find errors , omissions, and contradictions

a. Requirements analysis and specification

i. Use cases (scenarios of expected system use) help in establishing correctness; can also be used to generate test cases

ii. Formal requirements (state charts, OCL) can be checked for properties such as such as consistency and completeness using a model-checker

iii. Earliest possible discovery of errors saves time and money

b. Design Phase

i. OCL constraints help in understanding the computational model

ii. Model-checkers help in developing correct synchronization

iii. Walk-throughs, inspections, and design review help in finding erroneous assumptions and in building group understanding

c. Implementation

i. Program testing is essential at every step of implementation

ii. Code walk-throughs, code reviews, audits all are useful

iii. Dynamic analysis tools such as dynamic monitoring of assertions and dynamic design by contract monitoring

7. Project and Code Reviews: the process of meeting during which a work product, or a set of work products, is presented to project personnel, managers, users, customers, or other parties for comment or approval. The types of review include model review, code review, design review, requirements review, and test case review

a. Should result in a written report on the status

b. Active and open participation

c. All participants should have a “vested interest” in the result

8. Walk-through: a manual static analysis technique in which a designer or programmer leads members of the development team through the code or documentation.

a. A manual simulation of the program execution. The group discussing each statement of the program.

b. Leads to finding errors in assumptions and a better understanding of the semantics of the software

9. Audit: a independent examination of the software product (documentation and code) to assess compliance with requirements, specifications, contractual agreements, and maintenance standards