Quantitative Process Management
a key process area for level 4: Managed
The purpose of Quantitative Process Management is to
control the process performance of the software project quantitatively.
Software process performance represents the actual results achieved from
following a software process.
Quantitative Process Management involves establishing goals for the performance
of the project's defined software process, which is described in the Integrated
Software Management key process area, taking measurements of the process
performance, analyzing these measurements, and making adjustments to maintain
process performance within acceptable limits. When the process performance is
stabilized within acceptable limits, the project's defined software process,
the associated measurements, and the acceptable limits for the measurements are
established as a baseline and used to control process performance
quantitatively.
The organization collects process performance data from the software projects
and uses these data to characterize the process capability (i.e., the process
performance a new project can expect to attain) of the organization's standard
software process, which is described in the Organization Process Definition key
process area. Process capability describes the range of expected results from
following a software process (i.e., the most likely outcomes that are expected
from the next software project the organization undertakes). These process
capability data are, in turn, used by the software projects to establish and
revise their process performance goals and to analyze the performance of the
projects' defined software processes.
Goals
Goal 1
The quantitative process management activities are planned.
Goal 2
The process performance of the project's defined software process is
controlled quantitatively.
Goal 3
The process capability of the organization's standard software process is
known in quantitative terms.
Commitment to perform
Commitment 1 -- The project follows a written organizational policy for
measuring and quantitatively controlling the performance of the project's
defined software process.
This policy typically specifies that:
- Each project implements a documented plan to bring the project's defined
software process under quantitative control.
The term "quantitative control" implies any quantitative or statistically-based
technique appropriate to analyze a software process, identify special causes of
variations in the performance of the software process, and bring the
performance of the software process within well-defined limits.
A special cause of variation is some transient circumstance (such as a specific
local condition, a single machine, a single individual, or a small group of
people performing in an unexpected way) that causes an unexpected, transient
change in the process performance.
- Sensitive data relating to individuals' performance are protected, and
access to these data is appropriately controlled.
Use of measurement data to evaluate individuals will negatively affect the
correctness and usefulness of the measurement data that are reported.
Commitment 2 -- The organization follows a written policy for
analyzing the process capability of the organization's standard software
process.
This policy typically specifies that:
- The projects' measurements of process performance are analyzed to
establish and maintain a process capability baseline for the organization's
standard software process.
The process capability baseline includes:
- the description of the organization's standard software process,
- the standard definitions of the measurements, and
- the expected range of values for the measurements.
- The process capability baseline for the organization's
standard software process is used by the software projects in establishing
their process performance goals.
Ability to perform
Ability 1 -- A group that is responsible for coordinating the quantitative
process management activities for the organization exists.
A group is the collection of departments, managers, and individuals who have
responsibility for a set of tasks or activities. A group could vary from a
single individual assigned part time, to several part-time individuals assigned
from different departments, to several individuals dedicated full time.
Considerations when implementing a group include assigned tasks or activities,
the size of the project, the organizational structure, and the organizational
culture. Some groups, such as the software quality assurance group, are
focused on project activities, and others, such as the software engineering
process group, are focused on organization-wide activities.
- This group is either part of the group responsible for the
organization's software process activities (e.g., software engineering process
group) or its activities are closely coordinated with that group.
Ability 2 -- Adequate resources and funding are provided for the
quantitative process management activities.
- The managers and task leaders of the software engineering groups and other
software-related groups perform the project's quantitative process management
activities.
Examples of software-related groups include:
- software quality assurance,
- software configuration management, and
- documentation support.
- An organization-wide measurement program exists.
The organization's measurement program includes:
- the definition of the organization-wide measurements,
- the collection of the organization's measurement data,
- the analysis of the organization's measurement data, and
- the quantitative measurement goals for the organization.
- Tools to support quantitative process management are made
available.
Examples of support tools include:
- software source code analyzers,
- automated test coverage analyzers,
- database systems,
- quantitative analysis packages, and
- problem tracking packages.
Ability 3 -- Support exists for collecting, recording, and analyzing data
for selected process and product measurements.
The product data referred to in these practices are product measurements used
in analyzing the software process.
Ability 4 -- The individuals implementing or supporting quantitative
process management receive required training to perform these activities.
Examples of training include:
- modeling and analyzing the software process;
- selecting, collecting, and validating process measurement data; and
- applying basic quantitative methods and analysis techniques (e.g.,
estimation models, Pareto diagrams, and control charts).
Refer to the Training Program key process area.
Ability 5 -- The members of the software engineering group and other
software-related groups receive orientation on the goals and value of
quantitative process management.
Refer to the Training Program key process area.
Activities performed
Activity 1 -- The software project's plan for quantitative process
management is developed according to a documented procedure.
This procedure typically specifies that:
- The quantitative process management plan is based on:
- the organization's strategic goals for product quality, productivity, and
product development cycle time;
- the organization's measurement program;
- the organization's standard software process;
- the project's goals for the software product's quality, productivity, and
product development cycle time;
- the measured performance of other projects' defined software processes; and
- the description of the project's defined software process.
- The plan undergoes peer review.
Refer to the Peer Reviews key process area.
- The plan is reviewed by the group responsible for the organization's
software process activities (e.g., the software engineering process group).
- The plan is managed and controlled.
"Managed and controlled" implies that the version of the work product in use at
a given time (past or present) is known (i.e., version control), and changes
are incorporated in a controlled manner (i.e., change control).
If a greater degree of formality than is implied by "managed and controlled" is
desired, the work product can be placed under the full discipline of
configuration management, as is described in the Software Configuration
Management key process area.
Activity 2 -- The software project's quantitative process management
activities are performed in accordance with the project's quantitative
process management plan.
The plan covers:
- The goals and objectives of the quantitative process management
activities.
- The software tasks or other software activities that will be measured
and analyzed.
- The instrumentation of the project's defined software process.
The instrumentation is based on the organization's measurement program, the
description of the organization's standard software process, and the
description of the project's defined software process.
- The quantitative process management activities to be performed and the
schedule for these activities.
In addition to current organizational and project needs, measurements that may
be useful to future efforts are included.
- The groups and individuals responsible for the quantitative process
management activities.
- The resources required to perform the quantitative process management
activities, including staff and tools.
- The procedures to be followed in performing the quantitative process
management activities.
Activity 3 -- The strategy for the data collection and the quantitative
analyses to be performed are determined based on the project's defined
software process.
The attributes of the project's defined software process that are considered
include:
- The tasks, the activities, and their relationships to each
other.
- The software work products and their relationships to each other and
to the project's defined software process.
- The process control points and data collection points.
Activity 4 -- The measurement data used to control the project's defined
software process quantitatively are collected according to a documented
procedure.
This procedure typically specifies that:
- The measurement data collected support the organization's and the software
project's measurement goals and objectives.
- The specific measurement data to be collected, their precise definitions,
the intended use and analysis of each measurement, and the process control
points at which they will be collected are defined.
Examples of measurement data include:
- estimated/planned versus actual data on software size, cost, and schedule;
- productivity data;
- quality measurements as defined in the software quality plan;
- coverage and efficiency of peer reviews;
- effectiveness of training;
- test coverage and efficiency;
- software reliability measures;
- number and severity of defects found in the software requirements;
- number and severity of defects found in the software code; and
- number and rate of closure on action items.
- The measurements are chosen from the entire software life cycle (e.g.,
both the development and post-development stages).
- The measurements cover the properties of the key software process activities
and major software work products.
- The measurement data that relate to the organization's standard software
process are uniformly collected across the software projects.
- The measurements to be controlled are a natural result of the software
activities where possible.
- The measurements are selected to support predefined analysis
activities.
In some cases, measurements may be research oriented and should be explicitly
designated as such.
- The validity of the measurement data is independently assessed.
- The collected measurement data are stored in the organization's software
process database as appropriate.
Refer to Activity 5 of the Organization Process Definition key process area for
practices covering the organization's software process database.
Activity 5 -- The project's defined software process is analyzed and
brought under quantitative control according to a documented procedure.
This procedure typically specifies that:
- The specific data analysis activities are predefined.
The description of the data analysis activities covers:
- input data required,
- tools used,
- data manipulations performed,
- information to be derived, and
- decision criteria used in performing the analysis and deciding what
actions to take as a result of the analysis.
Examples of analysis techniques include:
- Pareto diagrams,
- control charts,
- trend diagrams, and
- scatter diagrams.
- Measurement data on the process activities throughout the project's
defined software process are identified, collected, and analyzed.
- The selected measurements appropriately characterize the process they
represent.
- The expected values for mean and variance are specified for each
measurement.
- The acceptable limits for each measurement are defined and the project's
process performance baseline is established.
An example of establishing acceptable limits is to calculate the historical
deviation from the mean performance of the process.
- The actual values of each measurement are compared to the expected values
of the mean and variance.
Examples of comparing actual process performance to defined acceptable limits
include:
- comparing the peer review hours spent per thousand lines of source code to
upper and lower limits determined by analyzing historical data; and
- comparing the expansion ratio of software requirements (e.g., number of
"shalls") into the number of lines of source code to upper and lower limits
determined by analyzing historical data.
- Adjustments are made to bring the actual process performance in line with
the defined acceptable limits, as appropriate.
- When the project's defined software process is controlled quantitatively,
baselines are established for:
- the definition of the measurements,
- the actual measurement data, and
- the acceptable limits for the measurements.
- The process performance baseline for the software project is managed and
controlled.
Activity 6 -- Reports documenting the results of the software project's
quantitative process management activities are prepared and distributed.
- The results of the data analysis are reviewed with those affected by the
data before they are reported to anyone else.
- The software managers, software task leaders, and senior management
receive regular reports appropriate for their needs.
- The software quality assurance group receives regular reports appropriate
to its needs.
- The project manager, senior managers, software managers, and software task
leaders receive specialized reports on request.
Activity 7 -- The process capability baseline for the organization's
standard software process is established and maintained according to a
documented procedure.
This procedure typically specifies that:
- The project's software process data, as summarized in its process
performance baseline, are recorded in the organization's software process
database.
Refer to Activity 5 of the Organization Process Definition key process area for
practices covering the organization's software process database.
- The process performance baseline for each project's defined software
process is incorporated, as appropriate, into the process capability baseline
for the organization's standard software process.
- The process capability baseline for the organization's standard software
process is documented.
- Process capability trends for the organization's standard software process
are examined to predict likely problems or opportunities for
improvements.
Examples of using capability trends include:
- predicting the occurrence of software defects and comparing the
predictions to actuals, and
- predicting the distribution and characteristics of defects remaining in a
product based on the data from peer reviews and/or test.
Examples of areas that are likely sources of defects include:
- items for estimating and planning,
- activities performed early in the software life cycle such as requirements
analysis,
- major documentation items,
- items and activities that have been prone to defect insertion in the past,
- activities for implementing changes and fixing defects, and
- labor-intensive activities.
Examples of areas that are likely opportunities for improvement include:
- activities that other projects and organizations have successfully
automated,
- nondeliverable and support items and activities such as training and
tools,
- quality oriented activities such as peer reviews and testing, and
- labor intensive activities.
- The process capability baseline for the organization's standard software
process is managed and controlled.
- When a software project that is substantially different from past projects
is undertaken, a new process performance baseline is established for that
project as part of tailoring the organization's standard software
process.
Refer to Activity 1 of the Integrated Software Management key process area for
practices covering the project's tailoring of the organization's standard
software process.
Examples of substantial differences include:
- new application domains,
- use of radically different technologies, and
- significant change in the size of the application.
- Changes to the organization's standard software process are tracked and
analyzed to assess their effects on the process capability baseline.
Measurement and analysis
Measurement 1 -- Measurements are made and used to determine the status of
the activities for quantitative process management.
Examples of measurements include:
- the cost over time for the quantitative process management activities,
compared to the plan; and
- the accomplishment of schedule milestones for quantitative process
management activities, compared to the approved plan (e.g., establishing the
process measurements to be used on the project, determining how the process
data will be collected, and collecting the process data).
Verifying implementation
Verification 1 -- The activities for quantitative process management are
reviewed with senior management on a periodic basis.
Refer to Verification 1 of the Organization Process Focus key process area and
Verification 1 of the Software Project Tracking and Oversight key process area
for practices covering the typical content of senior management oversight
reviews.
Verification 2 -- The software project's activities for quantitative
process management are reviewed with the project manager on both a periodic
and event-driven basis.
Refer to Verification 2 of the Software Project Tracking and Oversight key
process area for practices covering the typical content of project management
oversight reviews.
Verification 3 -- The software quality assurance group reviews and/or
audits the activities and work products for quantitative process management
and reports the results.
Refer to the Software Quality Assurance key process area.
At a minimum, the reviews and/or audits verify that:
- The plans for the quantitative process management activities are followed.
- The procedures for quantitative process management are followed.
- The collection and analysis of quantitative process management data are
performed as required, including verification that:
- the needed data exist,
- the needed data are collected,
- the data collected are needed,
- the data collected support the goals and objectives of
the organization's measurement program,
- the cost of collecting the data is justified by the usefulness of the data,
- the data are collected at the correct point in the software life cycle,
- the data are accurate and correct,
- the data are timely, and
- the confidentiality of the data is properly protected.
Table of contents
Forward one chapter