0707.321 Principles of Software Engineering

Chapter 16: Critical Systems: Dependibility

Elements of Dependibility (informal):

Availability

How likely that system will be up and running

Reliability

How likely that system behaves correctly

Safety

How likely is system to cause damage

Security

How likely can system defend against accidental or deliberate intrusion

Quantifying dependibility:

Reliability

The probability of failure-free operation over a specified time in a given environment for a specific purpose.

Availability

The probability that a system, at a point in time, will be operational and able to deliver the requested services.

Reliability terminology:

System failure

When system doesn’t deliver user-expected service

System error

When system doesn’t behave to spec

System fault

Incorrect system state

Human error (mistake)

Source of error is human behavior

Process Strategies

• Avoidance (act so as to prevent their introduction)
• Detection and Removal (testing & debugging)
• Limitation of Influence (e.g., fault tolerance)

Types of damage through security break:

• Denial of service (if system forced offline)
• Corruption of programs or data
• Disclosure of confidential information

Chapter 19: Validation & Verification

Verification

Are we building the product right?

Validation

Are we building the right product?

Software inspections

Static evaluations of the product and components. No program execution required.

Automatic static analysis

Program analyzers checking for symptoms of bugs:

• Unreachable code
• Uninitialized variables
• Unreferenced variables

Software testing

Dynamic evaluations of product execution

Defect testing

Search for inconsistencies between (running) program and specification

Automatic dynamic analysis

Detecting problems static analysis cannot catch. Collect data during program run and report on problems afterwards. (e.g., memory management)

Statistical testing

Test of system's performance and reliability

Regression testing

Tests of previously-working program functionality

Cleanroom software development process

Code verified formally. Little need to do defect testing. Five key characteristics:

1. Formal specification
2. Incremental development
3. Structured programming (limited control constructs; created by mechanical transformations from specification)
4. (Formal) static verification
5. Statistical testing

Chapter 20: Software Testing

Component Testing

Testing of individual program components

Black-box testing

Developed from specification

Equivalence partitioning

Division of input domains into equivalence classes (inputs for which system behaves similarly)

White-box (clear-box; structural) testing

Developed from study of implementation

Path testing

Exercising every (static) path through a program/component

Program flow graph

Skeletal model of paths through program

Cyclomatic complexity

Number of paths through a program, or its flow graph (= |edges| - |nodes| + 2)

Integration Testing

Testing of groups of components integrated to create system or sub-system

Top-down testing

Integrate higher-level components, working down component hierarchy. Use stubs in place of (as yet) unintegrated components.

Bottom-up testing

Integrate lower-level components, working up component hierarchy. Create special programs to drive tests at each level.

Stub

Simplified component replacement. Same interface, reduced functionality.

Stress testing

Exercising program beyond maximum design load.

Chapter 21: Critical Systems Validation

Operational profile

(As close to actual) probabilistic mix of inputs for a program

Experienced-based security validation

Testing system security against well-known security-penetrating attacks

Tiger team

Group tasked with breaching system security. Search for novel attacks.