0707.321 Principles of Software Engineering

Chapter 16: Critical Systems: Dependibility

Elements of Dependibility (informal):

Availability

How likely that system will be up and running

Reliability

How likely that system behaves correctly

Safety

How likely is system to cause damage

Security

How likely can system defend against accidental or deliberate intrusion

Quantifying dependibility:

Reliability

The probability of failure-free operation over a specified time in a given environment for a specific purpose.

Availability

The probability that a system, at a point in time, will be operational and able to deliver the requested services.

Reliability terminology:

System failure

When system doesn’t deliver user-expected service

System error

When system doesn’t behave to spec

System fault

Incorrect system state

Human error (mistake)

Source of error is human behavior

Process Strategies

• Avoidance (act so as to prevent their introduction)
• Detection and Removal (testing & debugging)
• Limitation of Influence (e.g., fault tolerance)

Types of damage through security break:

• Denial of service (if system forced offline)
• Corruption of programs or data
• Disclosure of confidential information

Chapter 19: Validation & Verification

Verification

Are we building the product right?

Validation

Are we building the right product?

Software inspections

Static evaluations of the product and components. No program execution required.

Automatic static analysis

Program analyzers checking for symptoms of bugs (e.g., unreachable code; uninitialized or unreferenced variables)

Software testing

Dynamic evaluations of product execution

Defect testing

Search for inconsistencies between (running) program and specification

Automatic dynamic analysis

Detecting problems static analysis cannot catch. Collect data during program run and report on problems afterwards. (e.g., memory management)

Statistical testing

Exercising system, collecting data on system's performance and reliability, reaching conclusions from statistical viewpoint

Regression testing

Tests of previously-working program functionality

Cleanroom software development process

Code verified formally. Little need to do defect testing. Five key characteristics:

1. Formal specification
2. Incremental development
3. Structured programming (limited control constructs; created by mechanical transformations from specification)
4. (Formal) static verification
5. Statistical testing

Chapter 20: Software Testing

Component Testing

Testing of individual program components

Black-box testing

Developed from specification (not from implementation)

Equivalence partitioning

Division of input domains into equivalence classes (inputs for which system behaves similarly)

White-box (clear-box; structural) testing

Developed from study of implementation

Path testing

Exercising every (static) path through a program/component

Program flow graph

Skeletal model of paths through program

Cyclomatic complexity

Number of paths through a program, or its flow graph (= |edges| - |nodes| + 2)

Integration Testing

Testing of groups of components integrated to create system or sub-system

Top-down testing

Integrate higher-level components, working down component hierarchy. Use stubs in place of (as yet) unintegrated components.

Bottom-up testing

Integrate lower-level components, working up component hierarchy. Create special programs to drive tests at each level.

Stub

Simplified component replacement. Same interface, reduced functionality.

Stress testing

Exercising program beyond maximum design load.

Chapter 21: Critical Systems Validation

Operational profile

(As close to actual) probabilistic mix of inputs for a program

Experienced-based security validation

Testing system security against well-known security-penetrating attacks

Tiger team

Group tasked with breaching system security. Search for novel attacks.

Chapter 26: Legacy Systems

Legacy systems

Old, existing systems that are well-established components of the work process at the business/organization

Dispensation

Discard

implies removal of task from business process

Continue maintenance

Transform

so as to improve maintainability (but preserve system behavior)

Replace

with a new system that fill role

Assessment

Determining what to do with legacy system

Low (software) quality (i.e., high maintenance cost), low business value

Good candidate for discard

Low quality, high business value

Candidate for transformation or replacement

High quality, low business value

Continue to maintain or discard

High quality, high business value

Continue to maintain

Chapter 27: Software Change

Lehman's "Laws":

Continuing change

Program must evolve with times, or become less useful

Increasing complexity

As systems change: complexity increases, structure degrades

Large program evolution

Rate and size of system changes an inate characteristic of the system ("has an inertial mass")

Organizational stability

Rate of development/evolution more-or-less constant, independent of resources devoted

Conservation of familiarity

Incremental change in each release more-or-less constant

Sources that lead to higher maintenance costs (over initial development):

Team (in)stability

Development team wants to move on to newer (and sexier) tasks

Contractual failing

Development contracted separated from maintenance; little incentive to consider maintenance issues during development

Staff skills

Maintenance has low status; less-experienced programmers tasked with duties

System structure degredation

Harder to maintain as software quality decreases