Review By: Jon Hagar
09/13/2002As the title implies, this is a handbook for software safety testing. The book has it roots in government and military standards, particularly from Britain and Europe, which is appropriate because many safety issues are government controlled. The book may also be applied to other domains, such as embedded networked devices and high-reliability systems.

The book provides coverage of major safety concepts:

Safety cases—what they are, how to produce/use them, how they fit in the lifecycle;
Testability—how to design for it, why it is important in safety, complexity, and language issues;
Timing—why time correctness is important, how timing impacts safety, how to assess and test it;
Environments/tools—what they are, what tools are in them, specific instance and issues of tools;
Simulation—types of simulator, how they can be used to improve safety and analyze systems and limitations;
Adequacy of testing—theory of testing, what is “good enough” for safety, and test criteria;
Statistical testing—reliability modeling, confidence and related statistical issues; and
Measurement—methods including test cost factors (TCF), and other quantifiable items.

Each topic is introduced, developed in detail, and summarized with a review section.

Critical issues related to standard safety industry concepts and safety cases are detailed. The book discusses growing concern for software safety issues as modern systems become increasingly dependent upon software for operation (e.g., aircraft, power plants, elevators, home appliances). Software errors have caused fatalities. The book outlines the basic test approaches to help increase or assess the safety of software including a good test program. The practical application of topics is emphasized in the book. Some cover fairly advanced concepts such as simulated annealing and reliability growth models, though these are large topic areas, and so are only summarized. The book’s appendices provide a helpful list of other reference material and standards.

The book sometimes used terms before they were defined; however, they were eventually defined somewhere in the text. The book provides a comprehensive summary of safety-related test terms and concepts.

Some topic coverage was not comprehensive. For example, the section on timing aspects, while important to safety and other systems, neglected issues such as the impact from interrupt-driven systems and concepts such as Rate Monotonic Analysis (RMA). These are minor defects and no single book can cover every aspect of testing. The same observation can be made about the section dealing with test tools, where a few specific tools were examined, but this obviously was not a comprehensive treatment. This tool approach is useful because we see how the authors conducted a tool-trade study of what was important to them.

The book has many good references to other information and standards. Concepts presented are well illustrated with useful figures, tables, diagrams, and charts. The text is very readable and not overly difficult. Anyone contemplating establishing a safety testing program would do well to start with this book.