The Software Test Engineer's Handbook, 2nd Edition, 2nd Edition

The Software Test Engineer's Handbook, 2nd Edition, 2nd Edition

by Graham Bath, Judy McKay
Released June 2014
Publisher(s): Rocky Nook
ISBN: 9781492014737

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Many books cover functional testing techniques, but relatively few also cover technical testing. The Software Test Engineer's Handbook-2nd Edition fills that gap. Authors Graham Bath and Judy McKay are core members of the ISTQB Working Party that created the new Advanced Level Syllabus-Test Analyst and Advanced Level Syllabus-Technical Test Analyst. These syllabi were released in 2012.

This book presents functional and technical aspects of testing as a coherent whole, which benefits test analyst/engineers and test managers. It provides a solid preparation base for passing the exams for Advanced Test Analyst and Advanced Technical Test Analyst, with enough real-world examples to keep you intellectually invested.

This book includes information that will help you become a highly skilled Advanced Test Analyst and Advanced Technical Test Analyst. You will be able to apply this information in the real world of tight schedules, restricted resources, and projects that do not proceed as planned.

Table of contents

  1. Cover Page
  2. Title Page
  3. Copyright Page
  4. Preface
  5. Acknowledgements
  6. Contents
  7. 1 Introduction
    1. 1.1 Structure of the Book
    2. 1.2 Requirements for This Book
      1. 1.2.1 Completeness
      2. 1.2.2 Readability
    3. 1.3 What Does “Advanced” Mean?
    4. 1.4 What Is a Test Analyst?
  8. 2 Example Application, Marathon
    1. 2.1 Overview of Marathon
    2. 2.2 General Requirements
    3. 2.3 Use of the Marathon System
    4. 2.4 Availability of the Marathon System
    5. 2.5 Caveats about Marathon
  9. 3 Types of Systems
    1. 3.1 Introduction
      1. 3.1.1 Systems of Systems
      2. 3.1.2 Safety-Critical Systems
      3. 3.1.3 Real-Time and Embedded Systems
  10. 4 Test Management Responsibilities for the Test Analyst
    1. 4.1 Introduction
    2. 4.2 Monitoring and Controlling a Project
      1. 4.2.1 Product (Quality) Risks
      2. 4.2.2 Defects
      3. 4.2.3 Test Cases
      4. 4.2.4 Traceability
      5. 4.2.5 Confidence
    3. 4.3 Talking with Other Testers Wherever They Are
    4. 4.4 Let’s Be Practical
    5. 4.5 Exercises
  11. 5 The Test Process
    1. 5.1 Introduction to the Test Process
    2. 5.2 Fitting the Process to the Life Cycle
    3. 5.3 The Steps of the Test Process
      1. 5.3.1 Test Planning, Monitoring, and Control
        1. 5.3.1.1 Test Planning
        2. 5.3.1.2 Test Monitoring and Control
      2. 5.3.2 Test Analysis
      3. 5.3.3 Test Design
      4. 5.3.4 Test Implementation
      5. 5.3.5 Test Execution
      6. 5.3.6 Test Closure Activities
    4. 5.4 Exercises
  12. 6 Specification-Based Testing Techniques
    1. 6.1 Introduction
    2. 6.2 Individual Specification-Based Techniques
      1. 6.2.1 Equivalence Partitioning
      2. 6.2.2 Boundary Value Analysis (BVA)
      3. 6.2.3 Decision Tables
      4. 6.2.4 Cause-Effect Graphing
      5. 6.2.5 State Transition Testing
      6. 6.2.6 Combinatorial Testing—Pairwise and Orthogonal Arrays
      7. 6.2.7 Combinatorial Testing—Classification Trees
      8. 6.2.8 Use Case Testing
      9. 6.2.9 User Story Testing
      10. 6.2.10 Domain Analysis
    3. 6.3 Selecting a Specification-Based Technique
    4. 6.4 Let’s Be Practical
    5. 6.5 Exercises
  13. 7 Defect-Based Testing Techniques
    1. 7.1 Introduction
    2. 7.2 Taxonomies
    3. 7.3 Applying the Technique
    4. 7.4 Let’s Be Practical
    5. 7.5 Exercises
  14. 8 Experience-Based Testing Techniques
    1. 8.1 Introduction
    2. 8.2 Error Guessing
    3. 8.3 Checklist-Based Testing
    4. 8.4 Exploratory Testing
    5. 8.5 Strengths and Weaknesses
    6. 8.6 Let’s Be Practical
    7. 8.7 Exercises
  15. 9 Functional Testing
    1. 9.1 Introduction
    2. 9.2 Accuracy Testing
    3. 9.3 Suitability Testing
    4. 9.4 Interoperability Testing
    5. 9.5 Let’s Be Practical
    6. 9.6 Exercises
  16. 10 Usability and Accessibility Testing
    1. 10.1 Usability Testing
      1. 10.1.1 Effectiveness
      2. 10.1.2 Efficiency
      3. 10.1.3 Satisfaction
      4. 10.1.4 Subcharacteristics of Usability
    2. 10.2 Accessibility Testing
    3. 10.3 Test Process for Usability and Accessibility Testing
      1. 10.3.1 Planning Issues
      2. 10.3.2 Test Design
      3. 10.3.3 Specifying Usability Tests
    4. 10.4 Let’s Be Practical
    5. 10.5 Exercises
  17. 11 Reviews for the Test Analyst
    1. 11.1 Introduction
    2. 11.2 What Types of Work Products Can the Test Analyst Review?
    3. 11.3 When Should the Test Analyst Do the Reviews?
    4. 11.4 Issues
      1. 11.4.1 How Do We Make Our Review Effective?
      2. 11.4.2 Do We Have the Right People?
      3. 11.4.3 What Should We Do with the Defects?
      4. 11.4.4 But We Don’t Have Time to Do Reviews!
    5. 11.5 Using Checklists for Reviews
    6. 11.6 Checklist for Requirements Reviews
    7. 11.7 Checklist for Use Case Reviews
    8. 11.8 Checklist for Usability Reviews
    9. 11.9 Checklist for User Story Reviews
    10. 11.10 Checklist for Success
    11. 11.11 Let’s Be Practical
    12. 11.12 Exercises
  18. 12 Defect Management
    1. 12.1 Introduction
    2. 12.2 What Is a Defect?
    3. 12.3 When Can We Find Defects?
    4. 12.4 Defect Fields
      1. 12.4.1 Classification Information for Defects
    5. 12.5 Defect Life Cycles
    6. 12.6 Metrics and Reporting
      1. 12.6.1 Test Progress Monitoring
      2. 12.6.2 Defect Density Analysis
      3. 12.6.3 Found vs. Fixed Metrics
      4. 12.6.4 Convergence Metrics
      5. 12.6.5 Phase Containment Information
      6. 12.6.6 Is Our Defect Information Objective?
    7. 12.7 Process Improvement Opportunities
    8. 12.8 Let’s Be Practical
    9. 12.9 Exercises
  19. 13 Tools Concepts
    1. 13.1 What Is a Test Tool?
    2. 13.2 Why Would We Use a Tool?
    3. 13.3 Types of Tools
      1. 13.3.1 Test Design Tools
      2. 13.3.2 Data Tools
      3. 13.3.3 Test Execution Tools
      4. 13.3.4 When Should We Automate?
      5. 13.3.5 Things to Know About Automation
      6. 13.3.6 Implementing Automation
        1. 13.3.6.1 Data-Driven Automation
        2. 13.3.6.2 Keyword-Driven Automation
        3. 13.3.6.3 Benefits of Automation Techniques
    4. 13.4 Should We Automate All Our Testing?
    5. 13.5 Let’s Be Practical
    6. 13.6 Exercises
  20. 14 Test Management Responsibilities for the Technical Test Analyst
    1. 14.1 Introduction
      1. 14.1.1 Technical Risks
    2. 14.2 Let’s Be Practical
    3. 14.3 Exercises
  21. 15 Analysis Techniques
    1. 15.1 Static Analysis
      1. 15.1.1 Benefits
      2. 15.1.2 Limitations
      3. 15.1.3 Control Flow Analysis
      4. 15.1.4 Data Flow Analysis
      5. 15.1.5 Compliance to Coding Standards
      6. 15.1.6 Generating Code Metrics
      7. 15.1.7 Static Analysis of a Website
      8. 15.1.8 Call Graphs
    2. 15.2 Dynamic Analysis
      1. 15.2.1 Benefits
      2. 15.2.2 Limitations
      3. 15.2.3 Memory Leaks
      4. 15.2.4 Problems with Pointers
      5. 15.2.5 Analysis of Performance
    3. 15.3 Let’s Be Practical
    4. 15.4 Exercises
  22. 16 Structure-Based Testing Techniques
    1. 16.1 Benefits
    2. 16.2 Drawbacks
    3. 16.3 Application of Structure-Based Techniques
    4. 16.4 Individual Structural Techniques
      1. 16.4.1 Statement Testing
      2. 16.4.2 Decision Branch Testing
      3. 16.4.3 Condition Testing
      4. 16.4.4 Decision Condition Testing
      5. 16.4.5 Multiple Condition Testing
      6. 16.4.6 Modified Condition/Decision Coverage (MC/DC) Testing
      7. 16.4.7 Path Testing
      8. 16.4.8 API Testing
    5. 16.5 Selecting a Structure-Based Technique
    6. 16.6 Exercises
  23. 17 Efficiency Testing
    1. 17.1 Overview
    2. 17.2 Performance Testing
    3. 17.3 Load Testing
    4. 17.4 Stress Testing
    5. 17.5 Scalability Testing
    6. 17.6 Resource Utilization Testing
    7. 17.7 Measuring Efficiency
    8. 17.8 Planning of Efficiency Tests
      1. 17.8.1 Risks and Typical Efficiency Defects
      2. 17.8.2 Different Types of Test Objects
      3. 17.8.3 Requirements for Efficiency Tests
      4. 17.8.4 Approaches to Efficiency Tests
      5. 17.8.5 Efficiency Pass/Fail Criteria
      6. 17.8.6 Tooling for Efficiency Tests
      7. 17.8.7 Environments
      8. 17.8.8 Organizational Issues
      9. 17.8.9 Life Cycle Issues
    9. 17.9 Specifying Efficiency Tests
    10. 17.10 Executing Efficiency Tests
    11. 17.11 Reporting Results of Efficiency Tests
    12. 17.12 Tools for Performance Testing
    13. 17.13 Let’s Be Practical
    14. 17.14 Exercises
  24. 18 Security Testing
    1. 18.1 Overview of Security Testing
    2. 18.2 Defining Security
    3. 18.3 Typical Security Threats
    4. 18.4 Approach to Security Testing
    5. 18.5 Organizational Issues
    6. 18.6 Life Cycle Issues
    7. 18.7 Planning Security Tests
    8. 18.8 Security Test Analysis and Design
      1. 18.8.1 Software Attacks
      2. 18.8.2 Other Design Techniques for Security Tests
    9. 18.9 Execution of Security Tests
    10. 18.10 Reporting Security Tests
    11. 18.11 Tools for Security Testing
    12. 18.12 Let’s Be Practical
    13. 18.13 Exercises
  25. 19 Reliability Testing
    1. 19.1 Overview
      1. 19.1.1 Maturity
      2. 19.1.2 Fault Tolerance
      3. 19.1.3 Recoverability
    2. 19.2 Reliability Test Planning
      1. 19.2.1 Assessing the Risk
      2. 19.2.2 Setting Reliability Goals
      3. 19.2.3 Life Cycle Issues
      4. 19.2.4 Approaches to Reliability Testing
      5. 19.2.5 Approach for Measuring Reliability Levels
      6. 19.2.6 Approach for Establishing Fault Tolerance
      7. 19.2.7 Approach to Failover Testing
      8. 19.2.8 Approach to Backup and Restore Testing
    3. 19.3 Reliability Test Specification
      1. 19.3.1 Test Specification for Reliability Growth
      2. 19.3.2 Test Specification for Fault Tolerance
      3. 19.3.3 Test Specification for Failover
      4. 19.3.4 Test Specification for Backup and Restore
    4. 19.4 Reliability Test Execution
    5. 19.5 Reporting Reliability Tests
    6. 19.6 Tools for Reliability Testing
    7. 19.7 Let’s Be Practical
    8. 19.8 Exercises
  26. 20 Maintainability Testing
    1. 20.1 Overview
    2. 20.2 Testing for Maintainability
      1. 20.2.1 What Is Maintainability?
      2. 20.2.2 Why Is Maintainability Underrepresented?
      3. 20.2.3 The Causes of Poor Maintainability
    3. 20.3 Maintainability Test Planning
    4. 20.4 Maintainability Test Specification
    5. 20.5 Performing Maintainability Tests and Analysis
    6. 20.6 Maintenance Testing
    7. 20.7 Tasks of the Technical Test Analyst
    8. 20.8 Let’s Be Practical
    9. 20.9 Exercises
  27. 21 Portability Testing
    1. 21.1 Adaptability
      1. 21.1.1 Reasons for Poor Adaptability Characteristics
      2. 21.1.2 Adaptability Testing
    2. 21.2 Replaceability
      1. 21.2.1 Replaceability Considerations
      2. 21.2.2 Replaceability Testing
    3. 21.3 Installability
      1. 21.3.1 Risk Factors for Installability
      2. 21.3.2 Installability Testing
    4. 21.4 Co-existence/compatibility
      1. 21.4.1 Co-existence Testing
    5. 21.5 Let’s Be Practical
    6. 21.6 Exercises
  28. 22 Reviews for the Technical Test Analyst
    1. 22.1 Introduction
    2. 22.2 Checklists for Reviews
    3. 22.3 Checklist for Code Reviews
    4. 22.4 Checklist for Architectural Reviews
    5. 22.5 Exercises
  29. 23 Tools for the Technical Test Analyst
    1. 23.1 Introduction
    2. 23.2 Tasks and Skills of the Technical Test Analyst in Test Automation
    3. 23.3 Integration and Information Interchange between Tools
    4. 23.4 Defining the Test Automation Project
      1. 23.4.1 Approaches to Test Automation
    5. 23.5 Should We Automate All Our Testing?
    6. 23.6 Types of Tools
      1. 23.6.1 Fault Seeding and Fault Injection Tools
      2. 23.6.2 Test Tools for Component Testing and Build
      3. 23.6.3 Tools for Static Analysis of a Website
      4. 23.6.4 Tools to Support Model-Based Testing
      5. 23.6.5 Test Tools for Static and Dynamic Analysis
      6. 23.6.6 Performance Test Tools
      7. 23.6.7 Simulation and Emulation Tools
      8. 23.6.8 Debugging and Troubleshooting Tools
    7. 23.7 Exercises
  30. A Glossary
    1. Test Analyst Glossary
    2. Technical Test Analyst Glossary (ATT)
  31. B Literature
    1. Books
    2. ISTQB Publications
    3. Standards
    4. WWW Pages
  32. Index

Product information

  • Title: The Software Test Engineer's Handbook, 2nd Edition, 2nd Edition
  • Author(s): Graham Bath, Judy McKay
  • Release date: June 2014
  • Publisher(s): Rocky Nook
  • ISBN: 9781492014737