Table of Contents Preface... Glossary... xi xv Chapter 1. Fundamentals of Testing... 1 1.1.Whyistestingnecessary?(FL1.1)... 1 1.1.1.Softwaresystemscontext... 1 1.1.2.Causesofsoftwaredefects... 3 1.1.3. Role of testing in software development, maintenance and operations... 5 1.1.4.Test and quality... 5 1.1.5.Terminology... 6 1.2.Whatistesting?(FL1.2)... 7 1.2.1.Originofdefects... 8 1.2.2.Commongoalsoftesting... 9 1.2.3.Examplesofobjectivesfortesting... 9 1.2.4.Testanddebugging... 10 1.3.Paradoxesandmainprinciples(FL1.3)... 11 1.3.1.Testingidentifiesthepresenceofdefects... 11 1.3.2.Exhaustivetestingisimpossible... 11 1.3.3.Earlytesting... 12 1.3.4.Defectclustering... 12 1.3.5.Pesticide paradox... 13 1.3.6.Testingiscontextdependent... 13 1.3.7.Absenceoferrorsfallacy... 14 1.4.Fundamentaltestprocess(FL1.4)... 14 1.4.1.Planning... 15 1.4.2.Control... 15
vi Fundamentals of Software Testing 1.4.3. Test analysis and design... 16 1.4.4.Testimplementation... 18 1.4.5.Testexecution... 19 1.4.6.Analysisofexitcriteria... 20 1.4.7.Reporting... 20 1.4.8.Test closure activities... 21 1.5.Psychology of testing (FL 1.5)... 22 1.5.1.Levelsofindependence... 23 1.5.2.Adaptationtogoals... 24 1.5.3.Destructiveorconstructive?... 25 1.5.4.Relational skills... 25 1.5.5.Changeofperspective... 26 1.6.Testersandcodeofethics(FL1.6)... 27 1.6.1.Public... 27 1.6.2.Customerandemployer... 28 1.6.3.Product... 29 1.6.4.Judgment... 30 1.6.5.Management... 31 1.6.6.Profession... 32 1.6.7.Colleagues... 33 1.6.8.Self... 33 1.7.Synopsisofthischapter... 34 1.8.Sampleexamquestions... 35 Chapter2. Testing Throughout the Software Life Cycle... 43 2.1.Softwaredevelopmentmodels(FL2.1)... 43 2.1.1.Sequentialmodels... 44 2.1.2.Iterativemodels(FL2.1.2)... 47 2.1.3.Incrementalmodel... 49 2.1.4.RAD... 50 2.1.5.Agilemodels... 52 2.1.6.Selectionofadevelopmentmodel... 56 2.1.7.Positioning tests.... 57 2.2.Testlevels(FL2.2)... 58 2.2.1.Componentleveltestingorcomponenttests... 59 2.2.2.IntegrationleveltestingorIntegrationtests... 60 2.2.3.Systemtests... 62 2.2.4.Acceptancetests... 64 2.2.5.Otherlevels... 66 2.3.Typesoftests(FL2.3)... 66 2.3.1.Functionaltests... 67 2.3.2.Non-functionaltests... 68
Table of Contents vii 2.3.3. Tests based on the structure or architecture of the software... 71 2.3.4.Testsassociatedwithchanges... 72 2.3.5.Comparisonsandexamples... 73 2.4.Testandmaintenance(FL2.4)... 74 2.4.1.Maintenancecontext... 75 2.4.2.Evolutivemaintenance... 76 2.4.3.Correctivemaintenance... 76 2.4.4. Retirement and replacement... 77 2.4.5.Regressiontestpolicies... 78 2.4.6.SLAvalidationandacceptance... 81 2.5.Oracles... 81 2.5.1.Problemswithoracles... 81 2.5.2.Sourcesoforacles... 82 2.5.3.Oracleusage... 82 2.6.Specificcases... 83 2.6.1.Performancetests... 83 2.6.2.Maintainability tests... 84 2.7.Synopsisofthischapter... 84 2.8.Sampleexamquestions... 85 Chapter 3. Static Techniques (FL 3.0)... 91 3.1.Statictechniquesandthetestprocess(FL3.1)... 91 3.2.Reviewprocess(FL3.2)... 94 3.2.1.Typesofreviews... 95 3.2.2.Roles and responsibilities during reviews... 100 3.2.3.Phasesofreviews... 103 3.2.4.Successfactorsforreviews... 116 3.2.5.Comparisonofthetypesofreviews... 117 3.3.Staticanalysisbytools(FL3.3)... 119 3.3.1.Typesofstaticanalysis... 120 3.3.2.Typesofdefectsthatcanbeidentified... 125 3.3.3.Dataflowanalysis... 126 3.4.Added value of static activities... 130 3.5.Synopsisofthischapter... 131 3.6.Sampleexamquestions... 131 Chapter 4. Test Design Techniques (FL 4.0).... 137 4.1.Thetestdevelopmentprocess(FL4.1)... 139 4.1.1.Terminology... 139 4.1.2. Traceability... 140 4.2.Categoriesoftestdesigntechniques(FL4.2)... 143 4.2.1.Blackbox,whitebox,orgraybox... 143
viii Fundamentals of Software Testing 4.2.2. Experience-based techniques... 145 4.2.3.Testcharacteristics... 145 4.2.4.Limitationsandassumptions... 146 4.3.Black-boxtechniques(FL4.3)... 147 4.3.1.Equivalence partitioning (FL 4.3.1)... 148 4.3.2.Boundaryvalueanalysis(FL4.3.2)... 155 4.3.3.Decisiontables(FL4.3.3)... 158 4.3.4.Othercombinationaltechniques... 162 4.3.5.State transition testing (FL 4.3.4)... 163 4.3.6.Usecasetesting(FL4.3.5)... 170 4.3.7.Limitationsandassumptions... 172 4.4.Structure-basedtechniques(FL4.4)... 172 4.4.1.Statementtestingandcoverage(FL4.4.1)... 174 4.4.2.Decisiontestingandcoverage(FL4.4.2)... 180 4.4.3.Otherstructure-basedtechniques(FL4.4.3)... 183 4.4.4.MC/DCcoverage(FL4.4.3)... 185 4.4.5.Limitationsandassumptionsofstructure-basedtesting... 186 4.4.6.Coveragelevelandexitcriteria... 187 4.5.Experience-basedtechniques(FL4.5)... 187 4.5.1.Attacks... 188 4.5.2.Defecttaxonomies... 189 4.5.3.Errorguessingandadhoctesting... 190 4.5.4.Exploratorytesting... 190 4.5.5.Limitationsandassumptions... 192 4.6.Choosingtesttechniques(FL4.6)... 193 4.7.Synopsisofthischapter... 194 4.8.Sampleexamquestions... 195 Chapter 5. Test Management (FL 5.0)... 209 5.1.Testorganization(FL5.1)... 209 5.1.1.Independencelevels... 209 5.1.2.Roles and responsibilities... 212 5.1.3.Humanandcontractaspects... 214 5.2.Testplanningandestimation(FL5.2)... 215 5.2.1.Planning and evaluation activities... 218 5.2.2.Test planning activities... 228 5.2.3.Testdocumentation... 229 5.2.4.Entry and exit criteria for test activities... 235 5.3.Testprogressmonitoringandcontrol(FL5.3)... 239 5.4.Reporting... 242 5.4.1.Whattoreport,towhom,andhow?... 242 5.4.2.Statisticsandgraphs... 244
Table of Contents ix 5.5. Transverse processes and activities... 246 5.5.1.Test data definition... 247 5.5.2.Configurationmanagement(FL5.4)... 247 5.5.3.Changemanagement... 248 5.6.Risksmanagement(FL5.5)... 248 5.6.1.Principlesofriskmanagement... 249 5.6.2.Projectrisksandproductrisks... 253 5.6.3.Introductiontoriskmanagement... 254 5.7.Defectmanagement(FL5.6)... 257 5.7.1.Introductiontodefectmanagement... 257 5.7.2.Defectidentification... 258 5.7.3.Actionsappliedtodefects... 263 5.7.4.Defectdisposition... 264 5.8.Synopsisofthischapter... 264 5.9.Sampleexamquestions... 265 Chapter 6. Tools support for Testing (FL 6.0)... 277 6.1.Typesoftesttools(FL6.1)... 278 6.1.1.Testtoolclassification... 278 6.1.2.Toolssupportingtestmanagement... 278 6.1.3.Toolssupportingrequirementmanagement... 279 6.1.4.Toolssupportingstatictests... 279 6.1.5.Modelingtools... 280 6.1.6.Toolssupportingtestdesignandtestdatacreation... 280 6.1.7.Toolssupportingtestexecution... 281 6.1.8.Toolssupportingtestenvironmentmanagement... 281 6.1.9.Toolssupportingtestdatacomparison... 282 6.1.10.Toolssupportingtestcoveragemeasurement... 282 6.1.11.Othertestsupportingtools... 283 6.2.Assumptionsandlimitationsoftesttools(FL6.2)... 283 6.2.1.Advantagesandrisksofthetools... 284 6.2.2.Specificconsiderationsforsometools... 285 6.3.Selectingandintroducingtoolsinanorganization(FL6.3)... 289 6.3.1.Mainprinciples... 290 6.3.2.Toolsselectionprocess... 290 6.3.3.Testtoolimplementation... 293 6.3.4.Tobuildortobuytesttools?... 295 6.4.Synopsisofthischapter... 295 6.5.Sampleexamquestions... 296
x Fundamentals of Software Testing Chapter 7. Mock Exam.... 301 Chapter 8. Templates and Models... 315 8.1.Mastertestplan... 315 8.2.Testplan... 317 8.2.1. Test plan as per IEEE 829-1998... 317 8.2.2. Test plan as per IEEE 829-2008... 317 8.3.Testdesigndocument... 319 8.3.1.Test design specifications as per IEEE 829-1998... 319 8.3.2. Test design document as per IEEE 829-2008... 319 8.4.Testcase... 320 8.4.1.Test case document as per IEEE 829-1998... 320 8.4.2.Test case document as per IEEE 829-2008... 320 8.5.Testprocedure... 321 8.5.1.TestproceduredocumentasperIEEE829-1998... 321 8.5.2.TestproceduredocumentasperIEEE829-2008... 321 8.6.Testlog... 322 8.6.1. Test log as per IEEE 829-1998... 322 8.6.2. Test log as per IEEE 829-2008... 322 8.7.Defectreport... 322 8.7.1.DefectreportasperIEEE829-1998... 322 8.7.2.DefectreportasperIEEE829-2008... 323 8.8.Testreport... 324 8.8.1.Test report as per IEEE 829-1998... 324 8.8.2. Interim test report as per IEEE 829-2008... 324 8.8.3. Level test report as per IEEE 829-2008... 325 8.8.4. Master test report as per IEEE 829-2008... 325 Chapter 9. Answers to the Questions... 327 9.1.Answerstotheendofchapterquestions... 327 9.2.Correctanswerstothesamplepaperquestions... 329 Bibliography... 331 Index... 335