Queen's University - Utility Bar

QSpace at Queen's University >
Theses, Dissertations & Graduate Projects >
Queen's Theses & Dissertations >

Please use this identifier to cite or link to this item: http://hdl.handle.net/1974/1765

Title: Using Code Mutation to Study Code Faults in Scientific Software
Authors: Hook, Daniel

Files in This Item:

File Description SizeFormat
Hook_Daniel_A_200904_MSc.pdf1.14 MBAdobe PDFView/Open
Keywords: Mutation Sensitivity Testing
scientific software
software testing
mutation testing
code faults
code scrutinization
oracle problem
tolerance problem
Issue Date: 2009
Series/Report no.: Canadian theses
Abstract: Code faults can seriously degrade scientific software accuracy. Therefore, it is imperative that scientific software developers scrutinize their codes in an attempt to find these faults. This thesis explores, particularly, the efficacy of code testing as a method of scientific software code fault detection. Software engineers, as experts in code quality, have developed many code testing techniques, but many of these techniques cannot readily be applied to scientific codes for at least two reasons. First, scientific software testers do not usually have access to an ideal oracle. Second, scientific software outputs, by nature, can only be judged for accuracy and not correctness. Testing techniques developed under the assumption that these two problems can be ignored--as most have been--are of questionable value to computational scientists. To demonstrate the reality of these problems and to provide an example of how software engineers and scientists can begin to address them, this thesis discusses the development and application of a novel technique: Mutation Sensitivity Testing (MST). MST is based on traditional mutation testing, but--in place of a focus on mutant "killing"--MST focuses on assessing the mutation sensitivity of a test set. In this thesis, MST experiments are conducted using eight small numerical routines, four classes of mutation operators, and 1155 tests. The results are discussed and some conclusions are drawn. Two observations are of particular interest to computational scientists. First, it is found that oracles that exhibit uncertainties greater than (approximately) 80% of the expected output are of questionable value when they are used in the testing of scientific software. Second, it is found that a small number of carefully selected tests may be sufficient to falsify a code.
Description: Thesis (Master, Computing) -- Queen's University, 2009-04-19 13:34:08.943
URI: http://hdl.handle.net/1974/1765
Appears in Collections:Computing Graduate Theses
Queen's Theses & Dissertations

Items in QSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

 

  DSpace Software Copyright © 2002-2008  The DSpace Foundation - TOP