Metamorphic testing of constraint solvers
Abstract
Constraint solvers are complex pieces of software and are notoriously difficult to debug. In large part this is due to the difficulty of pinpointing the source of an error in the vast searches these solvers perform, since the effect of an error may only come to light long after the error is made. In addition, an error does not necessarily lead to the wrong result, further complicating the debugging process. A major source of errors in a constraint solver is the complex constraint propagation algorithms that provide the inference that controls and directs the search. In this paper we show that metamorphic testing is a principled way to test constraint solvers by comparing two different implementations of the same constraint. Specifically, specialised propagators for the constraint are tested against the general purpose table constraint propagator. We report on metamorphic testing of the constraint solver Minion. We demonstrate that the metamorphic testing method is very effective for finding artificial bugs introduced by random code mutation.
Citation
Akgun , O , Gent , I P , Jefferson , C A , Miguel , I J & Nightingale , P W 2018 , Metamorphic testing of constraint solvers . in J Hooker (ed.) , Principles and Practice of Constraint Programming : 24th International Conference, CP 2018, Lille, France, August 27-31, 2018, Proceedings . Lecture Notes in Computer Science , vol. 11008 , Springer , pp. 727-736 , 24th International Conference on Principles and Practice of Constraint Programming (CP 2018) , Lille , France , 27/08/18 . https://doi.org/10.1007/978-3-319-98334-9_46 conference
Publication
Principles and Practice of Constraint Programming
ISSN
0302-9743Type
Conference item
Rights
© 2018, Springer Nature Switzerland. This work has been made available online in accordance with the publisher’s policies. This is the author created accepted version manuscript following peer review and as such may differ slightly from the final published version. The final published version of this work is available at https://doi.org/10.1007/978-3-319-98334-9_46 _ 46 _ 1
Description
Funding: EPSRC EP/P015638/1 and EP/P026842/1. Dr Jefferson holds a Royal Society University Research Fellowship.Collections
Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.