|
Testing Tool : LBTestResearch LeaderGraduate StudentsResearch Engineers
Masters Students
Short Tool DescriptionLBTest is a new open source tool for black-box requirements testing of embedded and reactive systems based on the principles of learning-based testing (LBT).Learning-based testing is an emerging paradigm for black-box requirements testing that fully automates the three core activities of: (1) test case generation (ATCG), (2) test execution, and (3) test verdict construction (the test oracle). Technical details can be found in e.g. Meinke and Sindhu 2011, Meinke and Sindhu 2013, and Meinke, Niu and Sindhu 2012.
Model-based Testing without a ModelThe basic idea of LBT is to use machine learning methods to reverse engineer a model of the system under test (SUT). We then apply model-based testing methods, such as model checking, to this structure.Since LBTest uses black-box learning, it can even be applied to testing third-party software.
Technical and Process AdvantagesReverse engineering methods are well suited to agile development processes, since the learned model is always highly synchronised with the current software implementation. LBTest avoids the problem of false positive and false negative test results due to a mismatch between a model and its implementation. LBTest users can avoid time consuming (manual) model construction and model updates, and instead raise testing productivity through a high degree of automation.LBTest can be used without any commitment to model-driven development, as it is a standalone tool which only requires a set of user requirements and an executable SUT.
State Space CoverageMachine learning and model checking are tightly integrated with the SUT in an iterative feedback loop which dynamically optimizes test case generation on-the-fly, based on previous test outcomes. This approach can achieve high state-space coverage levels in a relatively short time.
Requirements Modelling and TestingLBTest uses a formal requirements modelling language known as linear temporal logic (LTL). This is an expressive logical language that can be used to describe both safety and liveness properties of embedded systems at the unit, integration and system levels. Other requirements modeling languages are planned for future releases.Test verdicts are automatically constructed on-the-fly by comparing predicted output behaviours with observed output behaviours.
Tool Chain IntegrationWork is in progress to make LBTest OSLC compliant, in order to support flexible integration with existing tool chains.
FundingLBTest was originally developed within the EU FP7 FET project Highly Adaptable and Trustworthy Software using Formal Models HATS and is currently being further developed within the ARTEMIS JU project Model Based Analysis and Testing of Embedded Systems MBAT and the Swedish national VINNOVA project Virtualized Embedded Systems for Testing and Development VIRTUES.Related PublicationsK. Meinke and M. Sindhu, Incremental Learning-Based testing for Reactive Systems, pp 134-151 in: Proc. Int. Conf. on Tests and Proofs TAP 2011, LNCS 6706, Springer Verlag, 2011.
K. Meinke, F. Niu and M. Sindhu: Learning-Based Software Testing: a Tutorial, pp 200-219 in: R. Haehnle et al. (eds) Proc. ISoLA 2011, Int. Symp. on Leveraging Applications of Formal Methods, Verification, Validation, CCIS 336, Springer Verlag, 2012.
K. Meinke and M. Sindhu: LBTest: A Learning-based testing Tool for Reactive Systems, in Proc. Sixth Int. IEEE Conf. on Software Testing, Verification and Validation (ISCT-2013), 2013.
L. Feng, S. Lundmark, K. Meinke, F. Niu, M.A. Sindhu, P.Y.H. Wong: Case Studies in Learning-based Testing, pp. 164-179 in Proc. 25th IFIP Int. Conf. on Testing Software and Systems (ICTSS 2013), LNCS 8254, Springer Verlag, 2013.
LinksLBTest will soon be made available for evaluation purposes under an open source GPL license at the LBTest tool homepage. |
