Mutation testing can serve as indicators on test quality and relevance
How do you see mutation testing tools being adopted in business projects, as an effective method with significant benefits on distributed software updates?
I think mutation testing has its uses in specialized settings, where the software is small (or well-modularized). However, many companies struggle with the fact that they can't keep up with testing, so we also see the opposite (test selection), where tests are not analyzed as much in depth, but instead, the goal is to be as light-weight as possible. From this point of view, mutation techniques should in general also be efficient and light-weight.
Do you expect mutation testing to improve software quality and maintainability?
Mutation testing can serve as indicators on test quality and relevance, so even if used only periodically, it could have its uses. For example, tests that are ineffective no longer have to be maintained.
Do you see a potential complementarity between mutation tests and model-based test tools? For instance, could software-defined networks or IoT applications benefit from the combinaison of Modbat and Descartes?
We already use mutation testing in this context, to see how effective a tool or model is at detecting defects. Usually, there are not enough real defects to go around, or it is time-consuming to find enough failing unit tests in a bug tracker that still work today. So it is easier to generate artificial failures. Of course, they may not have all the characteristics of the real thing. I see mutation testing as a good way to test not only the quality of the inputs, but especially the quality of the test oracle as well.
Business software are now based on a DevSecOps approach, using open source infrastructure stacks. Do you see any downside in this trend?
Many frameworks are relying on shared open source software. The open collaborations have greatly improved the quality of distributed software. The only downside I can see: if you have security problems now, thousands of companies are affected. During the last ten years, the open source mouvement has grown, with a natural selection of the best components. Many OSS innovations are supported by the industry. However, I wish industrials share their concerns and let the community know when they fix software issues. Most of the companies are still too secretive. They often use open source software, but don’t contribute back. More mutual relationship would be beneficial to improve the quality of the codes. Also the contributing organisations would become more attractive as employers.
A word about yourself and your organization
Since August 2016, I am Associate Professor at the KTH Royal Institute of Technology in Stockholm. My main interests are software verification and software engineering. During my Master's thesis and my Ph.D. thesis at ETH Zurich, I investigated different approaches for finding faults in multi-threaded programs. I worked for two years at Tokyo's National Institute of Informatics as a Postdoctoral Researcher. Then, from April 2007 to July 2016, I worked as Senior Researcher at the National Institute of Advanced Industrial Science and Technology (AIST) in Tokyo and Osaka. In recent work, I extended my work on concurrent software to networked programs and I developed my own model-based test tool Modbat, which is specialized for generating tests for networked software.