Archive

Title: Test them all, is it worth it? Assessing configuration sampling on the JHipster Web development stack
Authors: Axel Halin, Alexandre Nuttinck, Mathieu Acher, Xavier Devroey, Gilles Perrouin, Benoit Baudry
Publication: Springer Empirical Software Engineering, April 2019, Vol24

A group of software researchers, partially involved in the EU-funded STAMP project, has published interesting results based on configuration tests on JHipster, a popular open source application generator to create Spring Boot and Angular/React projects. 

Abstract: Many approaches for testing configurable software systems start from the same assumption: it is impossible to test all configurations.
This motivated the definition of variability-aware abstractions and sampling techniques to cope with large configuration spaces. Yet, there is no theoretical barrier that prevents the exhaustive testing of all configurations by simply enumerating them if the effort required to do so remains acceptable. Not only this: we believe there is a lot to be learned by systematically and exhaustively testing a configurable system. In this case study, we report on the first ever endeavour to test all possible configurations of the industry-strength, open source configurable software system JHipster, a popular code generator for web applications. We built a testing scaffold for the 26,000+ configurations of JHipster using a cluster of 80 machines during 4 nights for a total of 4,376 hours (182 days) CPU time. We find that 35.70% configurations fail and we identify the feature interactions that cause the errors. We show that sampling strategies (like dissimilarity and 2-wise):
(1) are more effective to find faults than the 12 default configurations used in the JHipster continuous integration;
(2) can be too costly and exceed the available testing budget. We cross this quantitative analysis with the qualitative assessment of JHipster’s lead developers.

Read the full article on Springer website

Date: 20/3/2019
Place: ISTIC, Rennes University, France - Map

Talk title: The Continuous Integration as a partner for test improvement suggestions
Speaker: Caroline Landry (INRIA)

Date: 12 March 2019
Presenter: Benjamin Danglot, Inria researcher
Language: French

With 25 external participants from CGI digital service company, this webinar provided the opportunity to run STAMP toolset demos and to answer to the questions asked by industrial developers, testers and DevOps professionals. 

CGI is a Canadian global information technology (IT) consulting, systems integration, outsourcing, and solutions company headquartered in Montreal, Quebec, Canada. In 1976, Serge Godin and André Imbeau founded CGI, which originally stood for “Conseillers en gestion et informatique” (translating to “Consultants in management and information technology”).
In 2018, the group revenue reached $11.5 billion, up 6.1% or 4.6% in constant currency, with adjusted EBIT of $1.7 billion, or 14.8% of revenue.

Publication: Programmez!
Title: Automated tests to support DevOps teams
Date: March 2019
Authors: Olivier Bouzereau with the STAMP project team

A three pages article about STAMP tools and methodologies has been published in the March 2019 issue of the French printed publication Programmez!  The article explains the growing importance of automated tests, now required for continuous development and continuous integration. New testing tools are improving their speed to deploy more frequent releases of services with less bugs. Amplifying tests in a DevOps approach, the STAMP open source toolset (including DSpot, Descartes, CAMP and Botsing) improves Java test suite and reduce deployed application crashes. Mutation testing examples and use cases are provided in this article. Readers are driven to the beta testing webpage in order to download STAMP tools, try them and give their user experience feedback.

Programmez! March 2019.

This article was commented on Linkedin. 

We are finding new ways to test many configurations with minimum resources