Prerequisites

You need to have:

• JDK 8 or later
• git
• Apache Maven

What is it about ?

Descartes performs "extreme mutation" : it removes code in a software project, then passes the JUnit test suite.

Are you sure your tests fail if your code is removed ? Sounds like common sense... but in most real-life projects, many tests don't fail when code vanishes ! Descartes detects that, and tells you where it happens, so you can fix.

DSpot generates JUnit tests to complete an existing suite (it looks for missing assertions). In complement with Descartes, it can help detect some mutations that existing tests did not cover.

In this tutorial, you will run Descartes on a sample project to detect flaws in tests, then use DSpot to fix some of them.

Check your unit tests effectiveness with Descartes

As described in Descartes home page, you can evaluate the capability of your test suite to detect bugs using extreme mutation testing.

First of all, clone the example project DHell from STAMP official repository:

Then go to the DHell project folder:

You will find several pom.xml files each preconfigured to try the tools with specific configurations.

To try Descartes quickly, you can run the shell script run_pitest_descartes.sh.

The console output will show you the effect of applying Descartes to a Java project:

Descartes generates a detailed HTML output, available in target/pit-reports: there, you will find a sub-directory (its name is the date/time of the moment Descartes was run, like target/pit-reports/201907181115/ ). Open the "index.html" file that you find there, in your web browser:

This is Descartes statistics: browse the HTML files, to see what Descartes discovered.

For example, in HelloApp.java, line 124, method "returnSomething()" can be emptied (its content replaced by "return 1"), and JUnit tests detect nothing: you are prompted to fix the corresponding tests !

If you want to understand better how to use Descartes, you can open the run_pitest_descartes.sh script in the Leafpad available in the VM and study the content:

Amplify your unit tests with DSpot

As described in DSpot home page, you can enhance the capability of your test suite to detect bugs applying DSpot to it. DSpot will generate missing assertions to your test cases, taking them in input as well as your Java project.

If you still haven't done, clone the example project DHell from STAMP official repository:

Then go to the DHell project folder:

You will find several pom.xml files each preconfigured to try the tools with specific configurations.

To try DSpot quickly, you can run the shell script *run_dspot.sh.

The console output will show you the effect of applying Descartes to a Java project:

Now, you can just copy the test generated by DSpot to enhance the JUnit test suite:

Then run the JUnit test suite:

Run Descartes again to see what's been fixed

To compare results, save the content of target/pit-reports somewhere (as the next "mvn clean" would delete it - and Descartes scripts do that).

Then, as before, run the shell script run_pitest_descartes.sh again.

Have a look at target/pit-reports: the statistics are better than the 1st time, because DSpot fixed something !