When your deployment artefact is a Docker image, you should system test against a container based on that image. In this blog post, I will demonstrate how to get test coverage for a JVM app in that scenario.

Introduction

Test coverage is a useful metric to help you analyse which parts of your app are touched by tests. In the JVM world, this is usually done by using JaCoCo which provides an agent that records calls to your code.

When the app runs inside a Docker container, instrumentation is a bit more complicated but doable.

I will use the project from a previous article ‘Efficient docker images for Spring Boot applications‘, to show you what is necessary to determine test coverage.

Source code: https://github.com/akquinet/efficient-spring-boot-docker/tree/coverage

Setup

The project is written in Kotlin and already contains a system test that starts the Docker image and makes a call to its HTTP API:

class DemoApplicationIT {
 
    @get:Rule
    var appContainer = KGenericContainer("spring-docker-demo:latest")
            .waitingFor(Wait.forListeningPort())
            .withExposedPorts(8080)
    val client: RestTemplate = RestTemplateBuilder().build()
 
    @Test
    fun `can call ping`() {
        val url = "http://localhost:${appContainer.getMappedPort(8080)}/ping"
        val response = client.getForEntity(url, String::class.java)
        assertThat(response.statusCode).isEqualTo(HttpStatus.OK)
        assertThat(response.body).isEqualTo("pong")
    }
}

To show that the coverage report is correctly showing covered as well as uncovered code, I added a second controller method which is not called in the test.

@RestController
class RoutingConfiguration {
 
    @GetMapping("/ping")
    fun ping(): String = "pong"
 
    @GetMapping("/unused")
    fun unused(): String = "pong"
}

In order to generate the coverage, the following three steps are necessary:

1. Make the JaCoCo agent available inside the container.
2. Start the app with the JaCoCo agent.
3. Access the coverage report.

Make the JaCoCo agent available inside the container

We define a property for the JaCoCo version that we will use for the maven plugin as well as the dependency on the agent. Using the dependency-plugin, we copy the agent to target/jacoco-agent removing the version from its name.

Start the app with the JaCoCo agent

We use Testcontainers to start our image. If you use anything else, like e.g. docker-compose, the idea is the same.

We mount two volumes into the container. The first one will contain the agent library at location /jacoco-agent, while the location /jacoco-report will be used by the agent to save the generated coverage file.

To enable the agent, we override the CMD with:

[
  "/usr/bin/java",
  "-javaagent:/jacoco-agent/org.jacoco.agent-runtime.jar=destfile=/jacoco-report/jacoco-it.exec",
  "-jar",
  "/app.jar"
]

Using the Testcontainer API, the setup looks like this:

@get:Rule
var appContainer = KGenericContainer("spring-docker-demo:latest")
        .waitingFor(Wait.forListeningPort())
        .withExposedPorts(8080)
        .withFileSystemBind("./target/jacoco-agent", "/jacoco-agent")
        .withFileSystemBind("./target/jacoco-report", "/jacoco-report")
        .withCommand("/usr/bin/java",
                "-javaagent:/jacoco-agent/org.jacoco.agent-runtime.jar=destfile=/jacoco-report/jacoco-it.exec",
                "-jar",
                "/app.jar")

When executing mvnw verify, we will find a coverage file at target/jacoco-report/jacoco-it.exec.

Access the coverage report

The bad news is, this file is way too small and does not contain the desired coverage data. The problem here is that the JaCoCo agent only writes down the result when the JVM exits. As Testcontainers kills the container without giving it time for a graceful shutdown, the agent cannot write its result to the file.

As a workaround, we stop the container ourselves, which was previously done by the JUnit rule.

@After
fun stopContainerGracefully() {
    appContainer.dockerClient
        .stopContainerCmd(appContainer.containerId)
        .withTimeout(10)
        .exec()
}

Now the JaCoCo plugin can generate a nice coverage report.

Configure it like this to find a report in target/site/jacoco-it/index.html:

If you use SonarQube and don’t need the html report, you can skip the jacoco-maven-plugin completely. The report looks like this:

Image may be NSFW.
Clik here to view.

Conclusion