Skip to content
javahacks.dev
JUnit

Fix JUnit 5 Tests Not Running in Maven Surefire Builds

Fix a Maven or Gradle build that reports zero tests by checking the Surefire version, test class names, imports and the JUnit Platform setting.

5 min read
Summarize in AI
Flat illustration of a log with a failing line and a wrench, with the JUnit5, Apache Maven and Docker logos.

BUILD SUCCESS and Tests run: 0 is the most misleading result a build produces. The suite is green because it is empty:

Text
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

The causes are ordered below from the most common. How the three JUnit parts relate, and why one missing piece produces silence, is in JUnit 5 architecture. The framework itself is in the JUnit 5 handbook.

Cause 1: an old Surefire plugin

Surefire added JUnit Platform support in 2.22.0. An older version uses its JUnit 4 provider and does not see Jupiter tests. Spring Boot's parent POM manages a current version; a project with its own parent may not.

pom.xmlXML
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.5.4</version>
</plugin>

Check the version in use with ./mvnw help:effective-pom | grep -A2 surefire. Use a current 3.x release.

Cause 2: the class name does not match

Surefire's default includes are **/Test*.java, **/*Test.java, **/*Tests.java and **/*TestCase.java. A class called OrderSpec or OrderCheck is not run. A class called OrderServiceIT runs in the Failsafe integration-test phase, not in mvn test. Rename the class, or set the includes in the plugin configuration.

Cause 3: the wrong @Test import

java
import org.junit.Test;                       // JUnit 4: ignored by the Jupiter engine
import org.junit.jupiter.api.Test;           // JUnit 5: this one

IDE auto-import picks the first match, and a project with both on the classpath offers both. A Jupiter class that imports the JUnit 4 annotation is invisible to the Jupiter engine; the JUnit 4 provider needs the JUnit 4 runner. The related error message appears in no runnable methods.

Cause 4: Gradle needs to be told

build.gradle.ktskotlin
tasks.test {
    useJUnitPlatform()
}

Without that line Gradle uses the JUnit 4 runner. Tests compile and nothing runs. Add the engine dependency too: testRuntimeOnly("org.junit.platform:junit-platform-launcher") is needed on recent Gradle versions.

Cause 5: the engine is not on the test runtime classpath

Declaring only junit-jupiter-api compiles the tests and leaves no engine to run them, depending on build tool and version. Use the aggregate artifact, which brings the API, the engine and the parameterized-test support:

XML
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter</artifactId>
    <scope>test</scope>
</dependency>

Cause 6: test methods that cannot run

A test method that is private or static, a class that is abstract, or one disabled by @Disabled is skipped. Jupiter does not require public, but the method must be non-private, non-static and return void. With Maven, -DskipTests and -Dmaven.test.skip=true skip silently; look for them in the command, in .mvn/maven.config and in the CI settings.

See what Surefire decided

Shell
./mvnw -X test | grep -i "provider\|junit"

The debug output names the provider: surefire-junit-platform is the one you want, and surefire-junit4 means it picked the wrong engine.

Failsafe, the IT suffix and the verify phase

A project that follows the common convention names slow tests *IT and runs them with the Failsafe plugin. mvn test never runs those, and that is by design: Failsafe binds to integration-test and verify. A team that moves a test from OrderServiceTest to OrderServiceIT and then runs mvn test sees the count drop and no error. Run ./mvnw verify to include both, and check that the Failsafe plugin is declared at all, because without it *IT classes are not run by anything.

XML
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>3.5.4</version>
    <executions>
        <execution>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

The verify goal matters as much as integration-test. It reads the results and fails the build, so leaving it out gives a build that runs the tests and ignores the outcome.

Nested classes and the exclusion that looks wrong

Surefire excludes class names containing a $ by default, which removes inner classes. JUnit 5 @Nested classes are inner classes, so a reader may expect them to vanish. They do not, because the Jupiter engine discovers them through the enclosing class and the include pattern only has to match the outer name. If the outer class is named OrderServiceSpec, it is not selected, and neither are its nested classes. Rename the outer class and the nested ones return with it.

Filters that leave zero tests

A -Dtest filter that matches nothing makes Surefire fail with No tests matching pattern, which is loud. A group filter is quieter. Tags are the usual one:

XML
<configuration>
    <groups>fast</groups>
</configuration>

With <groups>fast</groups>, only methods and classes tagged @Tag("fast") run. If a configuration carries a stale tag name, the build reports Tests run: 0 and succeeds. Search the POM and the CI scripts for groups, excludedGroups and -Dgroups, and compare against the tags actually used in the code.

Mixing JUnit 4 and JUnit 5 in one module

A module that still has JUnit 4 tests needs the Vintage engine next to Jupiter, because the Jupiter engine ignores org.junit.Test:

XML
<dependency>
    <groupId>org.junit.vintage</groupId>
    <artifactId>junit-vintage-engine</artifactId>
    <scope>test</scope>
</dependency>

Without it, the JUnit 4 tests are skipped with no warning once Surefire switches to the platform provider. The reverse also happens: a module with only the Vintage engine on the classpath runs JUnit 4 tests and ignores every Jupiter test. When the numbers in the report fall after a dependency change, check which engines are present with ./mvnw dependency:tree -Dincludes=org.junit.*.

Check the report files, not just the console

Surefire writes one XML and one text file per class into target/surefire-reports. A class that is missing from that directory was never selected. A class whose file shows Tests run: 0 was selected and had nothing to run, which usually means every method failed the visibility or annotation check. Listing the directory gives a quick answer:

Shell
ls target/surefire-reports | head
grep -l 'tests="0"' target/surefire-reports/*.xml

A decision order that finds it fast

SymptomFirst check
Tests run: 0 for the whole moduleProvider in -X output, then the engine dependency
Some classes missing from the reportClass name against the include patterns
Only *IT classes missingFailsafe declared, and verify used instead of test
Count dropped after a dependency changeVintage engine, JUnit 4 imports
Works in IDE, nothing on CImaven.config, -DskipTests and groups in the CI command

Start with the debug output, because it names the provider and the reason a class was filtered, and the other four checks are guesses without it.

Where this does not apply

If tests run in the IDE and not in the build, the IDE has its own runner; compare with the command-line result. If a test count is lower than expected, parameterized tests count each invocation as a test, while a dynamic factory can hide cases. The Surefire documentation on JUnit Platform and the Gradle test documentation are the primary references.

Also inDevOps
Share

Keep reading

Get new posts by email

We send one email when a new post is published. No spam.

Unsubscribe in one click. No spam, ever.