Java – Gradle compiles but does not run TestNG tests


We just started using Gradle and TestNG on our project, so I am checking whether failing any test actually fails the build. I was quite surprised to see it does not. The tests are picked up and compiled correctly, so I see the class files. Also I do get a report of the run, but it says 0 tests (expected 2). Running gradle clean test -i gives me the following:

:contentplatform-service:compileTestJava (Thread[Daemon worker Thread 7,5,main])
Executing task ':contentplatform-service:compileTestJava' (up-to-date check took
 0.08 secs) due to:
  Output file D:\Dev\contentplatform-service\build\classes\test has changed.
  Output file D:\Dev\contentplatform-service\build\dependency-cache has changed.

  Output file D:\Dev\contentplatform-service\build\classes\test\nl\xillio\conten
tplatform\service\SuperSimpleTest.class has been removed.
All input files are considered out-of-date for incremental task ':contentplatfor
Compiling with JDK Java compiler API.
:contentplatform-service:compileTestJava (Thread[Daemon worker Thread 7,5,main])
 completed. Took 0.229 secs.
:contentplatform-service:processTestResources (Thread[Daemon worker Thread 7,5,m
ain]) started.
Skipping task ':contentplatform-service:processTestResources' as it has no sourc
e files.
:contentplatform-service:processTestResources UP-TO-DATE
:contentplatform-service:processTestResources (Thread[Daemon worker Thread 7,5,m
ain]) completed. Took 0.001 secs.
:contentplatform-service:testClasses (Thread[Daemon worker Thread 7,5,main]) sta
Skipping task ':contentplatform-service:testClasses' as it has no actions.
:contentplatform-service:testClasses (Thread[Daemon worker Thread 7,5,main]) com
pleted. Took 0.001 secs.
:contentplatform-service:test (Thread[Daemon worker Thread 7,5,main]) started.
Executing task ':contentplatform-service:test' (up-to-date check took 0.049 secs
) due to:
  Output file D:\Dev\contentplatform-service\build\test-results\binary\test has
  Output file D:\Dev\contentplatform-service\build\test-results has changed.
  Output file D:\Dev\contentplatform-service\build\reports\tests has changed.
Finished generating test XML results (0.0 secs) into: D:\Dev\contentplatform-ser
Generating HTML test report...
Finished generating test html results (0.014 secs) into: D:\Dev\contentplatform-
:contentplatform-service:test (Thread[Daemon worker Thread 7,5,main]) completed.
 Took 0.194 secs.

package nl.xillio.contentplatform.service;

import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

public class SuperSimpleTest {

    public void setUp() {
        // code that will be invoked when this test is instantiated

    public void testTest() {
        Assert.assertEquals(true, true);

build.gradle contains:

test {
    // enable TestNG support (default is JUnit)
    scanForTestClasses = false  
    include '**/*'

    testLogging {
        showStandardStreams = true

        // log results to "build/test-results" directory
        exceptionFormat "full"
        events "started", "passed", "skipped", "failed", "standardOut", "standardError"

I already took a look at other questions about this topic, and there I found the hint to use scanForTestClasses = false as a workaround (see However, this problem seems unrelated. Am I making some other noob mistake here? How can I get SuperSimpleTest to just execute?

UPDATE: I tried forcing gradle to run a specific test with an interesting error as a result: Gradle complains about JUnit version on TestNG task

Best Solution

It was a stupid noob mistake indeed. We are using a multi-project setup with a master project and several folders on the same level, which contain the code and the tests. The build.gradle I was using accidentally only configured the master project to use TestNG. The solution is to include the test target within the allprojects or subprojects closure. Which is explained here: What is the difference between allprojects and subprojects

So here is the working build.gradle:

def mainClassName = 'nl.xillio.contentplatform.view.Run'

// Load settings for all projects including master and subprojects
allprojects {
    apply plugin: 'java'
    apply plugin: 'eclipse'

    version '0.1'

    repositories {
        maven {
            url ''
        maven {
            url ''

// Load the dependencies for all subprojects (layers)
subprojects {
    dependencies {
    //to do: move these to the correct subprojects
        compile 'javax.inject:javax.inject:1'
        compile 'org.springframework:spring-context:4.1.4.RELEASE'
        compile 'org.springframework:spring-core:4.1.4.RELEASE'
        compile 'org.springframework:spring-beans:4.1.4.RELEASE'
        compile 'commons-logging:commons-logging:1.2'
        testCompile 'org.springframework:spring-test:4.1.4.RELEASE'
        testCompile 'org.testng:testng:6.1.1'

    test {
        // enable TestNG support (default is JUnit)

// Resolve the dependencies between the layers in the individual project's build.gradle files
// add ONLY specific per project behaviour of the GLOBAL build here:

project(':contentplatform-web') {

project(':contentplatform-service') {

project(':contentplatform-dao') {

// Return a list of all the external libraries
def getLibraries() {
    return configurations.runtime.filter{!'contentplatform')}

// Copy all the libraries to a libs folder
task copyLibraries(type: Copy) {
    group 'Content Platform'
    description 'Copy all the external libraries to the /libs folder.'

    destinationDir file('./build/')

    into('libs/') {
        from getLibraries()

// Perform the build task before building the big jar
jar {
    group 'Content Platform'
    description 'Package all the layers and dependencies into a big jar.'

    // The libraries are required to build

    // The final big jar needs all the layers
    dependencies {
        compile project(':contentplatform-web'), project(':contentplatform-dao'),
            project(':contentplatform-service'), project(':contentplatform-model')
    // Create a MANIFEST.MF file, the main class file is in the web layer
    manifest {
        attributes 'Implementation-Title': 'Content Platform',  
            'Implementation-Version': version,
            'Built-By': System.getProperty(''),
            'Built-Date': new Date(),
            'Built-JDK': System.getProperty('java.version'),
            'Class-Path': getLibraries().collect{'../libs/' + it.getName()}.join(' '),
            'Main-Class': mainClassName

    // Include the layers in the fat jar
    from(configurations.compile.filter{'contentplatform')}.collect{it.isDirectory() ? it : zipTree(it) }) {
        exclude "META-INF/*.SF"
        exclude "META-INF/*.DSA"
        exclude "META-INF/*.RSA"

    // Save the fat jar in the root of the folder instead of in build/libs
    destinationDir file('.')