How to Prevent Skipped Tests in Nightly Builds with ArchUnit

Once, I worked on a project where many test classes were unintentionally skipped during nightly builds due to incorrect naming. This happened because Surefire (used for running tests in Maven) is typically configured to recognize test classes based on their naming suffixes. If a class name doesn't match the expected pattern, it gets ignored.

For example, consider this test class:

public class ExampleTes {  

    @Test  
    void testSum() {  
    }  
}        

The class ExampleTes would be skipped during the nightly build because it doesn't follow the correct naming convention. Surefire expects the name to end with "Test", but ExampleTes is missing the last letter. As a result, Surefire doesn't recognize it as a test class.

You can configure Surefire to recognize test classes based on specific naming patterns in your pom.xml. For example, the following configuration tells Surefire to recognize classes ending with "Test", "IT", or "ST":

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${maven.surefire.plugin.version}</version>
    <configuration>
        <includes>
            <include>**/*Test.java</include>
            <include>**/*IT.java</include>
            <include>**/*ST.java</include>
        </includes>
    </configuration>
</plugin>        

With this configuration, Surefire will only run classes whose names end in "Test", "IT", or "ST". Any class that doesn't match these patterns will be skipped.

To prevent test classes from being skipped due to incorrect naming, you can use ArchUnit to enforce a naming convention. For example, you can define an ArchUnit rule requiring that any class containing a method annotated with @Test must have a name ending in IT:

import com.tngtech.archunit.base.DescribedPredicate;
import com.tngtech.archunit.core.domain.JavaClass;
import com.tngtech.archunit.core.domain.JavaField;
import com.tngtech.archunit.core.domain.JavaMethod;
import com.tngtech.archunit.junit.ArchTest;
import com.tngtech.archunit.lang.ArchRule;
import com.tngtech.archunit.lang.syntax.ArchRuleDefinition;
import org.junit.jupiter.api.Test;

public class TestClassNamingRule {

  public static ArchRule testClassesShouldFollowNamingConvention(String basePackage, String testClassSuffix) {
    return ArchRuleDefinition.classes()
        .that(new TestMethodPredicate())  // ? Uses the fixed predicate
        .and().resideInAPackage(basePackage + "..")
        .should().haveSimpleNameEndingWith(testClassSuffix)
        .because("Test classes should have names ending with '" + testClassSuffix + "' if they contain @Test methods");
  }

  // ? Inner class without "public" (fixes file name issue)
  static class TestMethodPredicate extends DescribedPredicate<JavaClass> {

    public TestMethodPredicate() {
      super("classes containing methods annotated with @Test");
    }

    @Override
    public boolean test(JavaClass javaClass) {  // ? Correct method name
      return javaClass.getMethods().stream().anyMatch(this::isTestMethod) ||
          javaClass.getFields().stream().anyMatch(this::isArchunitField);
    }

    private boolean isArchunitField(JavaField javaField) {
      return javaField.isAnnotatedWith(ArchTest.class);
    }

    private boolean isTestMethod(JavaMethod method) {
      return method.isAnnotatedWith(Test.class);
    }
  }
}        

import com.tngtech.archunit.junit.AnalyzeClasses;
import com.tngtech.archunit.junit.ArchTest;
import com.tngtech.archunit.lang.ArchRule;
import org.expectedbits.test.utils.archunit.TestClassNamingRule;

@AnalyzeClasses(packages = "org.expectedbits.test.it")
public class ArchUnitIT {

  @ArchTest
  static final ArchRule test_classes_naming =
      TestClassNamingRule.testClassesShouldFollowNamingConvention("org.expectedbits.test.it", "IT");
}        

This ArchUnit rule checks that all classes containing @Test methods in the org.expectedbits.test.it package must end with "IT". This ensures your test classes follow the naming conventions and prevents skipped tests due to naming errors.

If you're unsure about how to define such rules, ChatGPT can help you generate ArchUnit rules based on your specific requirements. Just describe the rule you'd like to enforce (for example, class naming conventions), and ChatGPT can generate the appropriate code for you!

By using this strategy, you can make sure all test classes are recognized and executed by Surefire, reducing the risk of missing tests due to naming mistakes.