Hot questions for Using Cucumber in junit5

Question:

For test framework were used next stack of technologies: Java, Maven, Selenium, Junit, Cucumber, Spring Boot, YAML

cucumber.version = 5.4.0 Cucumber-JVM now has JUnit5 support and we can use parallel I have tried to add -Dcucumber.execution.parallel.enabled=true -Dcucumber.execution.parallel.config.strategy=dynamic

https://github.com/cucumber/cucumber-jvm/blob/master/release-notes/v5.0.0.md

was used :

            <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${maven-surefire-plugin.version}</version>
            <configuration>
                <includes>
                    <include>**/RunCucumberIT.java</include>
                    <release>11</release>
                </includes>
                <!--                    <parallel>methods</parallel>-->
                <!--                    <threadCount>4</threadCount>-->
            </configuration>
        </plugin>

Answer:

You can provide options by adding a junit-platform.properties file to your classpath root. For example:

src/test/resources/junit-platform.properties

cucumber.execution.parallel.enabled=true
cucumber.execution.parallel.config.strategy=fixed
cucumber.execution.parallel.config.fixed.parallelism=10

You can also pass options to the JUnit Platform via Surefires/Failsafes configurationParameters parameter field.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>3.0.0-M3</version>
    <configuration>
        <properties>
            <configurationParameters>
                cucumber.execution.parallel.config.fixed.parallelism=24
            </configurationParameters>
        </properties>
    </configuration>
</plugin>

And because Cucumber is a JUnit Platform Engine you can also use any of the other ways you'd pass configuration parameter to the JUnit Platform.

Note that -D will not work because surefire starts a new JVM, you'd have to use `-DargLine='....' for this.

Question:

Consider this simple project: https://github.com/ekalin/cucumber-junit5-gradle. It contains a Cucumber feature, using JUnit5.

Following the instruction from https://github.com/cucumber/cucumber-jvm/tree/master/junit-platform-engine, I’ve created a class anotated with @Cucumber so that tests can be discovered. And when I run ./gradlew test, the feature is discovered and run.

But if I try to select only that test with ./gradlew test --tests cucumber.junit5.gradle.SampleSteps (or with other ways to select the test), I get this error:

Execution failed for task ':test'.
> No tests found for given includes: [cucumber.junit5.gradle.SampleSteps](--tests filter)

I’ve tried adding includeEngines("cucumber") to useJUnitPlatform but it didn’t make any difference.

Is there some configuration to allow selecting a single test when it’s using a different engine?


Answer:

Gradle does not yet support discovery or filtering of non-class based tests (see: gradle/#4773). Instead Gradle asks the JUnit Platform to discover tests in all classes in the test package. The @Cucumber annotation is a work around to discover all feature files in the package of the annotated class.

When you use --tests cucumber.junit5.gradle.SampleSteps you apply a class based filter on all test results. Because the @Cucumber annotated class is a work around it is not part of the discovered tests and because feature files do not have a ClassSource but rather a FileSource gradle will exclude them all.

The best thing you can currently do is use the work around suggested in gradle/#4773. You may also want to request the Gradle team to provide a better integration with the JUnit Platform so all its feature can be used.

Question:

I'm trying to run Cucumber features in JUnit 5 Jupiter. I've lifted some code from the Cucumber-jvm source and adapted it for JUnit 5's TestFactory. It is working: I see my features running when I run all JUnit tests (this is Kotlin code, but the same applies to Java):

@CucumberOptions(
        plugin = arrayOf("pretty"),
        features = arrayOf("classpath:features")
)
class Behaviours {
    @TestFactory
    fun loadCucumberTests() : Collection<DynamicTest> {
        val options = RuntimeOptionsFactory(Behaviours::class.java).create()
        val classLoader = Behaviours::class.java.classLoader
        val resourceLoader = MultiLoader(classLoader)
        val classFinder = ResourceLoaderClassFinder(resourceLoader, classLoader)
        val runtime = Runtime(resourceLoader, classFinder, classLoader, options)
        val cucumberFeatures = options.cucumberFeatures(resourceLoader)
        return cucumberFeatures.map<CucumberFeature, DynamicTest> { feature ->
            dynamicTest(feature.gherkinFeature.name) {
                var reporter = options.reporter(classLoader)
                feature.run(options.formatter(classLoader), reporter, runtime)
            }
        }
    }
}

However, JUnit reports that every feature was successful, whether or not it actually was. When features fail, the results are correctly pretty-printed, but the generated DynamicTest passes. Neither gradle test nor Intellij notice the error: I have to inspect the text output.

I think I have to figure out, in the Executable passed as the second parameter to dynamicTest, what the result of the feature was, and raise an assertion when appropriate. How do I determine the result of feature or feature.gherkinFeature at that point?

And is there a way to get at the results for each scenario in the feature? Or better, is there a way to run a specific scenario, so that I can create a DynamicTest for each scenario, giving me better reporting granularity in JUnit?


Answer:

To record the result of a Cucumber scenario as a JUnit5, I found it easiest to implement a JunitLambdaReporter, which is essentially a simpler version of the existing JunitReporter. Once you have a reporter that remembers what the current scenario is, then you can create a @TestFactory that uses this logic:

return dynamicTest(currentScenario.getName(), () -> {
  featureElement.run(formatter, reporter, runtime);
  Result result = reporter.getResult(currentScenario);

  // If the scenario is skipped, then the test is aborted (neither passes nor fails).
  Assumptions.assumeFalse(Result.SKIPPED == result);

  Throwable error = result.getError();
  if (error != null) {
    throw error;
  }
});