Hot questions for Using Cucumber in allure

Question:

I am able to generate allure report in a java-cucumber-Junit based Project. However, I am unable to get the cucumber steps in the execution section. Also, The attachments are not getting attached to the reports. I am able to generate the Allure report based on the steps given on Allure site. However, I am unable to see the steps(Given, when then) in the execution section of the report. Also, The attachments are also not generated in the report.

Runner class snippet

    @RunWith(Cucumber.class)
    @CucumberOptions(features = { "src/test/java/features" }, 
    plugin = { "pretty", "html:target/cucumber-html-reports",
            "json:target/cucumber-html-reports/cucumber.json","rerun:target/failed_scenarios.txt" }, monochrome = true, glue = { "definitions" }, tags = {"@SmokeTest"}
            )

POM snippet

     <properties>
            <maven.compiler.source>1.8</maven.compiler.source>
            <maven.compiler.target>1.8</maven.compiler.target>
            <!-- <downloadSources>true</downloadSources> <downloadJavadocs>true</downloadJavadocs> -->
            <aspectj.version>1.8.10</aspectj.version>
            <allureVersion>2.10.0</allureVersion>
            <cucumberversion>4.2.0</cucumberversion>

            <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        </properties>

        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.7.0</version>
                    <configuration>

        <testFailureIgnore>false</testFailureIgnore>

                        <encoding>UTF-8</encoding>
                        <source>1.8</source>
                        <target>1.8</target>
                        <compilerArgument>-Werror</compilerArgument>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.22.0</version>

                    <configuration>
                        <forkCount>3</forkCount>
                        <reuseForks>true</reuseForks>
                        <argLine>-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar" -Dcucumber.options="--plugin io.qameta.allure.cucumber4jvm.AllureCucumber4Jvm" -Xmx1024m -XX:MaxPermSize=256m
                        </argLine>
                        <properties>
                        <property>
                            <name>listener</name>

        <value>io.qameta.allure.junit4.AllureJunit4</value>
                        </property>
                    </properties>
                    </configuration>
                    <dependencies>
                        <dependency>
                            <groupId>org.aspectj</groupId>
                            <artifactId>aspectjweaver</artifactId>
                            <version>${aspectj.version}</version>
                        </dependency>
                    </dependencies>
                </plugin>

            </plugins>
        </build>
        <dependencies>
            <dependency>
            <groupId>io.qameta.allure</groupId>
            <artifactId>allure-junit4</artifactId>
                <version>${allureVersion}</version>
            <scope>test</scope>
        </dependency>
            <dependency>
                <groupId>io.qameta.allure</groupId>
                <artifactId>allure-cucumber4-jvm</artifactId>
                <version>${allureVersion}</version>
            </dependency>
            <!-- <dependency> <groupId>io.qameta.allure</groupId> 
        <artifactId>allure-plugin-api</artifactId> 
                <version>${allureVersion}</version> </dependency> -->
            <dependency>
                <groupId>io.cucumber</groupId>
                <artifactId>cucumber-java</artifactId>
                <version>${cucumberversion}</version>
            </dependency>
            <dependency>
                <groupId>io.cucumber</groupId>
                <artifactId>cucumber-core</artifactId>
                <version>${cucumberversion}</version>
            </dependency>

            <dependency>
                <groupId>io.cucumber</groupId>
                <artifactId>cucumber-junit</artifactId>
                <version>${cucumberversion}</version>
            </dependency>
            <dependency>
                <groupId>io.cucumber</groupId>
                <artifactId>cucumber-picocontainer</artifactId>
                <version>${cucumberversion}</version>
            </dependency>

            <dependency>
                <groupId>org.apache.commons</groupId>
                <artifactId>commons-configuration2</artifactId>
                <version>2.4</version>
            </dependency>
            <dependency>
                <groupId>io.rest-assured</groupId>
                <artifactId>rest-assured</artifactId>
                <version>3.3.0</version>
            </dependency>
            <dependency>
                <groupId>io.rest-assured</groupId>
                <artifactId>json-schema-validator</artifactId>
                <version>3.3.0</version>
            </dependency>
            <dependency>
                <groupId>io.rest-assured</groupId>
                <artifactId>json-path</artifactId>
                <version>3.3.0</version>
            </dependency>
            <dependency>
                <groupId>io.rest-assured</groupId>
                <artifactId>xml-path</artifactId>
                <version>3.3.0</version>
            </dependency>
            <dependency>
                <groupId>junit</groupId>
                <artifactId>junit</artifactId>
                <version>4.12</version>
            </dependency>

sample cucumber scenario

    @SmokeTest  

        Scenario: multiply a and b 
            Given I have variable a 
            And I have variable b 
            When I multiplication a and b 
            Then I display the Result




step def

        @Given("^I have variable a$")
        public void i_have_variable_a() throws Exception {
            System.out.println("at step : I have variable a");

        }

        @Given("^I have variable b$")
        public void i_have_variable_b() throws Exception {
            System.out.println("at step : I have variable b");

        }

        @When("^I multiplication a and b$")
        public void i_multiplication_a_and_b() throws Exception {
            System.out.println("at step : multiplication");

        }

        @Then("^I display the Result$")
        public void i_display_the_Result() throws Exception {
            System.out.println("at step : result display");

        }

Attachment code -- (not used in above example code but used in actual code

       @Attachment(fileExtension = "json", type = "text/json", value = "RestJsonResponse")
            public String attachResponse(Response strResponse) {
                return strResponse.asString();
            }

Expected: Able to see the Given when then statements somewhere in the report. Also, to see attachments in the report as json files.

Actual: Report generates but no given when then statements and no attachments.


Answer:

First of all, you've messed allure JUnit plugin with cucumber one. So you need to remove Allure JUnit listener from the surefire configuration and remove it from dependencies.

Next problem is in the overriding allure cucumber plugin in the class configuration. You should configure cucumber plugin in pom or class only because class config overrides pom configuration. The simplest way to fix it - remove --plugin from pom and add it to class configuration to other configs like this:

@RunWith(Cucumber.class)
@CucumberOptions(features = { "src/test/java/features" }, 
plugin = { "pretty", "html:target/cucumber-html-reports", "io.qameta.allure.cucumber4jvm.AllureCucumber4Jvm",
        "json:target/cucumber-html-reports/cucumber.json","rerun:target/failed_scenarios.txt" }, monochrome = true, glue = { "definitions" }, tags = {"@SmokeTest"}
        )

Question:

At this point with the recent release of cucumber-jvm 3.x.x, there does not seem to be a viable tutorial out there except for the older 1.x.x (docs).

I am unsure which one to use?

Expected Result:

To generate an Allure2 test report in target/ or target/site with Cucumber-JVM 3 using Maven after running mvn test or mvn verify.

A working pom.xml and @CucumberOptions are sought after. Thanks!


Update: (2018-08-31)

pom.xml

<properties>
    <!-- Project -->
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <app.name>Simply Do - Balance Projector</app.name>
    <!-- Maven -->
    <timestamp>${maven.build.timestamp}</timestamp>
    <maven.build.timestamp.format>yyyyMMdd'T'HHmmss'Z'</maven.build.timestamp.format>
    <!-- Allure Test Report -->
    <aspectj.version>1.9.1</aspectj.version>
    <!-- Core Dependencies -->
    <selenium.version>3.14.0</selenium.version>
    <ngwebdriver.version>1.1.4</ngwebdriver.version>
    <cucumber.version>3.0.2</cucumber.version>
    <java.version>1.8</java.version>
</properties>

<dependencies>
    <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-server -->
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-server</artifactId>
        <version>${selenium.version}</version>
        <scope>test</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/com.paulhammant/ngwebdriver -->
    <dependency>
        <groupId>com.paulhammant</groupId>
        <artifactId>ngwebdriver</artifactId>
        <version>${ngwebdriver.version}</version>
        <scope>test</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/io.cucumber/cucumber-junit -->
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-junit</artifactId>
        <version>${cucumber.version}</version>
        <scope>test</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/io.cucumber/cucumber-picocontainer -->
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-picocontainer</artifactId>
        <version>${cucumber.version}</version>
        <scope>test</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.assertj/assertj-core -->
    <dependency>
        <groupId>org.assertj</groupId>
        <artifactId>assertj-core</artifactId>
        <version>3.11.0</version>
        <scope>test</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-slf4j-impl -->
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-slf4j-impl</artifactId>
        <version>2.11.1</version>
        <scope>test</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/io.qameta.allure/allure-cucumber3-jvm -->
    <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-cucumber3-jvm</artifactId>
        <version>2.7.0</version>
        <scope>test</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/com.itextpdf/itextpdf -->
    <dependency>
        <groupId>com.itextpdf</groupId>
        <artifactId>itextpdf</artifactId>
        <version>5.5.13</version>
        <scope>test</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.poi/poi-ooxml -->
    <dependency>
        <groupId>org.apache.poi</groupId>
        <artifactId>poi-ooxml</artifactId>
        <version>3.17</version>
        <scope>test</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/com.codoid.products/fillo -->
    <dependency>
        <groupId>com.codoid.products</groupId>
        <artifactId>fillo</artifactId>
        <version>1.18</version>
        <scope>test</scope>
    </dependency>
</dependencies>

<build>
    <plugins>
        <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-site-plugin -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-site-plugin</artifactId>
            <version>3.7.1</version>
        </plugin>
        <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-project-info-reports-plugin -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-project-info-reports-plugin</artifactId>
            <version>3.0.0</version>
        </plugin>
        <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-jar-plugin -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-jar-plugin</artifactId>
            <version>3.1.0</version>
            <!-- Ignore src/main in JAR packaging -->
            <configuration>
                <classesDirectory>src</classesDirectory>
                <excludes>
                    <exclude>main</exclude>
                </excludes>
            </configuration>
        </plugin>
        <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-compiler-plugin -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.0</version>
            <configuration>
                <source>${java.version}</source>
                <target>${java.version}</target>
            </configuration>
        </plugin>
        <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-surefire-plugin -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.22.0</version>
            <configuration>
                <includes>
                    <include>**/RunCukesTest.java</include>
                </includes>
                <testFailureIgnore>true</testFailureIgnore>
                <reportsDirectory>${project.build.directory}/test-output/${timestamp}</reportsDirectory>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                </argLine>
            </configuration>
            <dependencies>
                <!-- https://mvnrepository.com/artifact/org.aspectj/aspectjweaver -->
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
        <!-- https://mvnrepository.com/artifact/io.qameta.allure/allure-maven -->
        <plugin>
            <groupId>io.qameta.allure</groupId>
            <artifactId>allure-maven</artifactId>
            <version>2.9</version>
            <configuration>
                <reportVersion>2.7.0</reportVersion>
                <allureDownloadUrl>https://github.com/allure-framework/allure2/releases/download/2.7.0/allure-2.7.0.zip</allureDownloadUrl>
            </configuration>
        </plugin>
        <!-- https://mvnrepository.com/artifact/net.masterthought/maven-cucumber-reporting -->
        <plugin>
            <groupId>net.masterthought</groupId>
            <artifactId>maven-cucumber-reporting</artifactId>
            <version>3.20.0</version>
            <executions>
                <execution>
                    <id>execution</id>
                    <phase>verify</phase>
                    <goals>
                        <goal>generate</goal>
                    </goals>
                    <configuration>
                        <projectName>${app.name}</projectName>
                        <outputDirectory>${project.build.directory}</outputDirectory>
                        <jsonFiles>
                            <param>**/cucumber-report.json</param>
                        </jsonFiles>
                        <classificationFiles>
                            <param>**/classifications/*.properties</param>
                        </classificationFiles>
                        <parallelTesting>false</parallelTesting>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Build Log:

pastebin


Answer:

due to breaking changes in Cucumber API, for different versions of Cucumber JVM you will need different Allure adapters:

  1. Cucumber JVM 1.x require allure-cucumber-jvm, formatter io.qameta.allure.cucumberjvm.AllureCucumberJvm
  2. Cucumber JVM 2.x require allure-cucumber2-jvm, formatter io.qameta.allure.cucumber2jvm.AllureCucumber2Jvm
  3. Cucumber JVM 3.x require allure-cucumber3-jvm, formatter io.qameta.allure.cucumber3jvm.AllureCucumber3Jvm
  4. Cucumber JVM 4.x require allure-cucumber4-jvm, formatter io.qameta.allure.cucumber4jvm.AllureCucumber4Jvm

allure-maven plugins is responsible report generation and can be used together with any of adapters above

Update: I have created example project with Cucumber JVM 3.x + Allure 2 + Maven here: https://github.com/letsrokk/stackoverflow-examples/tree/master/allure-cucumber-example

You can either run tests and serve report straight away

$ mvn clean test allure:serve

Or execute tests and just build report without opening it

$ mvn clean test allure:report
...
$ ls -l target/site/allure-maven-plugin/
total 4744
-rw-r--r--   1 user  CORP\Domain Users     2220 Aug 29 12:49 allure-maven.html
-rw-r--r--   1 user  CORP\Domain Users   726434 Aug 29 12:49 app.js
drwxr-xr-x  12 user  CORP\Domain Users      384 Aug 29 12:49 data
drwxr-xr-x   5 user  CORP\Domain Users      160 Aug 29 12:49 export
-rw-r--r--   1 user  CORP\Domain Users    15086 Aug 29 12:49 favicon.ico
drwxr-xr-x   7 user  CORP\Domain Users      224 Aug 29 12:49 history
-rw-r--r--   1 user  CORP\Domain Users      657 Aug 29 12:49 index.html
drwxr-xr-x   9 user  CORP\Domain Users      288 Aug 29 12:49 plugins
-rw-r--r--   1 user  CORP\Domain Users  1673197 Aug 29 12:49 styles.css
drwxr-xr-x  16 user  CORP\Domain Users      512 Aug 29 12:49 widgets

Question:

I'm able to run Cucumber JVM with Gradle but not sure what exactly I need to add in build.gradle to get Allure report after test execution. I checked an official website and read section Cucumber JVM but there is information only about maven project [cucumber jvm|https://docs.qameta.io/allure/#_cucumber_jvm]. Could anybody give an example with the appropriate setting for build.gradle file if we use the combination of Cucumber JVM + Gradle + Allure.


Answer:

Finally found the solution.

  1. Add task Cucumber as recommended here (https://cucumber.io/docs/tools/java/#gradle)
    • Add the following in args for task cucumber
args = [ '--plugin', 'io.qameta.allure.cucumber4jvm.AllureCucumber4Jvm', '--glue', 'path_to_your_glue_code', ("path_to_your_feature")]

  • Add dependency for the plugin
dependencies { testCompile("io.qameta.allure:allure-cucumber4-jvm:2.10.0") }
  1. Add allure plugin (https://plugins.gradle.org/plugin/io.qameta.allure)
  2. From the command line gradle cucumber and allure-reports directory will be generated in your root project.
  3. The final step to generate html report
allure generate 'path_to_allure_results_folder' -c -o 'path_to_your_report_folder\'