Hot questions for Using Cucumber in testng

Question:

I am trying Cucumber for my Selenium Java + TestNG project. Once I run the .feature file with empty stepdefinition, I should get a snippet that I can just copy paste to my step definition class. But I got a strange format that can't be used out of the box.

Given should be @Given , right? There is no public void() .

What could be the cause? Thank You.

2 Scenarios (2 undefined)
5 Steps (5 undefined)
0m0.000s
You can implement missing steps with the snippets below:
Given("^User is on Home Page$", () -> {
    // Write code here that turns the phrase above into concrete actions
    throw new PendingException();
});
When("^User enters UserName and Password$", () -> {
    // Write code here that turns the phrase above into concrete actions
    throw new PendingException();
});
Then("^He can visit the practice page$", () -> {
    // Write code here that turns the phrase above into concrete actions
    throw new PendingException();
});
When("^User LogOut from the Application$", () -> {
    // Write code here that turns the phrase above into concrete actions
    throw new PendingException();
});
Then("^he cannot visit practice page$", () -> {
    // Write code here that turns the phrase above into concrete actions
    throw new PendingException();
});

Answer:

You are using cucumber-java8 dependency, for that below is the format for stepDefintion file(make sure your java compiler version should be 1.8).

@Given Annotation will be in cucumber-java

Given Annotation will be in cucumber-java8

import cucumber.api.java8.En;

public class stepDefinitions implements En{

public stepDefinitions(){
    Given("^User is on Home Page$", () -> {
        // Write code here that turns the phrase above into concrete actions
        System.out.println("Test1");
    });

    When("^User enters UserName and Password$", () -> {
        // Write code here that turns the phrase above into concrete actions
        System.out.println("Test2");
    });

    Then("^He can visit the practice page$", () -> {
        // Write code here that turns the phrase above into concrete actions
        System.out.println("Test3");
    });

    When("^User LogOut from the Application$", () -> {
        // Write code here that turns the phrase above into concrete actions
        System.out.println("Test4");
    });

    Then("^he cannot visit practice page$", () -> {
        // Write code here that turns the phrase above into concrete actions
        System.out.println("Test5");
    });
   }
 }

I executed it in my machine its working, try it and let me know if it works for you.

Link for reference: http://codoid.com/cucumber-lambda-expressions/

Question:

I'm having a problem when I configure a WireMockServer inside a Cucumber with testng v7.0.0 (I think that's irrelevant here). I want to call a service of my own, which per se requests the url I'm defining.

When I try to invoke like this:

@When("the service is called")
public void theServiceIsCalled() {
    wireMockServer = new WireMockServer(options().port(PORT_NUMBER));
    wireMockServer.start();
    configureFor("127.0.0.1", wireMockServer.port());
    stubFor(get(urlEqualTo(url))
                .willReturn(aResponse()
                        .withStatus(HTTPOK)
                        .withHeader("Content-Type", "application/json")
                        .withBody(response)));
    myService.callService();
}

I always get

java.util.concurrent.CompletionException: javax.ws.rs.ProcessingException: org.apache.http.conn.HttpHostConnectException: Connect to 127.0.0.1:9090 [/127.0.0.1] failed: Connection refused

I know this can't be a problem of a wrongly defined port, url or headers. I say that because when I run a java instance with the exact same WireMock configurations (without the service call), and then the test, it runs smoothly, the service returns in fact what I have mocked. It seems like the WireMockServer lifetime must reside in a process outside of the one I want.

What's going on?

Thanks in advance ;)


Answer:

So, it seems that the problem was due to 2 causes:

1 - I wasn't actually initializing the wireMockServer (initialization was being held in a BeforeAll method which was not actually being called), sorry about that.

2 - The stubFor must be used against the wireMockServer instance.

Below is a working configuration:

WireMockServer server = new WireMockServer(options().bindAddress("127.0.0.1").port(port));
server.stubFor(get(urlEqualTo(urlWithId))
                .willReturn(aResponse()
                        .withStatus(HTTPOK)
                        .withHeader("Content-Type", "application/json")
                        .withBody(response)));
server.start()
myService.callService();

Hope someone finds this useful.

Question:

I am trying to run my cucumber tests under different browser, so I have test.xml (yes, currently just one browser)

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="SelfCare" verbose="1" thread-count="1" parallel="tests" configfailurepolicy="continue">
    <parameter name="environment" value="TEST" />
    <test name="Chrome">
        <parameter name="browser" value="chrome" />
        <parameter name="width" value="1024" />
        <parameter name="height" value="768" />
        <classes>
            <class name="TestNGRunner">
                <methods>
                    <include name="scenario"/>
                </methods>
            </class>
        </classes>
    </test>
</suite>

And this TestNG runner class, I am able to access parameter browser from this runner class, but I would need to pass this variable(s) to Cucumber where are create instance of browser based on passed variables. So far I use it without TestNG and I'm passing those variables through environment variables, however I assume this won't work if I run tests in parallel on different browsers.

import cucumber.api.CucumberOptions;
import cucumber.api.testng.CucumberFeatureWrapper;
import cucumber.api.testng.PickleEventWrapper;
import cucumber.api.testng.TestNGCucumberRunner;
import org.testng.annotations.*;

@CucumberOptions(
        plugin = {"html:target/cucumber-html-report",
                "json:target/cucumber.json",
                "pretty:target/cucumber-pretty.txt",
                "junit:target/cucumber-results.xml"},
        strict = true,
        extraGlue = {"src/test/resources/Hooks.java"},
        features = {"src/test/resources/Header.feature"}
)
public class TestNGRunner {
    private TestNGCucumberRunner testNGCucumberRunner;
    private String browser;

    @BeforeClass(alwaysRun = true)
    public void setUpClass() throws Exception {
        testNGCucumberRunner = new TestNGCucumberRunner(this.getClass());
    }

    @Parameters("browser")
    @BeforeTest
    public void beforeSuite(String browser) {
        System.out.println("Browser for this test set is " + browser);
        this.browser = browser;
    }

    @Test(groups = "Cucumber", description = "Runs Cucumber Feature", dataProvider = "scenarios")
    public void scenario(PickleEventWrapper pickleEvent, CucumberFeatureWrapper cucumberFeature) throws Throwable {

        testNGCucumberRunner.runScenario(pickleEvent.getPickleEvent());
    }

    @DataProvider
    public Object[][] scenarios() {
        return testNGCucumberRunner.provideScenarios();
    }

    @AfterClass(alwaysRun = true)
    public void tearDownClass() throws Exception {
        testNGCucumberRunner.finish();
    }

}

Any ideas how to achieve that?


Answer:

Apparently, I can access to parameters from anywher usgin Reporter.getCurrentTestResult().getTestContext().getCurrentXmlTest().getParameter("browser")

Question:

I have a selenium cucumber testng framework using maven. I was hoping to get step descriptions when I run my cucumber tests with testng. It was an old project so I have changed my dependencies from info.cukes to the latest version of io.cucumber. However it seems to be throwing out errors whenever I run the runner test file.

java.lang.NoSuchMethodError: cucumber.runtime.RuntimeOptions.getPluginFormatterNames()Ljava/util/List;

    at cucumber.runtime.formatter.Plugins.createPlugins(Plugins.java:64)
    at cucumber.runtime.formatter.Plugins.<init>(Plugins.java:37)
    at cucumber.api.testng.TestNGCucumberRunner.<init>(TestNGCucumberRunner.java:56)
    at cucumber.api.testng.AbstractTestNGCucumberTests.setUpClass(AbstractTestNGCucumberTests.java:16)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
    at org.testng.internal.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:59)
    at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:458)
    at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:222)
    at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:142)
    at org.testng.internal.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:168)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:105)
    at org.testng.TestRunner.privateRun(TestRunner.java:648)
    at org.testng.TestRunner.run(TestRunner.java:505)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
    at org.testng.SuiteRunner.run(SuiteRunner.java:364)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)
    at org.testng.TestNG.runSuites(TestNG.java:1049)
    at org.testng.TestNG.run(TestNG.java:1017)
    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)

I have the following maven dependencies but have no idea if it is a dependency that I am missing to get the test to run? Any Help would be greatly appreciated. As when I have search this issue it seems that it is mostly solved by having the same version dependencies of all the io.cucumber files which I do.

 <!-- https://mvnrepository.com/artifact/info.cukes/cucumber-junit -->
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-junit</artifactId>
    <version>4.2.0</version>
    <scope>test</scope>
</dependency>

 <!-- https://mvnrepository.com/artifact/info.cukes/cucumber-java -->
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-java</artifactId>
    <version>4.2.0</version>
</dependency>

 <!-- https://mvnrepository.com/artifact/info.cukes/cucumber-testng -->
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-testng</artifactId>
    <version>4.2.0</version>

</dependency>

        <!-- https://mvnrepository.com/artifact/org.mockito/mockito-core -->
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-core</artifactId>
            <version>2.21.0</version>
            <scope>test</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/io.cucumber/tag-expressions -->
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>tag-expressions</artifactId>
            <version>1.1.1</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/io.cucumber/cucumber-expressions -->
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-expressions</artifactId>
            <version>6.2.0</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/io.cucumber/messages -->
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>messages</artifactId>
            <version>2.1.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/io.cucumber/cucumber-java8 -->
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java8</artifactId>
            <version>4.2.0</version>
        </dependency>



        <!-- https://mvnrepository.com/artifact/com.github.mkolisnyk/cucumber-report-generator -->
<dependency>
    <groupId>com.github.mkolisnyk</groupId>
    <artifactId>cucumber-report-generator</artifactId>
    <version>1.3</version>
</dependency>



    <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.14.3</version>
            <scope>test</scope>
        </dependency>  

        <!-- https://mvnrepository.com/artifact/io.cucumber/gherkin -->
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>gherkin</artifactId>
    <version>6.0.14</version>
</dependency>

        <!-- https://mvnrepository.com/artifact/info.cukes/gherkin -->
<!--<dependency>
    <groupId>info.cukes</groupId>
    <artifactId>gherkin</artifactId>
    <version>2.12.2</version>
    <scope>provided</scope>
</dependency>-->
        <!-- https://mvnrepository.com/artifact/info.cukes/cucumber-core -->
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-core</artifactId>
    <version>4.2.0</version>
</dependency>



     <!-- https://mvnrepository.com/artifact/ru.sbtqa.tag/cucumber-runner -->
<dependency>
    <groupId>ru.sbtqa.tag</groupId>
    <artifactId>cucumber-runner</artifactId>
    <version>1.0.7</version>
</dependency>

Answer:

Find below some snippets to break down the problem

  1. check if there is a transitive dependency to groupid info.cukes

    $ mvn dependency:tree | grep 'info.cuke'
    [INFO] |  +- info.cukes:cucumber-junit:jar:1.2.5:compile
    [INFO] |  |  \- info.cukes:cucumber-core:jar:1.2.5:compile
    [INFO] |  |     +- info.cukes:cucumber-html:jar:0.2.3:compile
    [INFO] |  |     \- info.cukes:cucumber-jvm-deps:jar:1.0.5:compile
    [INFO] |  +- info.cukes:cucumber-testng:jar:1.2.5:compile
    [INFO]    |  +- info.cukes:cucumber-java:jar:1.2.5:compile
    [INFO]    |  +- info.cukes:gherkin:jar:2.12.2:compile
    

    when checking the full output reveals which of your dependency depends on it

    ...
    [INFO] +- com.github.mkolisnyk:cucumber-report-generator:jar:1.3:compile
    ...
    [INFO] |  +- info.cukes:cucumber-junit:jar:1.2.5:compile
    ...
    [INFO] |  +- info.cukes:cucumber-testng:jar:1.2.5:compile
    ...
    [INFO] \- ru.sbtqa.tag:cucumber-runner:jar:1.0.7:compile
    [INFO] \- ru.sbtqa.tag:cucumber-runner:jar:1.0.7:compile
    [INFO]    +- ru.yandex.qatools.allure:allure-cucumber-jvm-adaptor:jar:1.6.4:compile
    ...
    [INFO]    |  +- info.cukes:cucumber-core:jar:1.2.5:compile
    ...
    [INFO]    |  +- info.cukes:cucumber-java:jar:1.2.5:compile
    [INFO]    |  +- info.cukes:gherkin:jar:2.12.2:compile
    ...
    [INFO]    +- info.cukes:cucumber-junit:jar:1.2.5:compile
    
  2. as there might be an overlapping of classed with the same qualified name, exclude those transient dependencies in the pom.xml

        <dependency>
            <groupId>com.github.mkolisnyk</groupId>
            <artifactId>cucumber-report-generator</artifactId>
            <version>1.3</version>
            <exclusions>
                <exclusion>
                    <groupId>info.cukes</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    
  3. running the test again it throws an exception

    java.lang.NoClassDefFoundError: gherkin/IGherkinDialectProvider
    
  4. as we explicitly define the dependency io.cucumber:gherkin it might be related to the specified <version>, comment it out to see if another version is fetched

    <!--<dependency>-->
        <!--<groupId>io.cucumber</groupId>-->
        <!--<artifactId>gherkin</artifactId>-->
        <!--<version>6.0.14</version>-->
    <!--</dependency>-->
    

    this reveals that io.cucumber:cucumber-core:jar:4.2.0 depends on version 5.1.0 of io.cucumber:gherkin

    $ mvn dependency:tree
    ...
    [INFO] +- io.cucumber:cucumber-core:jar:4.2.0:compile
    [INFO] |  +- io.cucumber:cucumber-html:jar:0.2.7:compile
    [INFO] |  +- io.cucumber:gherkin:jar:5.1.0:compile
    

With those changes at least the TestNG unit test is running.

As the dependencies com.github.mkolisnyk:cucumber-report-generator and ru.sbtqa.tag:cucumber-runner depending on the info.cuke dependencies they might not work correctly anymore. To solve this kind of problem you could:

  • check for a newer version which might not depend on group id info.cuke
  • use earlier version of io.cucumber dependencies
  • use a more finegrained exclusion of the info.cuke dependencies, instead of excluding all artifact ids
  • a combination of above points
  • something else, depending on your use case and requirements

Maybe those steps help you to find a solution working for you.

Question:

I am working on a large Maven Java project that contains suites of TestNG automated tests and also suites of Cucumber automated tests. I realise that this is not ideal, but the different test suites were written by different sub-teams at different stages of the project. Going forward we intend to split this project out into smaller projects, but for now we are stuck with this mix.

The surefire plugin can be used to run these tests from Maven, but the plugin needs to be configured differently for each in our pom.xml.

For Cucumber, we are using it in conjunction with cucumber-jvm-parallel-plugin and it is configured like this:

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.20.1</version>
            <configuration>
                <forkCount>${threads}</forkCount>
                <reuseForks>true</reuseForks>
                <includes>
                    <include>**/Parallel*IT.class</include>
                </includes>
            </configuration>
        </plugin>

For our TestNG tests, it is configured like this:

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.20.1</version>
            <configuration>
                <suiteXmlFiles>${file}</suiteXmlFiles>
                <skipTests>false</skipTests>
                <properties>
                    <property>
                        <name>suitethreadpoolsize</name>
                        <value>${threads}</value>
                    </property>
                </properties>
            </configuration>
        </plugin>

We are not Maven experts and so, at the moment, we simply comment out the version of the plugin that we do not want to use for the suite we are running. Obviously this is cumbersome and not best practice, so I would be very grateful for any advice as to how we should address this. Can we legitimately define the plugin twice in the pom.xml and somehow pass a flag to indicate which version should be run? Many thanks for reading my question.


Answer:

Use maven profiles to select the proper configuration:

add this snippet:

<profiles>
    <profile>
        <id>Cucumber</id>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <configuration>
                    <forkCount>${threads}</forkCount>
                    <reuseForks>true</reuseForks>
                    <includes>
                        <include>**/Parallel*IT.class</include>
                    </includes>
                </configuration>
            </plugin>
        </plugins>
    </profile>
    <profile>
        <id>TestNG</id>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <configuration>
                    <suiteXmlFiles>${file}</suiteXmlFiles>
                    <properties>
                        <property>
                            <name>suitethreadpoolsize</name>
                            <value>${threads}</value>
                        </property>
                    </properties>
                </configuration>
            </plugin>
        </plugins>
    </profile>
</profiles>

and change this

<plugins>
  <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.20.1</version>
    </plugin>
   <!-- delete on of the entries -->
</plugins>

you have to pecify the desired profile at the command line:

   mvn test -P Cucumber
   mvn test -P TestNG 

bit you can also run both at once:

mvn test -P Cucumber -P TestNG 
mvn test -P Cucumber,TestNG 

Question:

I am executing my selenium automation scripts through testng, for the same I am having a testng xml form where I am passing parameter values like which test case has to be executed and on which environment it has to be executed.

E.g of testng xml is pasted below

**<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Automation Suite">
<parameter name="tags" value="MyTestCase"/>
    <!-- Default suite -->    
<test name="GGGGG Automation Test" verbose="2">
<parameter name="Environment" value="MyEnvironment" />
<classes>
<class name="com.XXXXX.TestRunner.XXXXXX"/>
   </classes>
 </test>  
</suite>**

here I am passing 2 parameters like test cases and environment to be executed form testng example. I am looking for a command line 'command' to execute the testng xml passing those 2 parameters from commandline argument.


Answer:

By using JVM arguments you can pass in parameters, such as -Dname=Environment. This has been answered previously here.

This blog post has examples

Question:

Is there a screenshot embed method for TestNG which is available for Cucumber?

I have the following Cucumber method up and running, but is there a similar method for JUnit or TestNG which will append images to created reports (XML reports)

public void close_browser_window(Scenario scenario) throws Exception {
        if (scenario.isFailed()) {
            scenario.embed(((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES), "image/png");
        }
    }

Answer:

I think your question has 2 paths. One is actually taking the screenshot and the other one is attaching it to an XML report.

So, in TestNG in order to take a screenshot you can override the OnTestFailure method like so:

public class onFailure extends TestListenerAdapter {

    @Override
    public void onTestFailure(ITestResult result) {

    File scrFile = ((TakesScreenshot)driver.getScreenshotAs(OutputType.FILE)); 
    FileUtils.copyFile(scrFile, new File("C:\\Screenshots\\Regression\\"+nameVar+"_"+envVar+".png")); 
    }

}

Then without too much hassle you can use Extent Reports which can attach the screenshot to your report, check out the community edition here! Update after OP's comment:

Question:

If I had two instances of a class instantiated in separate threads with the same annotation. And both threads were to modify the contents of the annotation for their class. Will the contents of the annotation be different for each instance or the same?

To set some context I am using testng and cucumber and would like to run the same test method twice with different tags. I was planning on editing the cucumber options at runtime via reflection and was sort of assuming that as they are in separate threads that would be fine (im pretty sure that was a dumb assumption)...

I'd seen the following example on how to modify the params of annotations at runtime


Answer:

An instance of a class only creates new field primitives or references on the heap; it does not modify the class template. If you use reflection to modify the annotations, you are modifying the class template, even if the annotations are on fields.

Threads should have no impact, apart from setting up race conditions if you do reflection in them.

Apparently, you have written code to do this and are seeing some sort of results; are they consistent with that explanation?