Hot questions for Using Cucumber in unit testing

Top Java Programmings / Cucumber / unit testing

Question:

I've written a JUnit test that uses Mockito and PowerMock to mock some classes. I'm trying to convert it a Cucumber test, but the static PowerMock features don't work.

Extracts of the two relevant Cucumber classes:

Runner

@RunWith(Cucumber.class)
public class JWTValidatorBDDTest {
}

Steps Class

public class JWTValidatorCukeTest {
String tokenValue;
JWTValidator jwtValidator;
MockHttpServletRequest mockRequest;

@Before
public void before() throws IOException {
    this.mockRequest = new MockHttpServletRequest();
    PowerMockito.mockStatic(JWTAuthConnectionManager.class);
    BDDMockito.given(JWTAuthConnectionManager.postToken(anyString(), anyString(), anyString())).willReturn(200);
    Mockito.doReturn(200).when(JWTAuthConnectionManager.postToken(anyString(), anyString(), anyString()));
}

@Given("^a JWT token with the value (.*)")
public void a_JWT_token_with_the_value_(String token) {
    this.jwtValidator = new JWTValidator("https://test.7uj67hgfh.com/openam", "Authorization", "Bearer");
    this.tokenValue = token;
}

Whilst this code works within the JUnit test, it fails here - it enters the JWTAuthConnectionManager.postToken() method that should be mocked and then fails by executing code within there. I've tried adding the lines:

@RunWith(PowerMockRunner.class)
@PrepareForTest(JWTAuthConnectionManager.class)

to both of the above classes (although of course I can't use RunWith in the Runner class as it already has one RunWith annotation), but this doesn't change anything.

How do I get PowerMock to work within Cucumber?


Answer:

Seems like it is possible now with @PowerMockRunnerDelegate annotation. I use @RunWith(PowerMockRunner.class) and @PowerMockRunnerDelegate(Cucumber.class) and it's working. Taken an advise from here: https://medium.com/@WZNote/how-to-make-spock-and-powermock-work-together-a1889e9c5692

Since version 1.6.0 PowerMock has support for delegating the test execution to another JUnit runner without using a JUnit Rule. This leaves the actual test-execution to another runner of your choice. For example tests can delegate to "SpringJUnit4ClassRunner", "Parameterized" or the "Enclosed" runner.

There are also options of using @Rule: PowerMockRule rule = new PowerMockRule(); instead of @RunWith(PowerMockRunner.class) (so Runner can be something else) - but the comment by Stefan Birkner suggests that Cucumber runner should support rules to use this and I am not sure if it does (now).

Hope it helps someone.

Question:

A question about best practices or practice at all ;)

I am currently working on a test automation system using Selenium in Java. It is supposed to be used for end-to-end acceptance testing of a webapp. The test cases are written in the Gherkin language and executed by the BDD framework Cucumber (Cucumber-JVM). The low-level functions use Selenium/WebDriver for interacting with the AUT and the browser. The Selenium code is structured using the PageObject pattern which abstracts the usage of WebDriver away. The cucumber step definitions call just the methods provided by the PageObjects.

As the project continues and becomes more and more complex I would like to start writing unit test to make sure the acceptance tests, and the utility functions around those, do what they should :)

Now to the question:

Is it feasible to write unit test for testing a test automation project?

The main problem is that during my first approach to unit testing using TestNG I realised, that my unit tests ended up doing more or less the same stuff the acceptance tests already did. This is counter productive, as the unit tests are very slow and have a lot dependencies.

Or does one just test the utility classes and leave the Selenium code be in such a case. ie. test just the stuff that can be tested without calling the Selenium WebDriver and interacting with the AUT?

Note just to be sure I have not been misunderstood. I'm asking about running unit test ON the acceptance test code and all the auxiliary code. Not about running the Selenium test cases using an unit testing framework like JUnit or TestNG.

Any help and/or ideas will be appreciated, as I am not sure how to tackle this one. That is if writing tests for tests is at all sensible ;)


Answer:

I'm sure someone will consider my response "opinionated" and vote it down, but nevertheless I think that

Yes, if you have a test framework you are relying upon for your acceptance testing, the framework itself needs to be tested.

From my experience the value is in 2 areas:

  1. Be able to change your framework with confidence. When you create some function, you know quite a lot about it, i.e. which use cases it supports, what it's designed to do, etc. But other people (or even you a year from now) may not have the same level of knowledge, even with documentation. So either new functions will pop up every time someone needs some slight modification in behavior (because they are not confident to change existing function), or someone may break whole bunch of acceptance tests.

    Best if those are true unit tests, able to run completely independently from anything (use mocks, predefined static test data, etc).

  2. Protect yourself from unexpected changes / bugs in Selenium itself (or other important 3rd-party libraries). When you're updating Selenium to next version (and that usually needs to be done every 3-6 months), there's always a chance that they changed some default you were relying upon (and didn't even know), or broke something, or suddenly something returns a different exception, or does not throw exception where it previously did and so on. Of course no need to get carried away and duplicate Selenium own unit tests, but when it comes to non-trivial things, or relying on some features with poor documentation, those tests may help a lot.

    Those are integration tests. Ideally they should run against test-only webapp (not the real application) that replicate specifically tested behaviors in a way convenient for tests.

Of course some compromises possible as well. For example having a small subset of acceptance tests that are serving as unit / integration tests (they run first, and other tests only run if they are passing). Might be cheaper to begin with those and slowly migrate to proper unit/integration tests when you debug / fix issues in the test framework.

Another question is how do you separate tests testing your framework from actual acceptance tests for the product. What worked for me is keeping test framework and acceptance tests in 2 separate projects. That way I can change the framework, build it (which also includes running unit and integration tests) many times if needed. When all unit and integration tests are passing, then I can update the version used by actual acceptance tests.

Question:

I am new to cucumber and wanted to understand if there is any plugin to generate java test class code from a cucumber feature file.

Example : I have the below scenario - 
  Scenario: Determine past date
    Given today is 2011-01-20
    When I ask if Jan 19, 2011 is in the past
    Then the result should be yes

Is there a way to generate test class with the methods for each? I am just looking to generate the skeleton of the class so that it speeds up the development process.


Answer:

You can run the feature with a runner class like:

import org.junit.runner.RunWith;

import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
@CucumberOptions(
dryRun = false,
strict = true,
plugin = {"pretty"},
features = {"path/to/features"},
glue = {"package.of.steps"},
tags = {"@TagsToRun"})

public class MyCucumberTestRunnner {
    public MyCucumberTestRunnner() {
    }
}

This can be executed as JUnit Test and Cucumber will tell you, that there are missing steps and will provide you the Step Skeletons.

if the glue code is in the same package you dont need to provide the information

Question:

I am a Java developer. We want to use cucumber testing in our project. We are working mainly on creating APIs. I am good with unit testing and researching about cucumber. I am thinking about testing persistence methods - CRUD operations as an starter. My questions is that what could be the scenerios in this testing. Also should I mock the database by creating tables in the feature file. Should I use mockito with Cucumber to mock call to some other services which connects to database and server. What should be the cucumber testing in these scenerios and whats the best way to create framework to use cucumber in our Java API's project.

Also, how to populate models if not using database


Answer:

IMO Gherkin (the language you write Cucumber features in), is good for writing business readable, simple scenarios.

To answer quickly, I would say that Cucumber is not a good fit for testing methods, if it is what you want to do.

As you can see with the file naming convention, you write *.feature files, and I think these files must only contains feature-related descriptions.

However, if you do have features to test, you have to choose how to test them

  • disconnected, can be run quicky by your CI
    • you will have to mock everything that cannot start-up in the build lifecycle
    • and they are solutions to start almost anything using Docker, like Testcontainers
  • connected to a environment
    • you do not have to mock anything
    • your tests may be slower
    • your tests may break because of the environement (failed deployement, server down, etc.)

Question:

I am following BDD apprach and using cucumber to perform unit testing I have a class as given below

public class EmployeeServiceImpl {

private static Log log = LogFactory.getlog(EmployeeServiceImpl.class);

@Autowired
private EmployeeDao employeeDao;

public void saveEmployee(Employee emp) throws Exception {
    try {
employeeDao.saveemployee(emp);
      } catch (Exception ex) {
            log.error("Error occured "+ex);
      }
  }
}

Can anyone please help me how to write exception scenario for the above code snippet?


Answer:

The algorithm I would use is something like this:

  • Prepare the argument emp so it will throw the expected error
  • Call the method
  • Verify that the interaction with the log happened

I would use Mockito for verifying the interaction, http://site.mockito.org/#how

A JUnit solution could look like this:

@Test
public void verify_logging() throws Exception {
    Log log = mock(Log.class);

    Employee emp = new Employee();

    saveEmployee(emp);

    verify(log, times(1)).error("Error occured");
}

Transforming this to Cucumber with three steps would be something like this:

private Log log;
private Employee emp;

@Given("prepare employee")
public void given() {
    Log log = mock(Log.class);

    Employee emp = new Employee();
}

@When("save employee")
public void when() throws Exception {
    saveEmployee(emp);
}

@Then("exception should be logged")
public void then() {
    verify(log, times(1)).error("Error occured");
}

Your task is to come up with better names for the step methods as well as better steps.