How to Cucumber : Test Report Plugin 2 with Maven and Java

cucumber-logoIt is often useful to show the results of our tests to others people who are not developers, like managers or executive. This tutorial shows how to integrate a Cucumber reporting a Java plugin in Maven project. The result is a simple web page, which does not require a special tool to be read.

This tutorial will cover 3 reports plugins : JSON, HTML Pretty and Cucumber Report 2.

screenshot000.png

Native JSON Result

screenshot001.png

How to:

pom.xml



  4.0.0
  com.damienfremont.blog
  20160509-test-cucumber_plugin_reporting_2
  0.0.1-SNAPSHOT
  jar
  
    1.2.5
    8
  
  
    
      info.cukes
      cucumber-junit
      ${cucumber.version}
      test
    
    
      info.cukes
      cucumber-java8
      ${cucumber.version}
      test
    
  
  
    
      
        org.apache.maven.plugins
        maven-compiler-plugin
        3.3
        
          1.${java.version}
          1.${java.version}
        
      
      
        org.apache.maven.plugins
        maven-surefire-plugin
        2.18.1
        
          true
          
            **/*BDDTest.java
          
        
      
    
  

RunBDDTest.java

package com.damienfremont.blog;

import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
@CucumberOptions(strict = false, features = "features", format = { "pretty",
    "json:target/cucumber.json" }, tags = { "~@ignore" })
public class RunBDDTest {

}

Demo:

Launch with JUnit or

mvn test

The result is at /target/cucumber.json

screenshot009.jpg

Native HTML Plugin

screenshot002.png

How to:

RunBDDTest.java

package com.damienfremont.blog;

import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
@CucumberOptions(strict = false, features = "features", format = { "pretty",
    "html:target/site/cucumber-pretty",
    "json:target/cucumber.json" }, tags = { "~@ignore" })
public class RunBDDTest {

}

Demo:

Launch with JUnit or

mvn test

The result is at /target/site/cucumber-pretty/index.html

screenshot008.jpg

Cucumber-Reports HTML Plugin

screenshot003.png

screenshot004.png

How to:

pom.xml



  4.0.0
  com.damienfremont.blog
  20160509-test-cucumber_plugin_reporting_2
  0.0.1-SNAPSHOT
  jar
  
    1.2.5
    8
  
  
    
      info.cukes
      cucumber-junit
      ${cucumber.version}
      test
    
    
      info.cukes
      cucumber-java8
      ${cucumber.version}
      test
    
  
  
    
      
        org.apache.maven.plugins
        maven-compiler-plugin
        3.3
        
          1.${java.version}
          1.${java.version}
        
      
      
        org.apache.maven.plugins
        maven-surefire-plugin
        2.18.1
        
          true
          
            **/*BDDTest.java
          
        
      
      
        net.masterthought
        maven-cucumber-reporting
        2.0.0
        
          
            execution
            verify
            
              generate
            
            
              cucumber-jvm-example
              ${project.build.directory}/site/cucumber-reports
              ${project.build.directory}/cucumber.json
              true
              false
              42
            
          
        
      
    
  

RunBDDTest.java

package com.damienfremont.blog;

import org.junit.runner.RunWith;

import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
@CucumberOptions(strict = false, features = "features", format = { "pretty",
    "json:target/cucumber.json" }, tags = { "~@ignore" })
public class RunBDDTest {

}

Demo:

Launch with

mvn test verify

The result is at /target/site/cucumber-reports/feature-overview.html

screenshot007.jpg

Conclusion

JSON is OK for integration with other tools (jenkins plugin, reports, etc).

Pretty HTML is OK during development phase (like a JUnit or Surefire report for versionning).

Cucumber-Reports is good for reporting outside of the dev team (like managers).

Source

https://damienfremont.com/2016/05/09/how-to-cucumber-test-report-plugin-2-with-maven-and-java

https://github.com/DamienFremont/blog/tree/master/20160509-test-cucumber_plugin_reporting_2

References

https://cucumber.io/docs/reference/jvm

https://github.com/masterthought/cucumber-reporting

http://www.masterthought.net/section/cucumber-reporting

30 thoughts on “How to Cucumber : Test Report Plugin 2 with Maven and Java

  1. Hi Damien,

    Do we need to execute runner class or we have to run via feature file for those cucumber-jvm report with charts.

    Thanks,
    Suresh

  2. Hi Damien,

    I need to generate report like the one Cucumber-reports HTML Plugin with features-overview.html file. Can u guide me on how to do that one.

    Thanks,
    Suresh

  3. Hi,

    Thanks for the such a amazing report. I am facing a problem.
    Assume that this is my first run and no json already exists. When I run a test case I see json file not found exception even though file exists. When I run a different test case now, I could see the report is generated for previous run perhaps using the json generated for last run. And this keeps happening every next run generates report for previous run. I tried adding report code in junit afterclass but no luck. Could you please help me to resolve this issue?

    • Hi Mukund,

      can u share the runner class how u can able to generate reports with graphical representation?

      Thanks,
      Suresh

  4. Hi,

    Could you please let me know if it is possible to include custom fields in the report dashboard.
    For eg: Browser used = Firefox, environment = UAT etc.

    Thanks

  5. Hi,
    Could you please help me to understand why feature-overview.html is not generating. Only index.html is generating

  6. Can anyone share the sample small project like that above reports generated I was not able to generate feature-overview.html file which is in a graphical representation of the reports

  7. Hi Does these reports work if I use TestNG insted of Junit? if not, how can I get these detailed reports with TestNG.

  8. Hi, thanks for the great tutorial !!
    I followed thru your steps and if I run TestRunner with junit, it runs fine but my report don’t get updated I have to do mvn install for it to update. how do I run my test and update reports at the same time?

    in other workd If I clean and run my TestRunner class thru junit it doesn’t generate cucumber-report unless I do mvn install.

  9. HI
    I am new to automation I wrote same code in pom.xml in eclipse but can not generate feature-overview.html, I am running my tests through maven build in eclipse run configration
    @CucumberOptions(features = “src/test/resources/feature”, plugin = { “json:target/cucumber.json”, “html:target/site/cucumber-pretty” })
    can somebody help me with this pls?

  10. Hi,I am getting below error..
    [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile (default-compile) on project XYZ: Compilation failure

    • this is solved.. Now I can able to generate the reports .Ran ‘mvn verify’ after ‘mvn test’. Thanks @manu.

  11. Hi Damien,

    How do we configure this for multiple runs and spit out the reports in a new directory each time so we do not lose or overwrite previous test reports?

    • Hi,
      Usually by using a CI plateforme like Jenkins or Bamboo. Each run/build result (target and source) will be save in a different folder (automatically deleted after some time, or saved if the build is tagged as a release).

      • Hi Damien,

        How about if we would be doing it offline?
        I even changed the directory in my POM to point to a new directory having timestamp information, but seems the code completely cleans up the reports directory
        This is what I have in POM.xml right now
        target/cucumber-reports/advanced-reports/${timestamp}

        but each run the advanced-reports is cleared up it seems

  12. Hi Damien,

    I figured a way to do that.
    Could you tell me how to use the advanced reports feature if we were not using Maven?
    How do I write a bunch of java code that could do the same for me? Probably in the @AfterClass testng runner to parse the cucumber json report and create this report

  13. Hello Damien, Is there a way that i can publish the feature report or tags report through email after?
    Can you please share your inputs

  14. Hi Damien,
    I had the reporting plugin working fine when I used the Surefire plugin, but I switch to the Failsafe plugin (running cucumber integration tests) and no longer get the cucumber reports from my runs. (cucumber.json is created).
    Does the reporting plugin only work with Surefire?

  15. Hi Damien,
    I am using testNG dependancy with Surefire plugin. I am able to generate the reports if all the tests are passed otherwise reports are not getting generated, but cucumber.json file getting generated.

    Does this plugin expect to work with the only JUnit?

    Please suggest what can we do to generate the reports properly.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s