It is often useful to show the results of our tests to others people who are not developers, like managers or executive. This tutorial shows how to integrate a Cucumber reporting a Java plugin in Maven project. The result is a simple web page, which does not require a special tool to be read.
This tutorial will cover 3 reports plugins : JSON, HTML Pretty and Cucumber Report 2.
Native JSON Result
How to:
pom.xml
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.damienfremont.blog</groupId> <artifactId>20160509-test-cucumber_plugin_reporting_2</artifactId> <version>0.0.1-SNAPSHOT</version> <packaging>jar</packaging> <properties> <cucumber.version>1.2.5</cucumber.version> <java.version>8</java.version> </properties> <dependencies> <dependency> <groupId>info.cukes</groupId> <artifactId>cucumber-junit</artifactId> <version>${cucumber.version}</version> <scope>test</scope> </dependency> <dependency> <groupId>info.cukes</groupId> <artifactId>cucumber-java8</artifactId> <version>${cucumber.version}</version> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.3</version> <configuration> <source>1.${java.version}</source> <target>1.${java.version}</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.18.1</version> <configuration> <testFailureIgnore>true</testFailureIgnore> <includes> <exclude>**/*BDDTest.java</exclude> </includes> </configuration> </plugin> </plugins> </build> </project>
RunBDDTest.java
package com.damienfremont.blog; import org.junit.runner.RunWith; import cucumber.api.CucumberOptions; import cucumber.api.junit.Cucumber; @RunWith(Cucumber.class) @CucumberOptions(strict = false, features = "features", format = { "pretty", "json:target/cucumber.json" }, tags = { "~@ignore" }) public class RunBDDTest { }
Demo:
Launch with JUnit or
mvn test
The result is at /target/cucumber.json
Native HTML Plugin
How to:
RunBDDTest.java
package com.damienfremont.blog; import org.junit.runner.RunWith; import cucumber.api.CucumberOptions; import cucumber.api.junit.Cucumber; @RunWith(Cucumber.class) @CucumberOptions(strict = false, features = "features", format = { "pretty", "html:target/site/cucumber-pretty", "json:target/cucumber.json" }, tags = { "~@ignore" }) public class RunBDDTest { }
Demo:
Launch with JUnit or
mvn test
The result is at /target/site/cucumber-pretty/index.html
Cucumber-Reports HTML Plugin
How to:
pom.xml
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.damienfremont.blog</groupId> <artifactId>20160509-test-cucumber_plugin_reporting_2</artifactId> <version>0.0.1-SNAPSHOT</version> <packaging>jar</packaging> <properties> <cucumber.version>1.2.5</cucumber.version> <java.version>8</java.version> </properties> <dependencies> <dependency> <groupId>info.cukes</groupId> <artifactId>cucumber-junit</artifactId> <version>${cucumber.version}</version> <scope>test</scope> </dependency> <dependency> <groupId>info.cukes</groupId> <artifactId>cucumber-java8</artifactId> <version>${cucumber.version}</version> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.3</version> <configuration> <source>1.${java.version}</source> <target>1.${java.version}</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.18.1</version> <configuration> <testFailureIgnore>true</testFailureIgnore> <includes> <exclude>**/*BDDTest.java</exclude> </includes> </configuration> </plugin> <plugin> <groupId>net.masterthought</groupId> <artifactId>maven-cucumber-reporting</artifactId> <version>2.0.0</version> <executions> <execution> <id>execution</id> <phase>verify</phase> <goals> <goal>generate</goal> </goals> <configuration> <projectName>cucumber-jvm-example</projectName> <outputDirectory>${project.build.directory}/site/cucumber-reports</outputDirectory> <cucumberOutput>${project.build.directory}/cucumber.json</cucumberOutput> <skippedFails>true</skippedFails> <enableFlashCharts>false</enableFlashCharts> <buildNumber>42</buildNumber> </configuration> </execution> </executions> </plugin> </plugins> </build> </project>
RunBDDTest.java
package com.damienfremont.blog; import org.junit.runner.RunWith; import cucumber.api.CucumberOptions; import cucumber.api.junit.Cucumber; @RunWith(Cucumber.class) @CucumberOptions(strict = false, features = "features", format = { "pretty", "json:target/cucumber.json" }, tags = { "~@ignore" }) public class RunBDDTest { }
Demo:
Launch with
mvn test verify
The result is at /target/site/cucumber-reports/feature-overview.html
Conclusion
JSON is OK for integration with other tools (jenkins plugin, reports, etc).
Pretty HTML is OK during development phase (like a JUnit or Surefire report for versionning).
Cucumber-Reports is good for reporting outside of the dev team (like managers).
Source
https://damienfremont.com/2016/05/09/how-to-cucumber-test-report-plugin-2-with-maven-and-java
https://github.com/DamienFremont/blog/tree/master/20160509-test-cucumber_plugin_reporting_2
References
https://cucumber.io/docs/reference/jvm
[…] EDIT: this post is deprecated. Please go to the new version here: How to Cucumber : Test Report Plugin 2 with Maven and Java […]
Hi Damien,
Do we need to execute runner class or we have to run via feature file for those cucumber-jvm report with charts.
Thanks,
Suresh
Hi Damien,
I need to generate report like the one Cucumber-reports HTML Plugin with features-overview.html file. Can u guide me on how to do that one.
Thanks,
Suresh
Hi,
Thanks for the such a amazing report. I am facing a problem.
Assume that this is my first run and no json already exists. When I run a test case I see json file not found exception even though file exists. When I run a different test case now, I could see the report is generated for previous run perhaps using the json generated for last run. And this keeps happening every next run generates report for previous run. I tried adding report code in junit afterclass but no luck. Could you please help me to resolve this issue?
Hi Mukund,
can u share the runner class how u can able to generate reports with graphical representation?
Thanks,
Suresh
Hi,
Could you please let me know if it is possible to include custom fields in the report dashboard.
For eg: Browser used = Firefox, environment = UAT etc.
Thanks
Hi,
Could you please help me to understand why feature-overview.html is not generating. Only index.html is generating
ya anil even I too facing the same issue but I need feature-overview.html
Guys, you need to run mvn verify after mvn test. i was able to generate the report.
Thks for the fix. Added to the post.
Can anyone share the sample small project like that above reports generated I was not able to generate feature-overview.html file which is in a graphical representation of the reports
Hi Does these reports work if I use TestNG insted of Junit? if not, how can I get these detailed reports with TestNG.
In my case the cucumber-report folder is not getting generated.
I am using cucumber with maven project
Hi, thanks for the great tutorial !!
I followed thru your steps and if I run TestRunner with junit, it runs fine but my report don’t get updated I have to do mvn install for it to update. how do I run my test and update reports at the same time?
in other workd If I clean and run my TestRunner class thru junit it doesn’t generate cucumber-report unless I do mvn install.
[…] How to Cucumber : Test Report Plugin 2 with Maven and Java […]
HI
I am new to automation I wrote same code in pom.xml in eclipse but can not generate feature-overview.html, I am running my tests through maven build in eclipse run configration
@CucumberOptions(features = “src/test/resources/feature”, plugin = { “json:target/cucumber.json”, “html:target/site/cucumber-pretty” })
can somebody help me with this pls?
Hi,I am getting below error..
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile (default-compile) on project XYZ: Compilation failure
this is solved.. Now I can able to generate the reports .Ran ‘mvn verify’ after ‘mvn test’. Thanks @manu.
Thks for the fix. Added to the post.
@Raveendar Reddy – Hi, I am getting the same error. Could you please let me know what did you do to solve this error.
@Shweta,
I did ‘mvn verify’ after ‘mvn test’ .. This solved the issue..
Hi Damien,
How do we configure this for multiple runs and spit out the reports in a new directory each time so we do not lose or overwrite previous test reports?
Hi,
Usually by using a CI plateforme like Jenkins or Bamboo. Each run/build result (target and source) will be save in a different folder (automatically deleted after some time, or saved if the build is tagged as a release).
Hi Damien,
How about if we would be doing it offline?
I even changed the directory in my POM to point to a new directory having timestamp information, but seems the code completely cleans up the reports directory
This is what I have in POM.xml right now
target/cucumber-reports/advanced-reports/${timestamp}
but each run the advanced-reports is cleared up it seems
Hi Damien,
I figured a way to do that.
Could you tell me how to use the advanced reports feature if we were not using Maven?
How do I write a bunch of java code that could do the same for me? Probably in the @AfterClass testng runner to parse the cucumber json report and create this report
Hi All,
When I run this locally it works but it doesn’t work on Jenkins. Can you please help me with the same?
Hello Damien, Is there a way that i can publish the feature report or tags report through email after?
Can you please share your inputs
Hi Damien,
I had the reporting plugin working fine when I used the Surefire plugin, but I switch to the Failsafe plugin (running cucumber integration tests) and no longer get the cucumber reports from my runs. (cucumber.json is created).
Does the reporting plugin only work with Surefire?