We are using Geb and Spock framework for testing web application and integrated with an extent report. We have created a separate class for extent report functionalities using JUnit #Rule to access among all Test cases. Now the problem is when TC fails, it's not generating a report for a failed scenario, instead it's creating a report for succeeded case. Is there any way to catch TestWatcher failed exception using Spock TC's. Or is there any similar class like TestWatcher in Spock which performs below functionality.
#Rule
public TestRule watchman = new TestWatcher() {
#Override
public Statement apply(Statement base, Description description) {
return super.apply(base, description)
}
#Override
protected void failed(Throwable e,Description description) {
TestUtils.takeScreenshot("testimg", "png", driver)
println("fail")
ExtentReports extent = createReport()
ExtentTest test = extent.startTest(description.getMethodName(), "Test failed, click here for further details")
// step log
String img = test.addScreenCapture(".testimg.png")
test.log(LogStatus.FAIL, "Failure : "+ e.toString(),img)
flushReports(extent, test)
}
#Override
protected void succeeded(Description description) {
ExtentReports extent = createReport()
ExtentTest test = extent.startTest(description.getMethodName(), "-")
test.log(LogStatus.PASS, "Passed")
flushReports(extent, test)
}
#Override
protected void skipped(AssumptionViolatedException e, Description description) {
ExtentReports extent = createReport();
ExtentTest test = extent.startTest(description.getMethodName(), "-")
// step log
test.log(LogStatus.SKIP, "Skipped")
flushReports(extent, test)
}
#Override
protected void finished(Description description) {
browser.close()
}
}
private ExtentReports createReport() {
ExtentReports extent = new ExtentReports(".testReport.html", false);
return extent
}
private void flushReports(ExtentReports extent, ExtentTest test){
extent.endTest(test)
extent.flush()
}
Spock has some basic support for JUnit rules.
If your particular case does not work you need to write a Spock extension that gives you hookups to the test lifecycle.
See also Execute some action when Spock test fails
Related
I am having trouble figuring out how I can use the structured logging, when NLog is in a wrapper class. I have an asp.net webapp that calls my nlog wrapper class.
It works fine for regular logging ex. logger.Info("Gets to here.") but i can't get it to work for structured logging calls ex. logger.Info("This is structured logging entry #{param1}", new {Status = "processing"})
This is my Wrapper class for NLog(EventLog.LogManager):
public static void Info(params object[] args)
{
private static readonly NLog.Logger Logger = NLog.LogManager.GetCurrentClassLogger();
Logger.Log(LogLevel.Info, args);
//This is what I've tried to no avail.
//var ci = new CultureInfo("en-US");
//LogEventInfo le = LogEventInfo.Create(LogLevel.Info, Logger.Name, ci, args);
//le.Parameters = args;
//string test = le.FormattedMessage;
//string test1 = string.Format(le.Parameters[0].ToString(), le.Parameters[1].ToString());
//Logger.Log(typeof(LogManager), le);
}
This is my asp.net application that calls the above method:
public ActionResult Index()
{
EventLog.LogManager.Info("Test #{param1}", new { OrderId = 2, Status = "Processing" });
return View();
}
If anyone can point me in the right direction, it would be greatly appreciated. Thank you in advance.
Try changing into this:
namespace EventLog
{
public static class LogManager
{
private static readonly NLog.Logger Logger = NLog.LogManager.GetCurrentClassLogger();
public static void Info(string message, params object[] args)
{
Logger.Info(message, args);
}
}
}
I'm using Java+TestNG+Allure. I need to get all test fails in Allure report, not only the first fail of the test but all, and the test should run from the beginning to the end despite failed steps.
For reporting the test failures in Allure report we have to do little bit of modifications in Allure Class. Here we want to report any of the sub step as a failure, execute the remaining steps and then mark the main test step as a failed test. For doing this we can use the concept of SoftAssertions. I had created one class called as AllureLogger. Inside the class we will have 5 Methods.
1)starttest() 2)endtest() 3) markStepAsPassed(String message) 4)marstepAsFailed(String message) 5)logStep().
public class AllureLogger {
public static Logger log = Logger.getLogger("devpinoylog");
private static StepResult result_fail;
private static StepResult result_pass;
private static String uuid;
private static SoftAssert softAssertion;
public static void startTest() {
softAssertion = new SoftAssert();
}
public static void logStep(String discription) {
log.info(discription);
uuid = UUID.randomUUID().toString();
result_fail = new StepResult().withName(discription).withStatus(Status.FAILED);
result_pass = new StepResult().withName(discription).withStatus(Status.PASSED);
}
public static void markStepAsFailed(WebDriver driver, String errorMessage) {
log.fatal(errorMessage);
Allure.getLifecycle().startStep(uuid, result_fail);
Allure.getLifecycle().addAttachment(errorMessage, "image", "JPEG", ((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES));
Allure.getLifecycle().stopStep(uuid);
softAssertion.fail(errorMessage);
}
public static void markStepAsPassed(WebDriver driver, String message) {
log.info(message);
Allure.getLifecycle().startStep(uuid, result_pass);
Allure.getLifecycle().stopStep(uuid);
}
public static void endTest() {
softAssertion.assertAll();
softAssertion = null;
startTest();
softAssertion = new SoftAssert();
}
}
In the above class, we are using different methods from allureClass and we are doing little bit of modification to add soft assertions.
Everytime we start a TestMethod in testClass we can call the starttest() and end testmethod().Inside the test methods if we have some substeps we can use try catch block to mark the substeps as pass or fail.Ex-Please check the below test method as an Example
#Test(description = "Login to application and navigate to Applications tab ")
public void testLogin()
{
AllureLogger.startTest();
userLogin();
navigatetoapplicationsTab();
AllureLogger.endTest();
}
Above is a test method which will login to one application and then navigate to application tab.Inside we have two methods which will be reported as substeps, 1)login()- For logging in the application 2) navigatetoapplicationsTab()-to navigate to application tab. If any of the substep fails then the main step and substep will be marked as fail and remaining steps will be executed.
We will define the body of the above functions which are defined in test method as below:
userLogin()
{
AllureLogger.logStep("Login to the application");
try
{
/*
Write the logic here
*/
AllureLogger.MarStepAsPassed(driver,"Login successful");
}
catch(Exception e)
{
AllureLogger.MarStepAsFailed(driver,"Login not successful");
}
}
navigatetoapplicationsTab()
{
AllureLogger.logStep("Navigate to application Tab");
try
{
/*
Write the logic here
*/
AllureLogger.MarStepAsPassed(driver,"Navigate to application Tab successful");
}
catch(Exception e)
{
e.printStackTrace();
AllureLogger.MarStepAsFailed(driver,"Navigate to application Tab failed");
}
}
Everytime any exception is thrown they will be caught in catch block and reported in the Allure Report. The soft assertion enables us to execute all the remaining steps successfully.
Attached is a screenshot of an Allure report generated by using the above technique.The main step is marked as Failed and remaining test cases have got executed.
The report attached here is not from the above example which is mentioned. It is just a sample as how the report would look.
I am using a suite file with multiple suites inside the testng.xml file as follows:
<suite-files>
<suite-file path="suite1"></suite-file>
<suite-file path="suite2"></suite-file>
</suite-files>
I am initializing ExtentReport in BeforeSuite.
private static void initializeExtentReport(Configuration config) {
if (extent == null) {
extent = new ExtentReports();
htmlReporter = new ExtentHtmlReporter("reportLocation");
ClassLoader classLoader = ExtentReportService.class.getClassLoader();
File extentConfigFile = new File(classLoader.getResource("extent-config.xml").getFile());
htmlReporter.loadXMLConfig(extentConfigFile);
extent.attachReporter(htmlReporter);
extent.setSystemInfo("Environment", config.getAutomationServer());
}
}
In AfterSuite, I am calling flush().
So basically the issue is, when the before suite is called for the second suite, The check (extent==null), is coming false. I also went through the JavaDocs for ExtentReports and I found a method detachReporter() there. But I am not able to access by my IDE. Tried many variations but to no fruition.
EDIT:
Right now what really happens is, I am using custom names for reports, so that no two report names are the same. And, when I was using with the same name, the results would be over written in the same file for the suites.
A better approach here is to use a singleton like so:
public class Extent
implements Serializable {
private static final long serialVersionUID = 1L;
private static class ExtentReportsLoader {
private static final ExtentReports INSTANCE = new ExtentReports();
static {
}
}
public static synchronized ExtentReports getInstance() {
return ExtentReportsLoader.INSTANCE;
}
#SuppressWarnings("unused")
private ExtentReports readResolve() {
return ExtentReportsLoader.INSTANCE;
}
}
Usage:
ExtentReports extent = Extent.getInstance();
So your code becomes:
private static void initializeExtentReport(Configuration config) {
extent = Extent.getInstance();
if (extent.getStartedReporters().isEmpty()) {
htmlReporter = new ExtentHtmlReporter("reportLocation");
ClassLoader classLoader = ExtentReportService.class.getClassLoader();
File extentConfigFile = new File(classLoader.getResource("extent-config.xml").getFile());
htmlReporter.loadXMLConfig(extentConfigFile);
extent.attachReporter(htmlReporter);
extent.setSystemInfo("Environment", config.getAutomationServer());
}
}
I would further recommend getting rid of all shared variables for extent/htmlReporter and directly use the Singleton
I have a Spring Boot / Batch application and would like to write several integration tests. The batch has a FlatFileItemReader and pulls in the file to be read in via yml configuration file. Here's the batch config class:
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
#Value("${file}")
private File file;
#Bean
public ItemReader<MyClass> reader(LineMapper<MyClass> lineMapper) {
FlatFileItemReader<MyClass> flatFileItemReader = new FlatFileItemReader<MyClass>();
flatFileItemReader.setResource(new FileSystemResource(file));
final int NUMBER_OF_HEADER_LINES = 1;
flatFileItemReader.setLinesToSkip(NUMBER_OF_HEADER_LINES);
flatFileItemReader.setLineMapper(lineMapper);
return flatFileItemReader;
}
The integration test class for testing the reader is:
#SpringApplicationConfiguration(classes = LoadApplication.class)
#TestExecutionListeners( { DependencyInjectionTestExecutionListener.class,
StepScopeTestExecutionListener.class })
#RunWith(SpringJUnit4ClassRunner.class)
public class StepScopeTestExecutionListenerIntegrationTests {
#Autowired
private ItemReader<MyClass> reader;
#Rule
public TemporaryFolder testFolder = new TemporaryFolder();
#Bean
public StepExecution getStepExection() {
StepExecution execution = MetaDataInstanceFactory.createStepExecution();
return execution;
}
#Test
public void testGoodData() throws Exception {
try {
File testFile = testFolder.newFile();
PrintWriter writer = new PrintWriter(testFile, "UTF-8");
writer.println("a,b,c");
writer.println("1,2,3");
writer.close();
//ReflectionTestUtils.setField(someObject, "file", testFile);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
assertNotNull(reader.read());
}
}
In the above test code, what should someObject be set to? Or is there some other way to inject the test file?
The reader has already been created before the call to testGoodData. Simply override the resource that was set on the reader.
That is, replace
ReflectionTestUtils.setField(someObject, "file", testFile);
with this line of code
reader.setResource(new FileSystemResource(testFile));
I'm learning Retrofit and RxJava and I'v created test to connect github:
public class GitHubServiceTests {
RestAdapter restAdapter;
GitHubService service;
#Before
public void setUp(){
Gson gson = new GsonBuilder()
.setFieldNamingPolicy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES)
.create();
restAdapter = new RestAdapter.Builder()
.setEndpoint("https://api.github.com")
.setConverter(new GsonConverter(gson))
.build();
service = restAdapter.create(GitHubService.class);
}
#Test
public void GitHubUsersListObservableTest(){
service.getObservableUserList().flatMap(Observable::from)
.subscribe(user -> System.out.println(user.login));
}
when I execute test, I see nothing in my console. But when I execute another test
#Test
public void GitHubUsersListTest(){
List<User> users = service.getUsersList();
for (User user : users) {
System.out.println(user.login);
}
it works, and I see user's logins in my console
Here is my Interface for Retrofit:
public interface GitHubService {
#GET("/users")
List<User> getUsersList();
#GET("/users")
Observable<List<User>> getObservableUserList();
}
where I'm wrong?
Because of the asynchronous call your test completes before a result is downloaded. That's typical issue and you have to 'tell' test to wait for the result. In plain java it would be:
#Test
public void GitHubUsersListObservableTest(){
CountDownLatch latch = new CountDownLatch(N);
service.getObservableUserList()
.flatMap(Observable::from)
.subscribe(user -> {
System.out.println(user.login);
latch.countDown();
});
latch.await();
}
Or you can use BlockingObservable from RxJava:
// This does not block.
BlockingObservable<User> observable = service.getObservableUserList()
.flatMap(Observable::from)
.toBlocking();
// This blocks and is called for every emitted item.
observable.forEach(user -> System.out.println(user.login));