After running a test case with Karate, some html reports are published with surefire plugin. In particular, I've found that there is an html report for each feature file. This is inconvenient when tests are run from an automated pipeline, like in my case, where I use htmlpublish Jenkins plugin to get a public link to access reports and spread them across slack channels or emails.
I've tried to add this snippet in my pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<aggregate>true</aggregate>
<!--also set this to link to generated source reports-->
<linkXRef>true</linkXRef>
</configuration>
</plugin>
But it had not the desired effect.
I'm trying to achieve a single index.html into the target/surefire-reports directory so i can publish and browse all test reports
Any suggestion?
Thanks
Are you using the parallel runner ? If not, please read up about it: https://github.com/intuit/karate#parallel-execution
Since we emit the cucumber-compatible JSON report in addition to the industry-standard JUnit XML format, you have the choice of any reporting solution that fits your needs. I think the maven-cucumber-reporting library should work for you - but you can decide: https://github.com/intuit/karate/tree/master/karate-demo#example-report
EDIT: For any other advanced needs, please consider writing your own report: https://stackoverflow.com/a/66773839/143475
Related
I'm using a package Xamanimation which has a dependency of Xamarin.Forms 4.1.0( written in its nuspec file):
<dependencies>
<group targetFramework=".NETStandard2.0">
<dependency id="Xamarin.Forms" version="4.1.0.581479" exclude="Build,Analyzers" />
</group>
</dependencies>
but I have built the Xamarin.Froms for my own and added the output dll files into my project's reference:
<Reference Include="Xamarin.Forms.Xaml">
<HintPath>..\thirdparty\xforms\Xamarin.Forms.Xaml.dll</HintPath>
</Reference>
according to the nuget's doc, I add the ExcludeAssets attribute(and other tests) to the section of PackageReference of Xamanimation:
<PackageReference Include="Xamanimation">
<IncludeAssets>compile</IncludeAssets>
<!-- <ExcludeAssets>compile</ExcludeAssets> -->
<!-- <PrivateAssets>all</PrivateAssets> -->
<!-- <ExcludeAssets>buildtransitive</ExcludeAssets> -->
<Version>1.3.0</Version>
</PackageReference>
but none of them work!
MSBuild will always use the trasitive dependency Xamarin.Forms.4.1.0 and ignore my own build ones( in which I have add new classes and use them in the main project so the linking failure is indicating the choosen is the old one).
so what's the correct method to exclude the transitive dependency?
I spend a whole day learning into this question and finally got a reasonable answer.
so I'd like to post my first LONG answer in stackoverflow by my poor english.
TL'DR:
the MSBuild's nuget plugin's ResolveNuGetPackageAssets target do the evil thing, make a custom target to revert it, go to bottom for the task code.
Studying story
first, I made a similar but simpler copy of this problem for studying.
after all, building a xamarin project is too slow,
the demo source is in github,
it has four project:
ConflictLib: the library used both by another lib and main application
DirectLib: the library used by main application, and is using ConflictLib
MyAppDev: the main application, with above two library as ProjectReference
MyAppConsumer: the other application, using DirectLib by PackageReference, and ConflictLib as ProjectReference. to test this situation, I pushed the ConflictLib and DirectLib to nuget.org, then made modify to local version of ConflictLib, so I can verify which one is using.
these projects and their relationship are very similar to my origin problem, the key point is : when application use a library with two different version simutanously, will(should) the local one(ProjectReference or HintPath) win?
for my origin case, xamarin project, it's No, so I come to study it.
for the test case, a dotnet core console project, it's Yes, so there must be something mysterious in the building process: MSBuild, which is a huge system, but now I'm going to dig into it.
then, I need a inspecting tool to find out what MSBuild does when building a project.
the simple tool is just invoke it in command line, it will display all the targets executed. a msbuild target is something like a target in makefile, and the tasks in target are similar to commands in makefile, the concept exist with slightly different term in many other system such as gradle, so it's easy to understand.
but, there are so many targets and tasks, and all they depend on others and interactive though Property and Items, it's hard to learn which target break my need from the text log.
fortunately, there's an advanced tool for inspecting everything in MSBuild: it's called MSBuild structured log viewer, I learn it from here.
now build the project with /bl option, it will produce a binary log file with full info, open it by the viewer mentioned above:(my origin xamarin project's build log)
obvious, the ResolveNuGetPackageAssets target changes the Reference items, which decides the final linked library assembly.
but why it doesn't make the wrong decision in the test case? let's view its log:
got the difference? -- there's no ResolveNuGetPackageAssets target!
it's same from ResolveReferences to ResolveAssemblyReferences, but diff in the nuget part.
while double clicking at ResolveAssemblyReferences, the viewer will open the targets file in which the target be defined.
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets
it's still same for both case:
ResolveAssemblyReferences doesn't depend on ResolveNuGetPackageAssets, so where does the later come? just click at it, the file opens:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\NuGet\16.0\Microsoft.NuGet.targets
it overrides ResolveAssemblyReferencesDependsOn and add the ResolveNuGetPackageAssets to dependency of ResolveAssemblyReferences.
the last question: why the above NuGet.targets file not appear in the test case? it could still answered by the viewer's Evaluation section:
clearly, this file is not imported because a property SkipImportNuGetBuildTargets set to true. after a simple search, I confirmed it's the default value in test case: it's set in Microsoft.NET.Sdk.targets. but in xamarin case, it's not set and means false, so all the things happened.
finally, I have to figure out some measures to fix the problem.
first of all, I won't add the SkipImportNuGetBuildTargets property to xamarin project, because I think it's a framework design and maybe has a great impact on others, I just want to fix a little, specific problem.
I decide to add a custom target immediately after the ResolveAssemblyReferences, remove the Nuget's Xamarin.Forms and add my own ones -- just revert what ResolveNuGetPackageAssets does.
the task code is simple (only just after I have written, actually it cost me lot of time to search for grammar/builtin function/etc and test):
notice how the Remove Item(see msbuild doc) works (and not work: the commented lines), I still don't understand it exactly, but it did work!
I wanted to customize my alllure report by adding 3 more extra fields to the xml file (Report) which gets generated in local system.
#Step is not working.
I tried by adding #Step annotation at parameterized method. But content in the #Step annotation is not reflecting in the xml report.
When i searched in google for this query, many were telling to add AspectJWeaver dependency in the pom.xml.
But, my project is not maven project and i don't have pom.xml.
In fact, i have aspectjweaver-1.8.9 jar in libraries.
Please guide me, is this configuration issue which i have to do to make #Step annotation work?
You just need to add it as javaagent to your VM options (e.g. in run configuration for IntelliJ IDEA). For win something like that
-javaagent:C:/Users/username/.m2/repository/org/aspectj/aspectjweaver/1.8.5/aspectjweaver-1.8.5.jar
In Apache Karaf, before I install my features.xml file I need to type.
feature:repo-add camel 2.17.3
Is there any way to express in the features.xml file some command which makes this occur ? So that I can avoid typing it on the command line inside Karaf.
The motivation for this question is that it is nice that generally when I add a features.xml file to the deploy directory of Karaf that it can pull down all the dependent bundles by itself automatically. I like the fact that it has this automatic feature and the only thing I need to send is the features.xml. So this leads to my desire to encapsulate all the work in the features.xml file so that I do not need to type anything into the Karaf console.
This works. You need to refer to the camel feature xml using the repository tag.
See the cxf-dosgi samples feature xml.
The repository xml tag must appear within the features XML tag. So its like:
<?xml version="1.0" encoding="UTF-8"?>
<features xmlns="http://karaf.apache.org/xmlns/features/v1.3.0" name="jump_micro">
<repository>mvn:org.apache.camel.karaf/apache-camel/2.17.3/xml/features</repository>
<feature description="jump_micro" version="1.0.0" name="jump_micro">
...
This is a shortcut for feature:repo-add mvn:org.apache.camel.karaf/apache-camel/2.17.3/xml/features
so no, this can't really be skipped as you somehow need to tell which feature file to use. If you skip the version it'll be that latest available version.
I've feel like I've tried everything we currently have a solution that upon checkin to TFS we force a build on CruiseControl.net. In our solution we use the Chutzpah JS Test Adapter. We were able to successfully use the Chutzpah.console.exe to fail the build if any of the JS tests fails, now we would like to fail the build on the coverage. I cannot find any way to have Chutzpah.console.exe output coverage to the XML file it generates.
I thought I could solve the problem by writing my own .xsl that would parse _Chutzpha.coverage.html. I was going to convert that to xml using the junit format that CruiseControl already can interpret. Since I just care about failing the build I was going to make the output of my transform just look like more unit tests that failed. in the xsl i would set the attribute failures > 0
<?xml version="1.0" encoding="UTF-8" ?>
<testsuites>
<testsuite name="c:\build\jstest\jstest\tests\TestSpec2.js" tests="1" failures="1">
<testcase name="Coverage" time="46" />
</testsuite>
</testsuites>
But I really can't since the incoming html has self closing tags.
So, now I want to just run Chutzpah.Console.exe and pipe the output to a file, because the console output does display the total average coverage, read that value, and fail the build if it drops below a threshold.
Is there a better option? Am I missing soemthing? I don't know that much about cruisecontrol.net
I think the output to a file, and parse that is the only option left.
A pity that the coverage info is not in the xml file :-(
This is in fact more a problem with Chutzpah than with CCNet,
think of CCNet as an upgraded task scheduler, it has a lot of options, but relies on the input it receives from the called program. If that one can not provide the data, you're stuck with these kind of workarounds :-(
There is another option, but it may be more of a trip than you're interested in embarking upon. Sonarqube is a server based tool for managing and analyzing code quality. It has a plugin called "Build Breaker", which allows you to fail the build if any number of code quality metrics are not met, including unit test coverage. Like I said, it's kind of involved because you have to set up a server and learn this new tool, but it's good to have anyway.
I use Chutzpah with the /lcov command line option so that it outputs the coverage to a file, then tell sonar to find the coverage in that file in the sonar configuration for that project. Then, you add a step to your cruise control.net build process to run the sonar analysis, and if you have configured your build breaker plugin right, it will fail the build in cruise control if the coverage is not at the level you specified.
So as you may or may not know, BlazeDS (open source version of LiveCycle Data Services) is a nice way to get your server-side Java and client-side Flex application to play together. Unfortunately, it does have several pitfalls that need to be corrected. I'll try to explain one of them here.
All of BlazeDS's configuration is written via XML files in the flex/ folder of your webapp. The default names are separated for clarity, such as services-config.xml, remoting-config.xml, messaging-config.xml, etc. In these configuration files (particularly services-config.xml), Channels are defined; these setup URIs and objects used to capture and send information between the server and the client. In these config files, it is quite common to use a syntax like so:
<channel-definition id="my-secure-amf" class="mx.messaging.channels.SecureAMFChannel">
<endpoint url="https://{server.name}:{server.port}/{context.root}/messagebroker/amfsecure" class="flex.messaging.endpoints.SecureAMFEndpoint"/>
<properties>
<add-no-cache-headers>false</add-no-cache-headers>
</properties>
</channel-definition>
Unfortunately, what they don't tell you is that some of these key-in replacements (ie: {context.root}) are not replaced dynamically upon execution but upon compilation of the WAR file you intend to distribute. Obviously not a good idea when switching domains.
So, instead I seek to dynamically define these channels. According to the documentation, that's all good and fine, but it only works if the channel already exists when the webapp is launched. I feel like that sort of defeats the point.
So my question is, how do you truly create channels dynamically so that both the client and the server recognize their existence?
Read this blog post; I believe it is what you're after.
I believe these xml config files have no direct relation to the server at all. They are used to tell the SWF how to find the server.
During Compile time of your Flex App; the services-config information is, in essence, hard coded into the SWF.