Specflow report differences between nunit and mstest - integration-testing

I'm going through an exercise to build some integration tests using specflow, and view some test results after the execution of these tests.
I'm seeing some differences in the quality of the report generated by the specflow executable between test results generated from nunit and mstest.
The mstest report seems to be more in keeping with what I want to see, however we as a company use nunit, and I would prefer to use nunit because of this, while generating the same report as we see from mstest.
The lacking report that we see from nunit looks like nunit report
The more desired report that we see from mstest looks like mstest report
The configuration I'm using for the specflow project is
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSections>
<section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />
</configSections>
<specFlow>
<trace
traceSuccessfulSteps="true"
traceTimings="true"
minTracedDuration="0:0:0.1"
stepDefinitionSkeletonStyle="RegexAttribute" />
<unitTestProvider name="nunit" /> <!-- delete as appropriate -->
<!-- unitTestProvider name="mstest" / -->
</specFlow>
</configuration>
The command line arguments I'm running are:
mstest:
MSTest.exe /testcontainer:%testAssembly% /resultsfile:%resultsTrx% /detail:testname
specflow.exe mstestexecutionreport %testProject% /testresult:%resultsTrx% /out:%resultsOut%
nunit:
nunit-console.exe /labels /out=%resultsTxt% /xml=%resultsXml% %testAssembly%
specflow.exe nunitexecutionreport %testProject% /xmlTestResult:%resultsXml% /testOutput:%resultsTxt% /out:%resultsOut%
What can I do to align these reports?

Related

NLog does not create log files on ASP.NET Core project

I am new to using NLog with ASP.NET Core, so I have followed the guide here:
https://github.com/NLog/NLog.Web/wiki/Getting-started-with-ASP.NET-Core-(project.json)
I have created the following nlog.config file at the root of the project directory:
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
autoReload="true"
internalLogLevel="Warn"
internalLogFile="c:\temp\internal-nlog.txt">
<extensions>
<add assembly="NLog.Web.AspNetCore"/>
</extensions>
<!-- define various log targets -->
<targets>
<!-- write logs to file -->
<target xsi:type="File" name="allfile" fileName="${basedir}\nlog-all-${shortdate}.log"
layout="${longdate}|${event-properties:item=EventId.Id}|${logger}|${uppercase:${level}}|${message} ${exception}" />
<target xsi:type="File" name="ownFile-web" fileName="${basedir}\nlog-own-${shortdate}.log"
layout="${longdate}|${event-properties:item=EventId.Id}|${logger}|${uppercase:${level}}| ${message} ${exception}|url: ${aspnet-request-url}|action: ${aspnet-mvc-action}" />
<target xsi:type="Null" name="blackhole" />
</targets>
<rules>
<!--All logs, including from Microsoft-->
<logger name="*" minlevel="Trace" writeTo="allfile" />
<!--Skip Microsoft logs and so log only own logs-->
<logger name="Microsoft.*" minlevel="Trace" writeTo="blackhole" final="true" />
<logger name="*" minlevel="Trace" writeTo="ownFile-web" />
</rules>
</nlog>
Inside a controller, I call a line like this one:
_logger.LogInformation("Entered CustomerRange method");
which returns the following in the output window in Visual Studio:
CustomerMgmtAPI.Controllers.CustomerController:Information: Entered CustomerRange method
However, the actual log files are never created by NLog. I was wondering if someone can point out the error in the NLog configuration here, since I have been reviewing the documentation of NLog for ASP.NET Core project and I can't find the error myself.
So the actual fix to the problem was the remove the first line from the nlog.config file:
<?xml version="1.0" encoding="utf-8" ?>
I remove this line and everything started working as expected. I also noticed that Visual Studio was giving me these errors when that line was present:
Invalid token 'Text' at root level of document.
Unexpected XML declaration. The XML declaration must be the first node in the document and no white space characters are allowed to appear before it.
It seems in this case that the NLog tutorial is broken, as I just took over this file from the sample for ASP.NET Core. I am using VS2017, so perhaps there is an incompatibility with this version of VS?
The dirty little secret about using NLog with ASP.NET Core is that you can configure and create logs just as you did in ASP.NET and ASP.NET MVC. You just use the regular NLog Nuget package like you normally would.
Just create an NLog.config in your root, etc. You don't even have to make any extra configurations in the config or elsewhere to get it to work. You just reference NLog in your class and then create a logger with the LogManager.
What this means is that you don't have all of the wireup in Program.cs etc.

publish a website through MSBuild

I am trying to publish an asp.net website through msbuild on command prompt, but I did not get success.
I tried by creating a new web application and execute below command, it works.
C:\Windows\system32>msbuild.exe "C:\VisualStudio 2012\Projects\HelloWorldSample\HelloWorldSample\HelloWorld\HelloWorldSample.csproj" /p:DeployOnBuild=true /p:PublishProfile="HelloDeploy"
But my problem is, in website I don't have .csproj file. That's why I am unable to execute above command for that.
So please any one help me how I can do deploy for a web site.
Thanks.
If your website has no csproj project file it most likely is a directory containing just your website files. You can convert it to a Web Application (thus getting a csproj file) or directly use msdeploy to generate a deployable package and then use msdeploy to deploy it to your website. This is wat actually would happen under the hood when you run msbuild /p:DeployOnBuild
To create a msdeploy deployable website you can use the following snippet
msdeploy.exe
-verb:sync
-source:iisApp="C:\development\mywebsite\the-website-dir"
-dest:package="C:\temp\mywebsite-package.zip"
-declareParamFile:"C:\development\mywebsite\parameters.xml"
You will need to create at a parameter for the site name in the parameters.xml file which will be packaged with the zipped website project
<?xml version="1.0" encoding="utf-8" ?>
<parameters>
<!--
This file contains (among others) references to web.config fields (xpath)
which will be 'parameterized' on package before deploy. The actual values will then be filled in based on the given deploy environment.
-->
<parameter name="IIS Web Application Name" tags="IisApp">
<parameterEntry kind="ProviderPath" scope="iisApp" match="" tags="IisApp" />
</parameter>
</parameters>
This will generate a deployable msdeploy package which you can upload to your IIS by using:
msdeploy.exe -source:package=c:\temp\mywebsite-package.zip -dest:auto,computerName=https://mysite.example.com:8172/MsDeploy.axd,userName=USERNAME,password=PASSWORD,authtype=Basic, -verb:sync -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension -setParamFile:example-setParameters.xml -allowUntrusted
Together with a setParameter file
<?xml version="1.0" encoding="utf-8" ?>
<parameters>
<setParameter name="IIS Web Application Name" value="mysite.example.com" />
</parameters>

MSBUILD script building solution, but not deploying publishing profile

I have created a publishing profile for my web page, and I can successfully publish using the web deploy interface in Visual Studio (right click project, deploy etc). It builds the project on my local PC, and copies the files across to destination IIS server.
But now I am trying to create a MSBuild command, which should do the same, but it is only building the solution, not copying it across to the server.
My MSbuild command looks like this, and I run it from the solution source directory
msbuild "test.co.za.sln" p:DeployOnBuild=true /p:PublishProfile="test.co.za" /p:UserName="domain\test" /p:Password="test"
Is there anything wrong with my build command? As you can see in the screenshot, it is building successfully, but no files are being copied across. Which makes me wonder if the publishing profile is executed at all.
I have tried various combinations of publishprofile paths, full paths, with extensions, without, nothing seems to copy my files accross.
My publishing profile looks like this App_Data\Publishprofile\test.co.za.pubxml
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<WebPublishMethod>MSDeploy</WebPublishMethod>
<LastUsedBuildConfiguration>Debug</LastUsedBuildConfiguration>
<SiteUrlToLaunchAfterPublish>http://test.co.za</SiteUrlToLaunchAfterPublish>
<LaunchSiteAfterPublish>True</LaunchSiteAfterPublish>
<MSDeployServiceURL>test.co.za</MSDeployServiceURL>
<DeployIisAppPath>test.co.za</DeployIisAppPath>
<SkipExtraFilesOnServer>True</SkipExtraFilesOnServer>
<MSDeployPublishMethod>WMSVC</MSDeployPublishMethod>
<UserName>domain\test</UserName>
<_SavePWD>True</_SavePWD>
</PropertyGroup>
</Project>
EDIT: This is an existing classic ASP website which I added to a solution, it doesn't contain a ".csproj" file, just a "website.publishproj", which doesn't seem to execute
It doesn't look like you are passing in the VisualStudioVersion property. You should include that. The value will either be 11.0, 12.0, or 14.0 based on the version of Visual Studio you are using. You should likely pass in a value for Configuration as well but it's not required. I've blogged about why this is important at http://sedodream.com/2012/08/19/VisualStudioProjectCompatabilityAndVisualStudioVersion.aspx.
You can find the docs for asp.net command line publishing at http://www.asp.net/mvc/tutorials/deployment/visual-studio-web-deployment/command-line-deployment
Looks like my "website.publishingproj" is never being built when I run the msbuild command. Not sure why, #Sayed do you perhaps know?
I had to create a targets file next to my solution called "after.solutionName.sln.targets" and copy the code from this answer : https://stackoverflow.com/a/15138373/1184603
<!--?xml version="1.0" encoding="utf-8"?-->
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="Deploy Website" AfterTargets="Build">
<Message Text="Starting Website deployment" Importance="high">
</Message>
<MSBuild Projects="$(MSBuildProjectDirectory)\MyWebs\website.publishproj"
BuildInParallel="true" />
</Target>
</Project>
It now publishes correctly from my msbuild script.

Sonar Analysis for website project - ASP.NET

I was trying to build and analyze a asp.net website project using maven
Command I used :
mvn sonar:sonar -e
I have single pom file in my root directory(same directory where website.sln is stored).
Directory structure:
|---Website (website project file)
|
|---website.sln
|---pom.xml
|
Result of executing command was : Build Error -Embedded error: Unable to execute maven plugin
I did same things and I could successfully build a web application project and console application project(which have visual studio project file). I thing reason for not working for website project is it does not have .csproj file inside website folder.
So how could successfully build and analyze a website project using maven.
Could someone please help me to fix this issue.
this is my pom file:
http://maven.apache.org/maven-v4_0_0.xsd">
<groupId>MindTree</groupId>
<artifactId>webbssite</artifactId>
<version>1.0</version>
<modelVersion>4.0.0</modelVersion>
<name>Maya</name>
<packaging>sln</packaging>
<url>http://maven.apache.org</url>
<properties>
<!--
NOTE : the versions and parameters may be defined as properties.
Prefer this option to the plugin configuration as it may be accessible to several plugins
-->
<!-- Name of the solution file, located in the same directory as the pom.xml -->
<visual.studio.solution>website.sln</visual.studio.solution>
<!-- Name pattern to recognize the test assemblies, so that unit tests are only launched on those,
and so that those are excluded from code coverage -->
<visual.test.project.pattern>*.Tests</visual.test.project.pattern>
<!-- Version of the .Net tools, which may be 2.0 or 3.5 only -->
<dotnet.tool.version>4.0</dotnet.tool.version>
<sonar.language>cs</sonar.language>
<msbuild.configurations>Debug</msbuild.configurations>
<maven.site.generateReports>false</maven.site.generateReports>
<sonar.dynamicAnalysis>false</sonar.dynamicAnalysis>
<sonar.cpd.skip>true</sonar.cpd.skip>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.sonar-plugins.dotnet</groupId>
<artifactId>maven-dotnet-plugin</artifactId>
<version>0.6</version>
<configuration>
<solutionName>website.sln</solutionName>
<language>cs</language>
<toolVersion>3.5</toolVersion>
<Platform>x86</Platform>
<buildConfigurations>Release,Debug</buildConfigurations>
<rebuild>true</rebuild>
</configuration>
<extensions>true</extensions>
</plugin>
</plugins>
</build>
You should rather send a mail to the user mailing list.
Anyway I have the feeling there is a misunderstanding between the "maven dotnet plugin" and the "sonar maven plugin".
The "maven dotnet plugin" provides the ability to build dotnet visual studio solution with maven. It relies mostly on msbuild.
The "sonar maven dotnet plugin" alows to bootstrap a sonar analysis with maven.
Check out the second pom example on the following page : Maven .NET Plugin
Both plugins need to be configured if you want to use maven to build and analyse your solution.
That being said, you should know that neither maven nor those two plugins are mandatory to analyse a C# project with sonar. You could use whatever build tool you want to compile your solution (i.e. msbuild, nant)and you could use the simple sonar java runner to trigger the analysis.
Check out the official wiki and feel free to ask questions to the user mailing list.
You can subscribe easily here : Support

How to precompile ASP.NET web application from TeamCity?

I have a question about precompiling ASP.NET web application projects from TeamCity. This is sort of a follow-up question to the following thread:
How to deploy after a build with TeamCity?
I'm done implementing CI from unit testing to autodeploy using the above thread and now I'd like to complement the process with precompiling the project. The project is rather large and I want to avoid any unnecessary delays in the response time after a new deploy.
So, is there a way to do this from TeamCity? Like calling MSBuild with some specific arguments?
Sure, it can be done with a custom MSBuild script. Here's the one we run to precompile our ASP.NET MVC 3 website (not that it really varies by ASP.NET version).
First it runs the regular build by running MSBuild against the solution file, then runs this custom MSBuild code:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>
<PropertyGroup>
<WebProject>Web\ChatPast.Web\ChatPast.Web.csproj</WebProject>
<WebProjectFolder>Web\ChatPast.Web</WebProjectFolder>
<WebPublishFolder>ChatPastWebPublish</WebPublishFolder>
</PropertyGroup>
<ItemGroup>
<ZipFiles Include="$(teamcity_build_workingDir)\src\ChatPast\$(WebPublishFolder)\**\*.*" />
</ItemGroup>
<Target Name="Build">
<!-- Compilation of all projects -->
<MSBuild Projects="ChatPast.sln" Properties="Configuration=Release"/>
<!-- Creating web publish folder. -->
<RemoveDir Directories="$(WebPublishFolder)"/>
<MakeDir Directories="$(WebPublishFolder)"/>
<!-- Running ASP.NET publish -->
<MSBuild Projects="$(WebProject)"
Targets="ResolveReferences;_CopyWebApplication"
Properties="Configuration=Release;WebProjectOutputDir=..\..\$(WebPublishFolder);OutDir=..\..\$(WebPublishFolder)\bin\" />
</Target>
</Project>

Resources