I've just used a large part of the day figuring this one out.
Here is the problem: Being used to use MSTest in Bamboo and it works fine.
The first project using Asp.Net.Core, with XUnit tests comes along, and needs to be setup in Bamboo.
Bamboo doesn't support XUnit test result xml files.. Sigh...
What to do?
After upgrading to .Net Core SDK 1.1.1 final bits, this is much easier achived.
Executing the below will execute test, and generate a trx output.
dotnet test --logger trx
Alternatively to determine filename as well
dotnet test --logger "trx;LogFileName=myTestResults.trx"
After a lot of fiddling around this here is the recipe I used.
Execute tests as usual, output result to xml file
dotnet test .\MyProject\test\UnitTests -xml .\TestResults\UnitTests.xml
Run the output xml through an XSLT conversion and convert to MsTest trx format
$xml = Process-XSLT $PSScriptRoot\TestResults\UnitTests.xml $PSScriptRoot\BuildScripts\XUnitToMsTest.xlst
And here comes the culprit. Ensure the TRX xml files is written as UTF-8 - not an ascii file. Tried for hours to figure out why Bamboo wouldn't pick up the test files - until I realized this.
Out-File -FilePath $PSScriptRoot\TestResults\UnitTests.trx -InputObject $xml -Encoding UTF8
Sources:
XUnit to TRX : https://github.com/deloitte-solvas/XSLT-xUnit-To-Trx
ProcessXSLT : https://gist.github.com/wschwarz/5073004 (Had a few bugs)
Entire source to my own ProcessXSLT:
function Process-XSLT([string]$inputFile, [string]$xsl)
{
$fileStream = New-Object -TypeName System.IO.FileStream($inputFile, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read);
$fileStream.position = 0
$xml = new-object System.Xml.XmlTextReader($fileStream)
$output = New-Object System.IO.MemoryStream
$xslt = New-Object System.Xml.Xsl.XslCompiledTransform
$arglist = new-object System.Xml.Xsl.XsltArgumentList
$reader = new-object System.IO.StreamReader($output)
$xslt.Load($xsl)
$xslt.Transform($xml, $arglist, $output)
$output.position = 0
$transformed = [string]$reader.ReadToEnd()
$reader.Close()
return $transformed
}
When all of the above is done correctly, all left to do is to add a MSTest Parser task to your Bamboo build, pointing at the TestResults folder.
https://confluence.atlassian.com/bamboo/mstest-parser-289277057.html
I hope I save someone else a few minutes with this post.
Bamboo was absolutely no way near telling me the files was invalid - it just stated that no results could be picked up. :-)
Best regards
/Anders
Related
We are getting the tests to execute from our testcase management tool and have stored their id in the list .
Example:
List<string> caseid = new List<string>() [ "tc1", "tc2", "tc3" ]
Now I want to execute all the test cases at once/simultaneously to take advantage of my parallel execution system where I an utilise up to 24 browser instance.
Currently I am iterating over this list and executing as
dotnet test --filter TestCategory=caseid[i]
Here testcase id is also used as #tag.
If there is a way to execute all the tests in the list at once
Try the following syntax:
dotnet test --filter "TestCategory=tc1|TestCategory=tc2|TestCategory=tc3"
If you have large number of tests, you can just generate .sh file for MacOS or .cmd file for windows to run desired batch of tests:
var filter = String.Join("|", caseid.Select(id => $"TestCategory={id}"));
File.WriteAllText("./runbatch.sh", $"dotnet test --filter \"{filter}\"");
and then run tests in terminal:
sh runbatch.sh
I'm able to set the project version by adding the tag to the .csproj file,
but I'd like to change that value programatically inside the pipeline.
I hacked together a script to do accomplish that, but it feels sloppy.
That's got to be a way to do this through the CLI, but I'm just not finding it.
Is there a command similar to this that I'm overlooking?
dotnet build -project-version 1.2.3
If no command exists, what have you done to set the project build version in your pipelines?
i use the following in my pipeline.
dotnet build -p:VersionPrefix="$(buildNumber)" -p:VersionSuffix="$(buildPipline)"
and in my csproj file i have
<PropertyGroup>
...
<VersionPrefix>1.0.1</VersionPrefix>
<VersionSuffix>local</VersionSuffix>
</PropertyGroup>
to get the version number in code, i use the following
public static string Version => System.Reflection.Assembly.GetEntryAssembly().GetCustomAttribute<System.Reflection.AssemblyInformationalVersionAttribute>().InformationalVersion;
if you're using azure pipelines, you can add a powershell script, which declares the version based on the date & time.
Write-Host "Generating Build Number"
$baseDate = [datetime]"01/01/2019"
$currentDate = [System.TimeZoneInfo]::ConvertTimeBySystemTimeZoneId((Get-Date), 'Eastern Standard Time')
$interval = NEW-TIMESPAN –Start $baseDate –End $currentDate
$days = $interval.Days
$hour = $currentDate.ToString("HH")
$minute = $currentDate.ToString("mm")
$version = "1.0.$days.$hour$minute"
$version_npm = "1.$days.$($currentDate.ToString("Hmm"))"
if($currentDate.ToString("HH") -eq "00")
{
$version_npm = "1.$days.25$($currentDate.ToString("mm"))"
}
Write-Host "Version: $version"
Write-Host "npm Version: $version_npm"
Write-Host "##vso[task.setvariable variable=buildNumber]$version"
Write-Host "##vso[task.setvariable variable=buildNumber_npm]$version_npm"
You may try Assembly Info extension, which can update project AssemblyInfo files and .Net Core / .Net Standard project files .csproj. For detailed instructions on how to configure the extension please see the following link:
https://github.com/BMuuN/vsts-assemblyinfo-task/wiki/Versioning
I found the easiest way to do this is using environment variables in the .csproj file like this:
<PropertyGroup Condition="'$(APPVERSION)' != ''">
<InformationalVersion>$(APPVERSION)</InformationalVersion>
</PropertyGroup>
And then set the environment variables in your build server.
I am using lintr library in R to find linting issues in the code. I put them into an xml format like this:
<lintsuites>
<lintissue filename="/home/.../blah.R" line_number="36" column_number="1" type="style" message="Trailing blank lines are superfluous."/>
<lintissue filename="/home/.../blahblah.R" line_number="1" column_number="8" type="style" message="Only use double-quotes."/>
</lintsuites>
Now, I would like to fail the Azure devops build when issues like this occur.
I was able to get my tests in a JUnit format like this:
<testsuite name="MB Unit Tests" timestamp="2020-01-22 22:34:07" hostname="0000" tests="29" skipped="0" failures="0" errors="0" time="0.05">
<testcase time="0.01" classname="1_Unit_Tests" name="1_calculates_correctly"/>
<testcase time="0.01" classname="1_Unit_Tests" name="2_absorbed_correctly"/>
...
</testsuite>
And when i do this step in the azure pipeline, my build fails if any tests in the test suite fail:
- task: PublishTestResults#2
displayName: 'Publish Test Results'
inputs:
testResultsFiles: '**/*.xml'
searchFolder: '$(System.DefaultWorkingDirectory)/fe'
mergeTestResults: true
failTaskOnFailedTests: true
I would like something similar for failing the build when there are linting issues. I would also like the users to see what those linting issues are in the build output.
Thanks
This is not possible to achieve a similar result for lintr xml with plishTestResults#2.
The workaround you can try is to use a powershell task to check for the content of your lintr xml file. If the content isnot empty, then fail the pipeline in the powershell task.
Below powershell task will check the content of lintr.xml(<car></car>) and will echo the content to the task logs and exit 1 to fail the task if the content is null.
- powershell: |
[xml]$XmlDocument = Get-Content -Path "$(system.defaultworkingdirectory)/lintr.xml"
if($XmlDocument.OuterXml){
echo $XmlDocument.OuterXml
}else{exit 1}
displayName: lintr result.
You can aslo use below statement in a powershell task to upload lintr xml file to the build summary page where you can download
echo "##vso[task.uploadsummary]$(system.defaultworkingdirectory)/lintr.xml"
You can check here for more logging commands.
Update:
A workaround to show the lintr results in a nice way is to create a custom extension to display html results in azure devops pipeline.
You can try creating a custom extension, and produce a html lint results. Please refer to the answer to this thread an example custom extension to display html
Other developers already submit requests to Microsoft for implementing this feature. Please vote it up here or create a new one.
I work on a .Net Core solution in which we just added .Net Core lambdas.
The newly created lambdas are all set, including the aws-lambda-tools.json.
All I have left to do is to automatically publish those lambdas using TeamCity.
(Continuous deployment is already set on TC for the rest of the solution)
Also I'd rather not update a TC build step every time we add a new Lambda.
How can I setup TC to automatically publish all lambdas?
Shall I use the .Net CLI or are there any plugin to help me configure this step?
Finally, I opted for a very simple powershell script at the root of the solution, that deploys all the projects whose aws-lambda-tools-defaults are set:
$lambdaProfile = "aws-lambda-tools-defaults.json"
$solutionFolder = (Get-Item -Path ".\" -Verbose).FullName;
$lambdaFolders = Get-ChildItem -Path $solutionFolder -File -Recurse aws-lambda-tools-defaults*.json | ForEach-Object {$_.DirectoryName } | Select-Object -uniq
forEach ($lambdaFolder in $lambdaFolders)
{
Write-Output "Deploying following lambda: $lambdaFolder"
Write-Output "with profile: $lambdaProfile"
Set-Location $lambdaFolder
dotnet lambda deploy-function -cfg $lambdaProfile
Set-Location $solutionFolder
}
Then just execute this script in a specific build step on TeamCity,
I am trying to get the results of a php unit tests folder , but not using the CLI, and instead using a php file.
I would like to get the answer ( OK, 4 tests passed ) in a variable or something so I can decide whether the script should execute or not, what's the best way to do this? i don't want to use batch files, I want to force the execution of the tests, inside the library itself.
I started with
require_once 'PHPUnit/Autoload.php';
When I included the tests, don't know how to start them thought.
Any advice?
You can check PHPUnits exit code, 0 means no test failed.
To run the tests from a php file, then to check the exit code, try something like this:
//Composer installs phpunit to /vendor/bin/phpunit
exec('/vendor/bin/phpunit', $result, $exitCode);
if ($exitCode == 0) {
// continue exiting the script
} else {
// there was a test failure, more info will be in $result
}
If you're looking for an enterprise quality solution, look into a Continuous Integration product like Jenkins or Bamboo.