I have a ASP.NET Web Forms project which is hosted on a Azure DevOps Repository. The Source Control is TFS.
I Created a continuous integration Pipeline in Azure Devops for the Repository.
Here is a Image of the Pipeline:
All the Task from the Pipeline are running correctly beside the VSTest-TestAssemblies Task.
My NUnit Test is failing in my local Visual Studio, as it should:
But when running my CI, it tells my that no test failed:
Here you can see my settings and the path to my tests of the VSTest-TestAssemblies Task:
This is what the Folder Structure of my Project looks like:
NR.TestAutomation
- NR.TestAutomation
- NR.TestAutomation.Tests
- bin
- obj
- DemoTests.cs
- NR.TestAutomation.Tests.csproj
- packages
- NR.TestsAutomation.sln
The Output of the CI Run tells me, that the folder exist and it even finds the test:
Source filter: **/*Tests/*.cs
SystemVssConnection exists true
vstest.console.exe "D:\a\9\s\NR.TestAutomation.Tests\DemoTests.cs"
Why are my failing tests not failing inside the Azure DevOps Pipeline?
You should specify the (output of the) TestProject to run, not the individual files that contain a test. The test runner cannot execute a .cs file.
So in short; the failing test doesn't run successful in an Azure DevOps pipeline... it doesn't run in the pipeline at all.
Test files
Run tests from the specified files. Ordered tests and webtests can be run by specifying the .orderedtest and .webtest files respectively. To run .webtest, Visual Studio 2017 Update 4 or higher is needed. The file paths are relative to the search folder. Supports multiple lines of minimatch patterns.
Source Visual Studio Test task - Arguments
Possibly interesting as well: more info on File matching patterns reference
EDIT:
what would be the proper Source Path to my tests?
Of course that depends on the name of your test project. Looking at your screenshots, you could use **/*.Tests.dll. This recursively matches all the assemblies where the name ends with .Tests.dll.
Since your project's name is NR.TestAutomation.Tests, the assembly should be named NR.TestAutomation.Tests.dll so it will match. It's also ready for future additions of test projects, as long as their name ends with .Tests.
Related
So, I generate the code coverage report by running dotnet test --no-build -l trx --collect "Code coverage". Unfortunately, it also computes the code coverage for the test assemblies. How do I tell it not to collect for test assemblies?
The answer is here - https://learn.microsoft.com/en-us/visualstudio/test/customizing-code-coverage-analysis?view=vs-2019 and here - https://learn.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2019
For some reason Microsoft thinks that scanning the test assemblies is a good default. Too bad for us. At the end I went up with the custom runsettings file that excludes the test assemblies manually and includes only those having our company name.
My dotnetcore app has one appsettings.json per environment (appsettings.json and appsettings.Development.json for example) and I would like to take advantage of this on my pipeline.
I see 2 options for the pipeline:
Build Artifact for Dev -> Deploy on Dev -> Build Artifact for Prod -> Deploy on Prod
or
Build Artifact -> Deploy on Dev -> Deploy on Prod
For the first option, I could set the environment as a parameter for the build.
For the second option, how could I build the App only once, and set the environment according to the current deployment step? Taking advantage of the multiple appsettings.json I have.
And finally, are these approaches aligned with the best practices? If not, what would be the best practices for pipelines with multiple environments?
Generally we can generate a single artifact, then deploy the artifact to different environments and perform the different transformations at any environment within it's own stage release phase. That means we can change and override the settings which defined in the appsettings.json in each release environment.
Please refer to File transforms and variable substitution reference on how to do the transformation with .json files.
Besides, we can try to install the Replace Tokens extension, then use Replace Tokens task to load and change the settings defined in the appsettings.json file in each release environment/stage.
You can also transform the settings or use File Creator to create a new appsettings.jsonfile to overwrite the existing one.
Below blogs for your reference:
Replace appsetting tokens in config files with Build & Release
Management in VSTS (TFS)
Transform configurations in a .NET Core 2.2 Web API using Azure
DevOps
Using custom appsettings.json with ASP.NET Core integration
tests
You could go with Azure AppConfiguration and add it as an extra source for the configuration. This way your building/releasing process stays extremely simple.
See this documentation: https://learn.microsoft.com/en-us/azure/azure-app-configuration/enable-dynamic-configuration-dotnet-core
It's very powerful: you can select only part of the configuration (through filters), you can have feature flags, and you can have secrets (from linked key vaults).
I am pretty new to CI/CD and mm currently struggling with getting VSTS to find my xUnit tests. My solution is a pure experiment, to try and isolate the problem and to learn. Before getting into my setup and what's been done so far, the result is this line in the VSTS build log:
Warning: No test is available in C:\agent\_work\1\s\Quotifier.Models.xUnit.Test\bin\Release\netcoreapp2.0\Quotifier.Models.xUnit.Test.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
From having googled the message I interpret it as VSTS not being able to find an xUnit test adapter. Is that a correct assumption? Also, I see the terms "test adapter", "test runner" and "test explorer" alot. Are these terms the same thing? Having checked out this article it seems they are indeed the same thing. Can anyone clarify?
Now, for my setup ...
The xUnit test project is a .NET Core 2 class library referencing NUget packages xunit and xunit.runner.visualstudio (2.2.0). The test project is also referencing a shared class library called "Quotifier.Models" which contains the tested code.
To make it easier to diagnose I am running a local (on prem) build agent on my PC. This enables me to investigate the file structure.
The build step of my build definition was initially a "Visual Studio Build" that built the whole solution. I can see that the resulting binaries ends up in
C:\agent\_work\1\s\Quotifier.Models.xUnit.Test\bin\Release\netcoreapp2.0.
Among the binaries are also xunit.runner.visualstudio.dotnetcore.testadapter.dll. Is this the xUnit test adapter VSTS can't find?
More googling tells me that the typical place for the test adapter to be found is in the $(Build.SourcesDirectory)\packages folder and, so, that's what the test step's "Path to custom test adapters" should be set to. That gave me this log entry:
Warning: The path 'C:\agent\_work\1\s\packages' specified in the 'TestAdapterPath' does not contain any test adapters, provide a valid path and try again..
I also checked that local folder and, indeed, the xunit NUget packages does not end up in there. Can anyone guess as to why?
Assuming the xunit.runner.visualstudio.dotnetcore.testadapter.dll is indeed the test adapter VSTS is looking for, I thought I should try and help it a bit. So I created a seperate MSBuild step and specified the output path via the "MSBuild arguments":
/p:OutputPath="$(build.binariesdirectory)/$(BuildConfiguration)/test-assemblies".
I then set the test step's "Path to custom test adapters" property to point at that same folder. With that I'm back to the original warning in the log:
Warning: No test is available in C:\agent\_work\1\b\release\test-assemblies\Quotifier.Models.xUnit.Test.dll. Make sure that installed ... bla ... bla
Now, to summarize ...
running xUnit test projects out of the box doesn't work with VSTS
The xUnit NUget packages does not end up in the /**/packages folder
I am assuming the xunit.runner.visualstudio.dotnetcore.testadapter.dll is the test adapter needed. Is this assumtion correct?
I'm out of ideas for now so any hints, links or suggestions will be greatly appreciated.
Use "dotnet test" to run Your NetCore-UnitTests.
The task is called ".NET Core" and has a command dropdown where you select build/restore/test etc... You also need to add "NET Core Tool Installer" if you are using a hosted agent
And your xunit-packages are not supposed to end up in "packages"-directory. The netcore2 projects handle nuget-packages differently and will reference them from the local nuget-cache
The question is a follow up to this one: Generate Web.Debug config which could be debugged](Generate Web.Debug.config which could be debugged)
I have defined a transformation for web.debug.config. During compilation I see the following:
Transformed Web.config using C:\data\Main\WebRole\Web.Debug.config into
C:\data\Main\obj\obj\x64\Debug\WebRole.csproj\TransformWebConfig\ [...]
transformed\Web.config.
Checked Web.config in the specified location - it is correct (transformation succeeded)
But when I start the service in the azure emulator I get an alert that
Why does it happen? Looks that incorrect web.config is taken. Where should I specify the location of correct (transformed) file?
The key thing to realise with web.config Transforms (and is mentioned in the answer to your linked question) is that they are only part of the story.
When you build your sources, the transformed web.config file is built into the /obj/ folder, ready for deployment.
It is only the act of deploying your solution somewhere that puts the transformed config file into use - as noted in the docs:
When you deploy the Web application by using the selected build configuration and by using either a deployment package or one-click publish, the Web.config file is transformed according to your specifications.
How are you running the application after you build it? You need to publish or deploy it using one of the built in mechanisms that support web transforms to see those changes on your site.
If you are running the emulator against the original source files, they won't see the transformed web.config file - which is why typically the debug build doesn't have any transforms and you then turn off debugging with your Release build which is then deployed to production.
As you're trying to test this in the emulator you should be able to do the following:
In the Solution Explorer, ensure you've selected a file within the project that runs in the emulator.
From the Build menu, select "Publish [Project Name".
In the Publish Wizard, create a new "Profile" using the "Custom" publish target.
In the "Connection" pane select "File System" as the publish method, and give it a suitable target location.
In the "Settings" pane choose the appropriate configuration (in your case probably "Debug"), and set any other options that you'd like.
Then press "Publish", and the project should be built, and then deployed to the new file location.
You should then be able to start the emulator from this newly published location, which will be using your transformed web.config.
I have found this solution and it works perfectly
https://translate.google.co.il/translate?hl=en&sl=de&tl=en&u=http%3A%2F%2Fwww.sascha-dittmann.de%2Fpost%2FWebConfig-Transformation-im-Windows-Azure-Compute-Emulator.aspx&anno=2
I'm struggling to get web.config transformations working with automated builds.
We have a reasonably large solution, containing one ASP.NET web application and eight class libraries. We have three developers working on the project and, up to now, each has "published" the solution to a local folder then used file copy to deploy to a test server. I'm trying to put an automated build/deploy solution in place using TFS 2010.
I created a build definition and added a call to msdeploy.exe in the build process template, to get the application deployed to the test server. So far, so good!
I then tried to implement web.config transforms and I just can't get them to work. If I build and publish locally on my PC, the "publish" folder has the correct, transformed web.config file.
Using team build, the transformation just does not happen, and I just have the base web.config file.
I tried adding a post-build step in the web application's project file, as others have suggested, similar to:
<target name="AfterBuild">
<TransformXml Source="Web.generic.config"
Transform="$(ProjectConfigTransformFileName)"
Destination="Web.Config" />
</target>
but this fails beacuse the source web.config file has an "applicationSettings" section. I get the error
Could not find schema information for the element 'applicationSettings'.
I've seen suggstions around adding arguments to the MSBuild task in the build definition like
/t:TransformWebConfig /p:Configuration=Debug
But this falls over when the class library projects are built, presumably because they don't have a web.config file.
Any ideas? Like others, I thought this would "just work", but apparently not. This is the last part I need to get working and it's driving me mad. I'm not an msbuild expert, so plain and simple please!
Thanks in advance.
Doug
I just went through this. Our build was a bit more complicated in that we have 8 class libraries and 9 web applications in one solution. But the flow is the same.
First off get rid of your after build target. You won't need that.
You need to use the MSDeployPublish service. This will require that it be installed and configured properly on the destination server. Check the following links for info on this part:
Note that the server in question MUST be configured properly with the correct user rights. The following sites helped me get that properly set up.
http://william.jerla.me/post/2010/03/20/Configuring-MSDeploy-in-IIS-7.aspx
http://vishaljoshi.blogspot.com/2010/11/team-build-web-deployment-web-deploy-vs.html
How can I get TFS2010 to run MSDEPLOY for me through MSBUILD?
The next part requires that your build definition have the correct MSBuild parameters set up to do the publish. Those parameters are entered in the Process > 3.Advanced > MS Build Arguments line of the build definition. Here's a hint:
(don't change the following for any reason)
/p:DeployOnBuild=True
/p:DeployTarget=MsDeployPublish
/p:CreatePackageOnPublish=False
/p:MSDeployPublishMethod=WMSVC
/p:SkipExtraFilesOnServer=True
/p:AllowUntrustedCertificate=True
(These control where it's going)
/p:MSDeployServiceUrl="https://testserver.domain:8172/msdeploy.axd"
/p:UserName=testserver\buildaccount
/p:Password=buildacctpassword
/p:DeployIisAppPath="MyApp - TESTING"
Obviously the user will have to be configured in IIS on the target server to be allowed access to that axd (see previous links). And the IisAppPath is the name of the website on the target server.
You won't have to do anything special for the config transformations as the build itself will take care of that for you. Just have the correct setting in the line at Process > 1. Required > Items to Build > Configurations To Build.
Instead of trying to do the deploy by adding tasks myself into the build process template, I followed advice in Vishal Joshi's blog post here.
Now the entire project is built and deployed and the web.config transformations work also. Brilliant!
I now have another problem to solve! The web application references web services and the build process results in an XmlSerializers dll. However, although this is built OK, it does not get deployed to the web host. I think this needs a new post!
Doug