I've created a Release definition in TFS 2015 update 2 as seen below:
I want to use this as part of my CI process. The build and unit tests are executed on a dedicated BizTalk build server for each check-in. Executed nightly, this (currently problematic) release should deploy to an "DevTest" BizTalk server and execute integration tests.
All is good up to the last "Run Functional Tests" task. So, the BizTalk applications are removed, then deployed to the target server in the correct order. The "IntegrationTests" assembly contains a few Specflow tests which make use of the Transmock framework (http://transmock.codeplex.com/). Steps for these tests are contained within an assembly called "Common.Commponents". This is referenced from a test assembly called "Finance.IntegrationTests.dll". The tests pass when run on my Dev VM. However, when executed by the "Run Functional Tests" task, I get the following error:
Test method Finance.IntegrationTests.ASC_BacsFeature.BacsFileMoved threw exception: NUnit.Framework.InconclusiveException: No matching step definition found for one or more steps.using System;using TechTalk.SpecFlow;namespace MyNamespace{ [Binding] public class StepDefinitions { [Given(#"an output path specified by sso app '(.)' and setting '(.)'")]public void GivenAnOutputPathSpecifiedBySsoAppAndSetting(string p0, string p1){ ScenarioContext.Current.Pending();} [Given(#"the inpu
I have set the App.config of my test project to use MsTest as seen below:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSections>
<section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
</configSections>
<specFlow>
<!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
<unitTestProvider name="MsTest"/>
<stepAssemblies>
<stepAssembly assembly="LCC.Integration.Common.Components" />
</stepAssemblies>
</specFlow>
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1"/></startup></configuration>
It seems that when the tests are executed remotely, NUnit is used rather than MSTest.
The "Common.Components.dll" is successfully deployed to the GAC earlier in the release via GacUtil /I, so I'm not sure why it can't be found?
I've tried running the tests directly on the remote server. I RDP'd on, then opened a command window and ran "vstest.console Finance.IntegrationTests.dll". I then received warnings that the test input files could not be found, followed by errors because "Could not load file or assembly 'nunit.framework'".
I think the problem is, App.config of the Integration Test assembly is not being used when the tests are executed remotely. This explains why it tries to use NUnit rather than MSTest and also why it is unable to bind to the common steps - because their location is given in the node.
Does anyone know how I can make the App.config be used when testing remotely?
There were a couple of issues. I'd forgotten that the app.config would be compiled into finance.integrationtests.dll.config. References to dlls in the config needed to have the full strong name rather than just the name.
Related
In our .NET Core 3.1 project (REST API), we've multiple NuGet packages. General packages comes from the nuget.org source, some custom made packages are retrieved from a private source.
In Azure DevOps, we've a build pipeline with a task to restore the NuGet packages. Here we saw that every packages was checked with every source. A general package such as Swashbuckle.AspNetCore.SwaggerGen was also searched on our private source.
Due to the amount of requests from DevOps, the first attempt of the pipeline was interpreted as a DOS attack on our system. When the failed run was started again, the task succeeds without any error.
- task: DotNetCoreCLI#2
displayName: dotnet restore
inputs:
command: 'restore'
projects: '**/*.sln'
feedsToUse: 'config'
nugetConfigPath: 'src/NuGet.config'
In the tasks detail, we see the below message returning for every package.
GET private_source/nuget/FindPackagesById()?id='xunit.analyzers'&semVerLevel=2.0.0
Retrying 'FindPackagesByIdAsyncCore' for source 'private_source/nuget/FindPackagesById()?id='Microsoft.AspNetCore.Mvc.Razor'&semVerLevel=2.0.0'.
An error occurred while sending the request.
The response ended prematurely.
How can we avoid that every package in our solution, is checked at every NuGet source? Or what can we change to get a successfull build the first time?
NuGet recently introduced the feature, called Package Source Mapping: https://devblogs.microsoft.com/nuget/introducing-package-source-mapping/
Here's the nuget.config snippet from the blog post:
<!-- Define my package sources, nuget.org and contoso.com. -->
<!-- `clear` ensures no additional sources are inherited from another config file. -->
<packageSources>
<clear />
<!-- `key` can be any identifier for your source. -->
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
<add key="contoso.com" value="https://contoso.com/packages/" />
</packageSources>
<!-- Define mappings by adding package ID patterns beneath the target source. -->
<!-- Contoso.* packages will be restored from contoso.com, everything else from nuget.org. -->
<packageSourceMapping>
<!-- key value for <packageSource> should match key values from <packageSources> element -->
<packageSource key="nuget.org">
<package pattern="*" />
</packageSource>
<packageSource key="contoso.com">
<package pattern="Contoso.*" />
</packageSource>
</packageSourceMapping>
Regarding the error message:
An error occurred while sending the request. The response ended prematurely.
This suggests there's something wrong with the server or networking. A good nuget server should return HTTP 404 for packages that doesn't exist on it. Implementing package source mapping might not solve your restore problem.
I suggest creating an Azure Devops Artifacts feed having upstream source both from nuget.org and your private feed. There is no other way you can use multiple sources to do partial restore.
How can we avoid that every package in our .NET Core 3.1 project, is checked at every NuGet source during Azure DevOps pipeline?
I am afraid there is no such out of box way to resolve this restriction.
That's because no matter how we set the resource, when we restore the package for the first time, nuget.exe will iterate over each resource for every package. This problem will be alleviated when we run the pipeline again, because it is from nuget.org The packages will be cached in our private feed. When we restore again, it will be retrieved from the private feed first：
Check my previous thread for some more details.
Besides, If you want to avoid this problem the first time, you can try not to restore the entire .sln file, you can choose package.configs for the specify reject:
We recently swapped to NetCore projects and made needed changes to the pipeline. The pipeline task PublishSymbols#2 doesn't seem to be publishing symbols for our NuGet packages. At least, the Debugger isn't working anymore. Should I be using another task? Are my parameters wrong?
The pipeline sends out a warning that
##[warning]No files selected for indexing.
It also has the log file line
[Symbol App][INFO] 2021-07-02T11:41:51.7320634+00:00 Published 1 files.
projectsNuGet: |
**/ParkInd_Modular_CommonPlugins.#(csproj|pdb)
- task: PublishSymbols#2
displayName: Publish debug information to symbol servers
inputs:
SearchPattern: |
$(projectsNuGet)
!*.csproj
!**/obj/**
SymbolServerType: 'TeamServices'
The code didn't work because my dll had a different name than my csproj. Apparently the name of a dll is based on the AssemblyName not the csproj name. It makes sense.
I have a solution that includes three web projects (as well as a lot of class library projects). The web projects all use Web.config transforms to specify per-environment configuration.
I have Web.config transforms for multiple build profiles, named Web.UAT.config, Web.Staging.config and Web.Release.config
I am building and deploying the project from my CI server using MSBuild with the following arguments:
/t:Clean,Build /p:Configuration=UAT;DeployOnBuild=True;PublishProfile=UAT
For exactly one of the three projects, the web.config transforms appear to get applied twice, with elements marked xdt:Transform="Insert" appearing twice. Looking in the build output, it seems that all three projects run the following targets:
PreTransformWebConfig
TransformWebConfigCore
PostTransformWebConfig
PreProfileTransformWebConfig
But the problematic project also runs these targets (immediately after those listed above):
ProfileTransformWebConfigCore
PostProfileTransformWebConfig
The .csproj files for web projects include the following by default:
<Import Project="$(VSToolsPath)\WebApplications\Microsoft.WebApplication.targets" Condition="'$(VSToolsPath)' != ''" />
This file in turn imports \Web\Microsoft.Web.Publishing.targets, also under the VSToolsPath (on my dev machine, this corresponds to C:\Program Files (x86)\MSBuild\VisualStudio\v12.0).
The interesting segment of this file looks like the following:
<ProjectProfileTransformFileName Condition="'$(ProjectProfileTransformFileName)'=='' And '$(PublishProfileName)' != '' ">$(_ProjectConfigFilePrefix).$(PublishProfileName)$(_ProjectConfigFileExtension)</ProjectProfileTransformFileName>
<!--if $(TransformWebConfigEnabled) is also enabled and the ConfigTransform and ProfileTransform happen to have same filename, we default $(ProfilefileTransformWebCofnigEnabled) to false so it doesn't do double transform-->
<ProfileTransformWebConfigEnabled Condition="'$(ProfileTransformWebConfigEnabled)'=='' And '$(TransformWebConfigEnabled)' == 'true' And ('$(ProjectProfileTransformFileName)' == '$(ProjectConfigTransformFileName)')">False</ProfileTransformWebConfigEnabled>
The double transform was happening as a result of ProfileTransformWebConfigCore running, which is conditional on ProfileTransformWebConfigEnabled, which only defaults to false if the ProjectProfileTransformFileName and ProjectConfigTransformFileName are equal.
I added the following target to all three of my projects:
<Target Name="DebugWebConfigTransform" AfterTargets="PreProfileTransformWebConfig">
<Message Text="ProjectProfileTransformFileName: $(ProjectProfileTransformFileName)"/>
<Message Text="ProjectConfigTransformFileName: $(ProjectConfigTransformFileName)"/>
</Target>
For the problematic project, this target output the following:
DebugWebConfigTransform:
ProjectProfileTransformFileName: Web.UAT.config
ProjectConfigTransformFileName: Web.Release.config
Since these two values were different, the double transform was occuring for the reasons described above.
The reason the ProjectConfigTransformFilename was set to Web.Release.config was that the ProjectConfigurationPlatforms in my .sln file was incorrect. The .sln file's Configuration|Platform pair of UAT|Any CPU was being mapped to Release|Any CPU for this project.
I think it was actually applying the UAT and Release transforms as a result (due to the exact nature of my transforms and the order in which they were applied, this was indistinguishable from applying the UAT transform twice).
Updating the ProjectConfigurationPlatforms mapping in the solution file resolved the issue for me.
This issue was occurring for me because I had multiple projects in my solution configuration using different configurations.
It was running more than one web.config transforms because of this configuration:
After switching the projects to use the same configuration I no longer received the issues in my web.config.
It seems this can also happen if you leave off the Configuration msbuild parameter. The PublishProfile isn't good enough.
I had a different issue than all the answers. In my case, I had a profile named staging.pubxml which was using the configuration prod. On publish both the staging and the prod transformation would occur.
Turns out, the name of the .pubxml file also triggers a transform if the same configuration can be found. I simply changed the name of the file and it solved my issue.
I'm trying to deploy an ASP.NET web application to Azure. It's hybrid Web Forms, MVC, and WebAPI, and there are a TON of aspx/ascx files, such that they really need to be precompiled or every deploy will render the site sluggish for awhile.
I am trying to deploy via SCM integration with GitHub via kudu, with precompiled views, all merged to a single assembly.
Note that:
Deploy works fine with precompilation disabled.
Deploy works fine from Visual Studio
Build works fine if I copy the msbuild command from the Azure log, replace the relevant paths, and run it locally on my Windows 8.1 machine.
I've set up the Advanced Precompile settings as:
Don't allow precompiled site to be udpatable
Don't emit debug information
Merge all pages and control outputs to a single assembly = AppViews.dll
Here's the .deployment file for Azure
[config]
project = WebSite/WebSite.csproj
SCM_BUILD_ARGS=/p:Configuration=Release;PublishProfile=azure-prod /v:n
You notice I'm sending the verbosity /v to "normal" for extra diagnostic information.
Here is info I get toward the tail of the deployment log:
AspNetPreCompile:
D:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_compiler.exe -v \ -p D:\home\site\repository\WebSite\obj\Release\AspnetCompileMerge\Source -c D:\home\site\repository\WebSite\obj\Release\AspnetCompileMerge\TempBuildDir
GenerateAssemblyInfoFromExistingAssembleInfo:
Creating directory "obj\Release\AssemblyInfo".
D:\Windows\Microsoft.NET\Framework\v4.0.30319\Csc.exe /out:obj\Release\AssemblyInfo\AssemblyInfo.dll /target:library Properties\AssemblyInfo.cs
AspNetMerge:
Running aspnet_merge.exe.
D:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools\aspnet_merge.exe D:\home\site\repository\WebSite\obj\Release\AspnetCompileMerge\TempBuildDir -w AppViews.dll -copyattrs obj\Release\AssemblyInfo\AssemblyInfo.dll -a
aspnet_merge : error 1003: The directory 'D:\home\site\repository\WebSite\obj\Release\AspnetCompileMerge\TempBuildDir' does not exist. [D:\home\site\repository\WebSite\WebSite.csproj]
Done Building Project "D:\home\site\repository\WebSite\WebSite.csproj" (Build;pipelinePreDeployCopyAllFilesToOneFolder target(s)) -- FAILED.
Build FAILED.
It looks like aspnet_compiler.exe runs, but doesn't do what it's supposed to, which is why the TempBuildDir directory (supposed to be the output of the compiler) does not exist in time for the AspNetMerge target. Contrast that with my system, where that directory DOES in fact exist, containing the marker aspx/ascx/etc. files, static content, a PrecompiledApp.config file, and a whole mess of stuff in the bin directory.
aspnet_compiler.exe has an -errorstack flag but it's not clear to me how I could get MSBuild to add this just via the .deployment file, or even if that app is really even throwing an error.
I could just deploy via Visual Studio, but I would really like to take advantage of the SCM integration so I can just push to my prod branch and let it go. Any suggestions?
I replied on https://github.com/projectkudu/kudu/issues/1341, but copying my answer here in case someone lands here...
Way back, we had found that aspnet_compiler.exe was not working within Azure Websites due to how it dealt with the profile folder. We made a change at the time that's a bit of a hack but got us going: we turned it into a no-op, by pointing HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\aspnet_compiler.exe to our own dummy exe (D:\Program Files (x86)\aspnet_compiler\KuduAspNetCompiler.exe).
But trying it now, it appears to work correctly today, likely thanks to improvements in the Azure Websites hosting environment. So we will try getting rid of this hack and doing a full test pass to make sure it doesn't cause any major regressions. If all goes well, we can get that into production, which should enable those scenarios.
In the short term, you may be able to work around this by having your build script:
copy aspnet_compiler.exe from D:\Windows\Microsoft.NET\Framework\v4.0.30319 into your own site files, but under a different name (e.g. aspnet_compiler2.exe)
convince msbuild to use that one
Note: This GitHub issue on projectkudu will eventually make this solution obsolete, but for the meantime, that issue is filed as Backlog, and this works right now.
Thank you thank you David Ebbo. With this information, I was able to bootstrap my build to work for the short term.
First, I downloaded the aspnet_compiler.exe from the Azure instance using the Diagnostic Console available at https://{WEBSITE_NAME}.scm.azurewebsites.net/DebugConsole and added that to my own repository. This way there's no question about any difference between 32/64-bit, etc. I renamed it to azure_aspnet_compiler.exe in my repository.
Second, the AspNetCompiler task doesn't give you the option to change the tool name. It's hardcoded, but as a virtual property so it's overrideable. So I had to create my own task class, and package it in its own assembly, which I built in Release mode and also included in my repository.
public class AzureAspNetCompiler : Microsoft.Build.Tasks.AspNetCompiler
{
private string _toolName = "aspnet_compiler.exe";
protected override string ToolName
{
get { return _toolName; }
}
public string CustomToolName // Because ToolName cannot have a setter
{
get { return _toolName; }
set { _toolName = value; }
}
}
Next I needed to replace the AspNetPreCompile task in MSBuild, but I couldn't figure out how to do that directly. But that task wasn't doing anything anyway, so why not just run right after it?
I added this to the top of my Website.csproj file to import the DLL containing the AzureAspNetCompiler class. Note that the path is relative to the Website.csproj file I'm editing.
<UsingTask TaskName="AzureBuildTargets.AzureAspNetCompiler"
AssemblyFile="..\DeploymentTools\AzureBuildTargets.dll" />
Then I added this right below it, which is basically stealing the MSBuild target definition of AspNetPreCompile from C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Web\Transform\Microsoft.Web.Publishing.AspNetCompileMerge.targets, with some of the property setting stuff near the top of it left out (because the original task will do that for us anyway.) Just take note of the ToolPath and CustomToolName values at the bottom of the (renamed) AzureAspNetCompiler element.
<PropertyGroup>
<!--Relative to solution root apparently-->
<LocalRepoDeploymentTools>.\DeploymentTools</LocalRepoDeploymentTools>
<AzureAspnetCompilerPath>$([System.IO.Path]::GetFullPath($(LocalRepoDeploymentTools)))</AzureAspnetCompilerPath>
</PropertyGroup>
<Target Name="NoReallyAspNetPreCompile" AfterTargets="AspNetPreCompile">
<AzureAspNetCompiler
PhysicalPath="$(_PreAspnetCompileMergeSingleTargetFolderFullPath)"
TargetPath="$(_PostAspnetCompileMergeSingleTargetFolderFullPath)"
VirtualPath="$(_AspNetCompilerVirtualPath)"
Force="$(_AspNetCompilerForce)"
Debug="$(DebugSymbols)"
Updateable="$(EnableUpdateable)"
KeyFile="$(_AspNetCompileMergeKeyFile)"
KeyContainer="$(_AspNetCompileMergeKeyContainer)"
DelaySign="$(DelaySign)"
AllowPartiallyTrustedCallers="$(AllowPartiallyTrustedCallers)"
FixedNames="$(_AspNetCompilerFixedNames)"
Clean="$(Clean)"
MetabasePath="$(_AspNetCompilerMetabasePath)"
ToolPath="$(AzureAspnetCompilerPath)"
CustomToolName="azure_aspnet_compiler.exe"
/>
<!--
Removing APP_DATA is done here so that the output groups reflect the fact that App_data is
not present
-->
<RemoveDir Condition="'$(DeleteAppDataFolder)' == 'true' And Exists('$(_PostAspnetCompileMergeSingleTargetFolderFullPath)\App_Data')"
Directories="$(_PostAspnetCompileMergeSingleTargetFolderFullPath)\App_Data" />
<CollectFilesinFolder Condition="'$(UseMerge)' != 'true'"
RootPath="$(_PostAspnetCompileMergeSingleTargetFolderFullPath)" >
<Output TaskParameter="Result" ItemName="_AspnetCompileMergePrecompiledOutputNoMetadata" />
</CollectFilesinFolder>
<ItemGroup Condition="'$(UseMerge)' != 'true'">
<FileWrites Include="$(_PostAspnetCompileMergeSingleTargetFolderFullPath)\**"/>
</ItemGroup>
With this in place, everything works as I would expect it to.
I am using the following MSDeploy command to pull content and settings from a production server to a staging server nightly.
msdeploy -verb:sync -source:webServer,wmsvc=xx.xx.xx.xx,username=xxxxx,password=xxxxxx,authType=basic -allowUntrusted=true -dest:webServer
As documented here the "webServer" provider uses several other providers. Is there any way to exclude the "machineConfig32" and "machineConfig64" linked providers from being executed?
Thanks.
Updated:
Using a manifest works:
<sitemanifest>
<appHostConfig path="siteNameHere" />
<appHostSchema />
<contentPath path="siteNameHere" />
<rootWebConfig32 />
<rootWebConfig64 />
</sitemanifest>
with MSDeploy command:
msdeploy" -verb:sync -source:manifest=d:\msdeploy\deploymanifest.xml,wmsvc=xx.xx.xx.xx,username=xxxxxx,password=xxxxxx,authType=basic -allowUntrusted=true -dest:manifest=d:\msdeploy\deploymanifest.xml
You can just skip those objects from the sync operation all together by adding the following to your command line: -skip:objectName=machineconfig32 -skip:objectName=machineConfig64
The linked extensions you can exclude are covered here and here:
I don't see yours listed there, but it may be worth a try. Sometimes this doco is out of date.
That said, you may want to try to "build up" less inclusive providers using manifests rather than "tearing down" an all-inclusive one. Try the following link for the info.. (package manifests)
Then you can have a package command like: (my IIS6 standard package command, note this won't work exactly as it's IIS6, just serving up an example)
-verb:sync -source:manifest=$manifestFile -dest:package=$appName.zip,encryptPassword=MyPassword -enableLink:AppPoolExtension -disableLink:CertificateExtension -disableLink:ContentExtension -declareParamFile:$parametersFile"
and deploy:
-verb:sync -source:package=$appname.zip,encryptPassword=MyPassword -dest:auto -setParamFile=$appname_$computernamesetParameters.xml
In the manifest I then have the site, the content directory, a registry location I need, and even com components and the like.
As far as I know - there is no direct way to exclude linked providers. However - you can sync individual providers by getting the dependency list.