I am working on Azure pipelines, running on a Windows self-hosted agent, behind a firewall to run automated tests.
I would want to see the tests results and code coverage in the pipeline results in Azure. My build pipelines are working but I never saw yet the results of automated tests so I'm not sure of what to expect.
I run this in my pipeline:
- job: Test
steps:
- task: NuGetAuthenticate#0
- task: DotNetCoreCLI#2
inputs:
command: test
projects: 'src/***.Test.Core/*.csproj'
testRunTitle: '***Tests'
arguments: '--collect "Code coverage"'
publishTestResults: true
Here are extracts of the logs, with my questions and comments:
Initialization of proxies and general stuff
##[debug]Agent.ProxyUrl=http://***proxy.***.***:443
##[debug]Agent.ProxyBypassList=["***.***,localhost,127.0.0.1","***.***","localhost","127.0.0.1"]
##[debug]Agent.CAInfo=*:\***Certificates\***PKIG2PEM.txt
##[debug]Agent.ClientCert=undefined
##[debug]Agent.SkipCertValidation=undefined
##[debug]command=test
##[debug]projects=src/***.Test.Core/*.csproj
##[debug]arguments=--collect "Code coverage"
##[debug]publishWebProjects=true
##[debug]workingDirectory=*:\Agent\_work\24\s
##[debug]check path : *:\Agent\_work\_tasks\DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b\2.179.2\node_modules\packaging-common\module.json
##[debug]adding resource file: *:\Agent\_work\_tasks\DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b\2.179.2\node_modules\packaging-common\module.json
Not sure what is CHCP but tests are run, with errors I believe linked to firewall so I'd expect to see those errors in the tests results
[command]C:\Windows\system32\chcp.com 65001
##[debug]publishTestResults=true
##[debug]Agent.TempDirectory=*:\Agent\_work\_temp
##[debug]defaultRoot: '*:\Agent\_work\_temp'
##[debug]pattern: '**/*.trx'
Results File: E:\Agent\_work\_temp\***_***_2020-12-17_16_13_24.trx
Attachments:
*:\Agent\_work\_temp\d992768e-f36f-4558-9b52-f42fc7cd3c74\***_***_2020-12-17.16_13_16.coverage
Failed! - Failed: 29, Passed: 0, Skipped: 0, Total: 29, Duration: 3 s - OECD.Glue.Contacts.API.Test.Core.dll (netcoreapp3.1)
Errors when pushing the results, what means "Failed to get FF TestManagement.Server.UsePublishTestResultsLibInAgent Value." ?
##[debug]Exit code 1 received from tool 'C:\Program Files\dotnet\dotnet.exe'
##[debug]STDIO streams have closed for tool 'C:\Program Files\dotnet\dotnet.exe'
##[error]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1
##[debug]Processed: ##vso[task.issue type=error;]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1
##[debug]BuildConfiguration=undefined
##[debug]BuildPlatform=undefined
##[debug]testRunTitle=***Tests
##[debug]adjustedPattern: '*:\Agent\_work\_temp\**/*.trx'
##[debug]Failed to get FF TestManagement.Server.UsePublishTestResultsLibInAgent Value.
Unable to get the FF: TestManagement.Server.EnablePublishToTcmServiceDirectlyFromTask. Reason: One or more errors occurred. (Moved Temporarily)
Failed to get FF TestManagement.PTR.CalculateTestRunSummary, what is CalculateTestRunSummary ?
##[debug]Failed to get FF TestManagement.PTR.CalculateTestRunSummary Value.
##[debug]Reading test results from file '*:\Agent\_work\_temp\***_***_2020-12-17_16_13_24.trx'
##[debug]Processed: ##vso[results.publish type=VSTest;mergeResults=false;runTitle=GlueContactsTests;publishRunAttachments=true;resultFiles=*:\Agent\_work\_temp\*_***_2020-12-17_16_13_24.trx;testRunSystem=VSTS - dotnet;]
I wonder what to do with these warnings, I don't know much about Dotnet core and Nuget, my job is only the pipeline, not the underlying project. In project settings, I see .Net Core 3.1, I don't understand where to look to find if I use Nuget version < 5.7 and if this warning applies to me
##[warning].NET 5 has some compatibility issues with older Nuget versions(<=5.7), so if you are using an older Nuget version(and not dotnet cli) to restore, then the dotnet cli commands (e.g. dotnet build) which rely on such restored packages might fail. To mitigate such error, you can either: (1) - Use dotnet cli to restore, (2) - Use Nuget version 5.8 to restore, (3) - Use global.json using an older sdk version(<=3) to build
##[debug]Processed: ##vso[task.issue type=warning;].NET 5 has some compatibility issues with older Nuget versions(<=5.7), so if you are using an older Nuget version(and not dotnet cli) to restore, then the dotnet cli commands (e.g. dotnet build) which rely on such restored packages might fail. To mitigate such error, you can either: (1) - Use dotnet cli to restore, (2) - Use Nuget version 5.8 to restore, (3) Use global.json using an older sdk version(<=3) to build
##[debug]task result: Failed
Then I have this, publish test results ends with Moved Temporarily, I'm guessing this is because of firewall, still, I did setup the proxy and other calls to Azure seems to work, what could be the issue ?
##[error]Dotnet command failed with non-zero exit code on the following projects : *:\Agent\_work\24\s\src\***.Test.Core\***.Test.Core.csproj
##[debug]Processed: ##vso[task.issue type=error;]Dotnet command failed with non-zero exit code on the following projects : E:\Agent\_work\24\s\src\***.Test.Core\***.Test.Core.csproj
##[debug]Processed: ##vso[task.complete result=Failed;]Dotnet command failed with non-zero exit code on the following projects : E:\Agent\_work\24\s\src\***.Core\***.Test.Core.csproj
##[section]Async Command Start: Publish test results
##[warning]Failed to publish test results: Moved Temporarily
##[section]Async Command End: Publish test results
##[section]Finishing: DotNetCoreCLI
Thanks for any help, Claude
Update 1 ---------------------------------
Thank you.
I understood that my version of DotNet Core on the agent is not good,
the agent has version 5 while the project references version 3.1.
I added this task :
- task: UseDotNet#2
inputs:
packageType: 'sdk'
version: '3.1.x'
But I am behind a firewall, still I have these logs:
##[debug]agent.proxyurl=http://***proxy.***.***:443
##[debug]agent.proxyusername=undefined
##[debug]agent.proxypassword=undefined
##[debug]agent.proxybypasslist=["***.***,localhost,127.0.0.1","***.***","localhost","127.0.0.1"]
##[debug]agent.proxybypasslist=["***.***,localhost,127.0.0.1","***.***","localhost","127.0.0.1"]
##[debug]AZURE_HTTP_USER_AGENT=VSTS_c11775e3-cd05-4e03-abd2-35da98ef2f0f_build_164_0
Tool to install: .NET Core sdk version 3.1.x.
##[debug]task result: Failed
##[error]Failed to download or parse releases-index.json with error: {"code":"UNABLE_TO_GET_ISSUER_CERT_LOCALLY"}
Would it be best to install this version locally ? In that case, would I have to do something special so it uses the correct version ?
Update 2 ---------------------------------
Thanks for that link to a known issue. I noticed that issue doesn't mention an error I have: {"code":"UNABLE_TO_GET_ISSUER_CERT_LOCALLY"}
Can it be that the task tries to reach 'https://dotnetcli.blob.core.windows.net/dotnet/release-metadata/3.1/releases.json' but can't as we are behind a firewall ??
I don't understand why it tries to install DotNet Core as it is already installed but I don't know much about DotNet Core, I am only trying to run automated tests.
My last test included:
- task: NuGetAuthenticate#0
displayName: 'Authenticate with NuGet'
- task: UseDotNet#2
inputs:
packageType: 'sdk'
version: '3.1.101'
installationPath: '$(Agent.ToolsDirectory)/dotnet'
- task: DotNetCoreCLI#2
inputs:
command: test
projects: 'src/myProject.Test.Core/*.csproj'
testRunTitle: 'myProject'
arguments: '--collect "Code coverage"'
publishTestResults: true
Logs contains:
##[debug]agent.proxyurl=http://proxy.myCie.com:443
##[debug]agent.proxyusername=undefined
##[debug]agent.proxypassword=undefined
##[debug]agent.proxybypasslist=["myCie.com,localhost,127.0.0.1","myCie.com","localhost","127.0.0.1"]
##[debug]agent.proxybypasslist=["myCie.com,localhost,127.0.0.1","myCie.com","localhost","127.0.0.1"]
##[debug]AZURE_HTTP_USER_AGENT=VSTS_xxx_build_173_0
Tool to install: .NET Core sdk version 3.1.101.
##[debug]task result: Failed
##[error]Failed to download or parse releases-index.json with error: {"code":"UNABLE_TO_GET_ISSUER_CERT_LOCALLY"}
##[debug]Processed: ##vso[task.issue type=error;]Failed to download or parse releases-index.json with error: {"code":"UNABLE_TO_GET_ISSUER_CERT_LOCALLY"}
##[section]Finishing: UseDotNet
Thanks for any help
I think this is similar to GitHub issue #10969. It has something to do with internal GitHub repo/url. This is because the version which we specify in the task is searched in releases-index file, which is accessible to public as well:
You could also check the releases-index.json
{
"channel-version": "3.1",
"latest-release": "3.1.10",
"latest-release-date": "2020-11-10",
"security": false,
"latest-runtime": "3.1.10",
"latest-sdk": "3.1.404",
"product": ".NET Core",
"support-phase": "lts",
"eol-date": "2022-12-03",
"releases.json": "https://dotnetcli.blob.core.windows.net/dotnet/release-metadata/3.1/releases.json"
},
In this case, I would recommend you to reopen the Issue #10969.
I have a .NET Core 2.2 Unit Test Setup in an Azure Dev Ops build pipeline, these are xunit tests
steps:
- task: DotNetCoreCLI#2
displayName: 'Unit Test'
inputs:
command: test
projects: '**/Wintegrate.[Tt]est*/*.csproj'
arguments: '--configuration $(buildConfiguration) '
timeoutInMinutes: 5
They are failing for a couple of reasons
Failed Wintegrate.Tests.BoomiRestClientTests.GetDeployedPackages
Error Message:
Assert.Equal() Failure
Expected: OK
Actual: 0
and
Failed Wintegrate.Tests.BoomiServiceTests.GetAtomsAndMolecules
Error Message:
Newtonsoft.Json.JsonReaderException : Error reading JObject from
JsonReader. Path '', line 0, position 0.
Stack Trace:
at Newtonsoft.Json.Linq.JObject.Load(JsonReader reader, JsonLoadSettings
settings)
at Newtonsoft.Json.Linq.JObject.Parse(String json, JsonLoadSettings
settings)
at Wintegrate.Domain.BoomiService.GetAtomsAndMolecules() in
d:\a\1\s\Wintegrate.Domain\BoomiService.cs:line 37
at Wintegrate.Tests.BoomiServiceTests.GetAtomsAndMolecules() in
d:\a\1\s\Wintegrate.Tests\BoomiServiceTests.cs:line 53
The tests work fine when running locally from Visual Studio
.NET Core XUnit Test Azure Dev Ops, Package Issues
According to the error message:
Failed Wintegrate.Tests.BoomiServiceTests.GetAtomsAndMolecules Error
Message: Newtonsoft.Json.JsonReaderException : Error reading JObject
from JsonReader. Path '', line 0, position 0. Stack Trace:
This error should not come from the Azure Devops tool or your build definition. It is more related to the project or the environment settings.
So, please try to run the test on the your agent server directly, and check if still have the same issue.
Hope this helps.
I am getting below error while building Corda Version 1 (branch name - release-V1)
Task :core:test
net.corda.core.flows.ContractUpgradeFlowTest > 2 parties contract upgrade using RPC FAILED
java.util.concurrent.TimeoutException at ContractUpgradeFlowTest.kt:123
329 tests completed, 1 failed
FAILURE: Build failed with an exception.
Please help...
You can build the project without running the tests by using the command ./gradlew build -x test or gradlew.bat build -x test.
The integration tests can occasionally time-out, so I wouldn't be concerned by this issue. We are working to fix this.
In general, you shouldn't need to build Corda itself. The binaries for release-V1 are published to Maven, so will be downloaded automatically. You should probably work from the templates:
https://github.com/corda/cordapp-template-java
https://github.com/corda/cordapp-template-kotlin
I'm building a Web API using .Net Core. I'm building and running my unit tests as I go. When I run my tests, I get something like this:
The-Monarch:MyAwesomeService.Tests homr$ dotnet test
Project MyAwesomeService (.NETCoreApp,Version=v1.0) was previously compiled. Skipping compilation.
Project MyAwesomeService.Tests (.NETCoreApp,Version=v1.0) will be compiled because inputs were modified
Compiling MyAwesomeService.Tests for .NETCoreApp,Version=v1.0
Compilation succeeded.
0 Warning(s)
0 Error(s)
Time elapsed 00:00:01.2618204
xUnit.net .NET CLI test runner (64-bit osx.10.11-x64)
Discovering: MyAwesomeService.Tests
Discovered: MyAwesomeService.Tests
Starting: MyAwesomeService.Tests
Finished: MyAwesomeService.Tests
=== TEST EXECUTION SUMMARY ===
MyAwesomeService.Tests Total: 20, Errors: 0, Failed: 0, Skipped: 0, Time: 3.522s
SUMMARY: Total: 1 targets, Passed: 1, Failed: 0.
The-Monarch:MyAwesomeService.Tests homr$
I'm curious about the line that says SUMMARY: Total: 1 targets, Passed: 1, Failed: 0.. So far, I've never seen a total higher than "1" here. How do you get more than one "Test Target" in a single output, and what's the purpose?
I got problem when running flexunit4 unit test
Flexmojos 4.2-beta
Apache License - Version 2.0 (NO WARRANTY) - See COPYRIGHT file
Running tests /Users/mordesku/NetBeansProjects/way2learn/builder/target/test-classes/TestRunner.swf
[LAUNCHER] Using regular flashplayer tests
------------------------------------------------------------------------
Tests run: 0, Failures: 0, Errors: 0, Time Elapsed: 0 sec
------------------------------------------------------------------------
BUILD FAILURE
------------------------------------------------------------------------
Total time: 1:10.516s
Finished at: Fri Dec 07 09:53:52 CET 2012
Final Memory: 11M/213M
------------------------------------------------------------------------
Failed to execute goal org.sonatype.flexmojos:flexmojos-maven-plugin:4.2-beta:test-run (default-test-run) on project builder: Unexpected return code 9 - /Users/mordesku/NetBeansProjects/way2learn/builder/target/test-classes/TestRunner.swf -> [Help 1]
To see the full stack trace of the errors, re-run Maven with the -e switch.
Re-run Maven using the -X switch to enable full debug logging.
When i am running adl from command line (from target directory), got this output:
jablecznik:test-classes mordesku$ adl TestRunner.xml
error while loading initial content
jablecznik:test-classes mordesku$
Root node of TestRunner.xml looks like this:
<application xmlns="http://ns.adobe.com/air/application/2.6">
When I changed it to
<application xmlns="http://ns.adobe.com/air/application/3.5">
Test from command line run normally.
Has anyone has the same problem as i? How configure flexmojos to output TestRunner.xml witch difrent version of airapplication version?