I'd like to measure the coverage from UI integration tests, so I need to apply OpenCover to the running ASP.NET application, not the test execution. (I already got coverage for unit tests by running OpenCover on VSTest.Console.Exe.)
When I launch the app from Visual Studio 2013, it looks like it's running iisexpress.exe. How could I figure out what command line VS is running so I could pass the same command line to OpenCover?
OK, I figured it out.
The right command line to use is iisexpress.exe /site:[sitename] where sitename matches whatever your web project is called in Visual Studio. Specifying just the path to the project will not provide the desired results.
And to get test coverage with OpenCover, you would run
OpenCover.console.exe -target:iisexpress.exe -targetargs:"/site:sitename" -register:user
assuming you're in the same directory as your web project build with the PDB files, otherwise you may need to use -targetdir argument to OpenCover
Related
I have recently learned about the Embedded debug type, which seems to be great - no longer do I need to upload PDBs to the Symbol Server during the CI build, which takes eternity when using the standard Index Sources & Publish Symbols task (for about 150+ projects it takes on average 9 minutes !!!)
However, the VS code coverage feature does not work without PDBs. So running dotnet test --collect "Code coverage" yields nothing if there are no PDBs.
What am I missing?
It seems, that there is no way to do that right now, according to open GitHub issue 1748 for adding this functionality.
You can wait until it'll be implemented or avoid using embedded PDBs for now (use full debug type instead)
I just want to know something about Katalon Studio. I have not worked in automation testing before but now I have some assignment about testing in Katalon.
My client wants to test in Katalon but his requirement is that he wants to run test cases on every build automatically and he also doesn't want to install Katalon IDE or any library he just want reference so that he just added that reference on every build so that all the test cases run automatically on every Dev build.
Is this possible using Katalon? Kindly help me, please. Thanks.
You have to establish full CI Pipeline for your requirements. My advice is, to use Katalon with Jenkins and your developers code repository (perhaps GIT or SVN). Than you are able to implement a server/slave pipeline, where you can execute your Katalon scripts on slave, every time DEV builds.
See:
Katalon/Jenkins Tutorial
I have a TFS (on premises version 15.105.25910.0) server with build and release management definitions. One of the definitions deploys a web site, the test assemblies and then runs my MSTest based Selenium tests. Most pass, some are not run, and a few fail.
When I attempt to view the test results in the TFS web portal the view of "failed" test results fails and it shows the following error message:
can't run your query: bad json escape sequence: \p. path
'build.branchname', line 1, position 182.
Can anyone explain how this fault arises? or more to the point what steps I might take to either diagnose this further or correct the fault
The troublesome environment and its "Run Functional Tests" task are shown below
Attempted diagnostics
As suggested by Patrick-MSFT I added the requisite three steps to a build (the one that makes the selenium tests)
Windows machine file copy (Copy MStest assembly containing selenium test to c:\tests on a test machine)
Visualstudio test agent deploy (to same machine)
Run functional tests (the assembly shipped in 1)
The test run (and have the same mix of pass fail, skipped) but the test results can be browsed just fine with the web pages test links.
Results after hammering the same test into a different environment to see how that behaves...
Well, same 3 steps (targeting the same test machine) in a different environment works as expected - same mix of results, but view shows results without errors.
To be clear this is a different (pre-existing) environment in the same release definition, targeting the same test PC. It would seem the issue is somehow tied to that specific environment. So how do I fix that then?
So next step, clone the failing environment and see what happens. Back later with the results.
Try to run the test with same settings in build definition instead of release. This could narrow down if the issue is related to your tests or task configuration.
Double check you have use the right settings of related tasks. You could refer related tutorial for Selenium test in MSDN: Get started with Selenium testing in a continuous integration pipeline
Try to run the same release in another environment.
Also go through your log files to see if there are some related info for troubleshooting.
I just got an ASP.NET 5 Web API Application running in Visual Studio code by following this guide and I can launch the application by executing the command "dnx . web". How can I make the application build on the fly when I make changes to the source file ?
Update - This seems to be an open issue. Link 1, Link 2
dnx . web --watch is the command that you are looking for.
However, there is a caveat when you do that from command line: when a file changes, the server is stopped.
When you do the same from VS 2015, VS knows how to restart the server when it stops.
I noticed I had the same problem, and it seems to work more or less as expected if you run your site without the debugger attached.
I think the debugger is more sensitive to changes in cs files, and does't want them to change while it's trying to trace/step through them.
At first I thought it was related to the CLR type you use to run it, but I can change that and it works as long as youre not debugging.
I am starting an ASP.Net application in MonoDevelop (blank slate, hobby). I recently used Selenium on a Java based project for a client so thought I would start implementing some testing right from the get go in this project.
I have a solution with two projects: "Website", and "Website Tests". Unfortunately, I can only run one project at a time. The Website Tests always fail because the website is not running.
Is there a way around this, or am I going in the wrong direction?
Updated Details:
Confirmed WebsiteTests is set as a "Library"
Project Options -> General -> Build -> Compile Target == Library
Exact steps taken
Select: "Website" Project
Select: Run with Debugging
Verify site is running
Select: View -> Pads -> Unit Tests
Select: Login -> TestTest (just a blank test, no contents)
RESULT: pop-up "An application is already running and will have to be stopped. Do you wish to continue?"
EXPECTED: ability to run tests
Running in separate MonoDevelop instances also has the same problem. This is unfortunate as it makes it impossible to step through your tests and site (have to choose one).
Selenium+Mono is a non-viable solution.
https://code.google.com/p/selenium/issues/detail?id=5258
Also, MonoDevelop (on Ubuntu) does not appear to have the ability to run multiple processes. This appears to extend to multiple instances of MonoDevelop.
As a work around, you can run the NUnit-Gui separately. Keep the NUnit GUI open as you develop the tests. This is a little old-school, but it works.
Open Test Project
Make changes to project
Compile
Switch to NUnit GUI window
Run Tests
This still doesn't work as the current Mono+Selenium stack can only run one test at a time. The test project needs to be shutdown and restarted between test runs.