Pre 1.0 I ran a Main method in one instance of sbt and did integration testing against it in another instance of sbt. With 1.0 I get server already running errors. Do I now have to run the Main outside sbt or is there a 1.0 way to test while running?
Thanks, laughedelic bgRun should be more prominent in the docs
You can use bgRun to run your application in background and run tests against it. I didn't find any documentation on this feature, but it is mentioned in the sbt 1.0.0 release notes:
Another related feature that was added is the bgRun task which, for example, enables a server process to be run in the background while you run tests against it.
Related
I have a TFS (on premises version 15.105.25910.0) server with build and release management definitions. One of the definitions deploys a web site, the test assemblies and then runs my MSTest based Selenium tests. Most pass, some are not run, and a few fail.
When I attempt to view the test results in the TFS web portal the view of "failed" test results fails and it shows the following error message:
can't run your query: bad json escape sequence: \p. path
'build.branchname', line 1, position 182.
Can anyone explain how this fault arises? or more to the point what steps I might take to either diagnose this further or correct the fault
The troublesome environment and its "Run Functional Tests" task are shown below
Attempted diagnostics
As suggested by Patrick-MSFT I added the requisite three steps to a build (the one that makes the selenium tests)
Windows machine file copy (Copy MStest assembly containing selenium test to c:\tests on a test machine)
Visualstudio test agent deploy (to same machine)
Run functional tests (the assembly shipped in 1)
The test run (and have the same mix of pass fail, skipped) but the test results can be browsed just fine with the web pages test links.
Results after hammering the same test into a different environment to see how that behaves...
Well, same 3 steps (targeting the same test machine) in a different environment works as expected - same mix of results, but view shows results without errors.
To be clear this is a different (pre-existing) environment in the same release definition, targeting the same test PC. It would seem the issue is somehow tied to that specific environment. So how do I fix that then?
So next step, clone the failing environment and see what happens. Back later with the results.
Try to run the test with same settings in build definition instead of release. This could narrow down if the issue is related to your tests or task configuration.
Double check you have use the right settings of related tasks. You could refer related tutorial for Selenium test in MSDN: Get started with Selenium testing in a continuous integration pipeline
Try to run the same release in another environment.
Also go through your log files to see if there are some related info for troubleshooting.
I was wondering if anyone could recommend best practise for SBT builds using Bamboo. I see that is a Bamboo plugin for SBT however it is a) unsupported and b) isn't compatible with later versions of Bamboo. This combination would almost certainly be a blocker for us as using it could lead to a position where we couldn't take a Bamboo update (potentially fixing a security issue) because it would break all of our SBT builds.
Presumably you can just set up Bamboo to build SBT projects as a script task but I'm a bit worried about the experience here as it's not clear to me how things like failing tests and code coverage will be represented.
Is it possible to have a reasonably slick SBT and Bamboo setup without using the plugin or is Bamboo not a suitable CI system to use with SBT?
We do heavily rely on bamboo in our sbt workflows. The plugin works fine but the only benefit over a short inline script is the parsing of tests which is also available as another task.
We love having some portable build scripts in the projects which can be also used by bamboo.
So here is the starter guide:
have a good portable build script in your project (presumably bash script)
call this script in an inline script in bamboo (so you can do some other stuff as well, e.g. checkout submodules, choosing docker host, ...)
We use grunt protractor runner and have 49 specs to run.
When I run them in sauce labs, there are times it just runs x number of tests but not all. Any idea why? Are there any sauce settings to be passed over apart from user and key in my protarctor conf.js?
Using SauceLabs selenium server at http://ondemand.saucelabs.com:80/wd/hub
[launcher] Running 1 instances of WebDriver
Started
.....
Ran 5 of 49 specs
5 specs, 0 failures
This kind of output is usually produced when there are "focused" tests present in the codebase. Check if there are fdescribe, fit in your tests.
As a side note, to avoid focused tests being committed to the repository, we've used static code analysis - eslint with eslint-plugin-jasmine plugin. Then, we've added a "pre-commit" git hook with the help of pre-git package that would run the eslint task before every commit eventually prohibiting any code style violations to be committed into the repository.
I am writing e2e Tests for some JS application at the moment. Since I am not a JS developer I was investigating on this theme for a while and ended up with the following setup:
Jasmine2 as testing framework
grunt as "build-tool"
protractor as test runner
jenkins as CI server (already in use for plenty java projects)
Although the application under tests is not written in angular I decided to go for protractor, following a nice guide on howto make protractor run nicely even without angular.
Writing some simple tests and running them locally worked like a charm. In order to implicitly wait for some elements to show up in den DOM I used the following code in my conf.js:
onPrepare: function() {
browser.driver.manage().timeouts().implicitlyWait(5000);
}
All my tests were running as expected and so I decided to go to the next step, i.e. installation in the CI server.
The development team of the aplication I want to tests was already using grunt to build their application so I decided to just hook myself into that. The goal of my new grunt task is to:
assemble the application
start a local webserver running the application
run my protractor test
write some test reports
Finally I accomplished all of the above steps, but I am dealing with a problem now I cannot solve and did not find any help googling it. In order to run the protractor test from grunt I installed the grunt-protractor-runner.
The tests are running, BUT the implicit wait is not working, causing some tests to fail. When I added some explicit waits (browser.sleep(...)) everything is ok again but that is not what I want.
Is there any chance to get implicitly waiting to work when using the grunt-protractor-runner?
UPDATE:
The problem does not have anything to do with the grunt-protractor-runner. When using a different webserver I start up during my taks its working again. To be more precisley: Using the plugin "grunt-contrib-connect" the tests is working using the plugin "grunt-php" the test fails. So I am looking for another php server for grunt now. I will be updating this question.
UPDATE 2:
While looking for some alternatives I considered and finally decided to mock the PHP part of the app.
Trying to setup my integration flow and I have some tests that are quite destructive using the velocity-cucumber package.
First issue I find is that these tests are being run on the standard Meteor db. Which on localhost and dev is fine, but not so great for production. As far as I can tell the velocity-cucumber doesn't do anything with mirrors yet.
Because of this I have two cases where I need Meteor to launch in a specific way.
1) On the CI server I need for JUST the tests to run then exit (hopefully with the correct exit code).
2) On the production server I need Meteor to skip all tests and just launch.
Is this currently possible with Meteor command line arguments? I'm contemplating making demeteorize a part of the process, and then use standard node.js testing frameworks.
To run velocity tests and then exit, you can allegedly run meteor with the --test option:
meteor run --test
This isn't working for me, but that's what the documentation says it is supposed to do.
To disable velocity tests, run meteor with the environment variable VELOCITY set to 0. This will skip setting up the mirror, remove the red/green dot, etc.:
VELOCITY=0 meteor run