sbt's test-only command can be used to run the tests found in a specific test class. With JUnit tests you can use test-only to run specific methods on a test class e.g. test-only mypackage.MyTestClass.test1Equals1 to run just that method.
Is such a thing possible with scalatest's more free-form test syntax, presumably by working out the name it uses internally to reference a specific test? If it isn't possible in FreeSpec (which is easy to imagine given its nature) is there a way to do it with a simpler testing approach like FunSuite?
For a general solution we need to wait for sbt#911, but apparently ScalaTest 2.1.3 has a support for running a specific test. Seth says:
This is now supported in ScalaTest 2.1.3 with:
test-only *MySuite -- -z foo
to run only the tests whose name includes the substring "foo". For exact match rather than substring, use -t instead of -z.
To run a test:
sbt test-only com.path.to.UnitTestSpec
To run a particular integration test:
sbt it:test-only com.path.to.IntegrationTestSpec
Related
Is there a way to run two separate test suites using it:testOnly with SBT?
I know that if I want to run one test suite I would use sbt "it:testOnly directory.testName", and that if I wanted to run a set of tests that all match a certain expression I could for example do sbt "it:testOnly directory.*", but I can't for the life of me figure out how to do two individual tests. I have tried
sbt "it:testOnly directory.test1 directory.test2"
and
sbt "it:testOnly directory.test1,directory.test2"
but neither of those work, and I can't find any documentation on it when I googled the issue.
Turns out I'd made a spelling mistake in the name of the second test. The correct format is
sbt "it:testOnly directory.test1 directory.test2"
The below is the code present in parallel_tests.txt
*** Settings ***
Library Parallel
*** Test Cases ***
Runner
Run Parallel Tests Hello World
Hello
[Tags] parallel
Log Hello123
World
[Tags] parallel
Log World123
I am trying to execute this test case file
from terminal using jybot as below:
jybot parallel_tests.txt;
I am getting the following error:
No keyword with name 'Run Parallel Tests' found.
How to execute both test cases Hello and World ,in parallel in robot framework using parallel library.
Slightly different use case but have you tried using pabot?
I haven't gone down the rabbit hole with this one so don't have an in-depth understanding but I do use it for cross-browser tests via sauceLabs. It will work differently from how you want in that it does parallel runs but by test suite rather than test case level. Perhaps this is configurable? Unfortunately as mentioned I just know enough to get it working for my needs and haven't done a deep dive.
Any questions let me know and I'll try to help.
Here's a quick example as requested, I've stripped lots of our stuff which is useless in your case but left some in so you can see how to build up. I changed the processes parameter from %processes% to 2, the end result is the same, in my case these values are coming from a teamcity build configuration:
pabot --processes 2 --variable environment:%environment% --name OS_X_10.9-Firefox --variable browser:%browser% --listener robot-tc-listener --include %includetags% --exclude %exclusiontags% --outputdir %PabotResults_DIR% --output output.xml --report report.html --log log.html tests
Which version of RobotFramework are you using? The Parallel library was removed in 2.5 (see this ticket, though there may be distant plans of reintroducing it).
I tried to reproduce your issue using this file with RF version 2.8, but there are several compatibility problems, as expected.
Are you sure that the library has been loaded properly? When it fails, in addition to the error, you will also get a message telling you that the keywords are unavailable.
I currently have a command line sbt -Dsome.configuration.option test doing what I want, but I would like it to apply that configuration option automatically for sbt test (and no other sbt phase). If my terminology is correct, then I want to set a Java Option for the Test Configuration. How do I do this?
Searching on these terms has led me to http://www.scala-sbt.org/release/docs/Testing.html but I have not yet been able to understand it.
This question looks similar to mine: Define custom test configurations in sbt
Try this:
testOptions in Test +=
Tests.Setup(() => sys.props += "some.configuration.option" -> "true")
Caveat:
Because you're not forking this mutates the state of the system property in the JVM running sbt itself.
Which means that after running test the first time it that system property will also be set if you, for instance, run your main from within sbt (run/runMain).
A "simple" question: how can I automatically run PHPunit tests with Arcanist?
According to the documentation I should first load a custom library. As stated here I should create a .arcconfig file and load the appropriate library.
So: I've create a dir "arc_libs" in my project and in the dir "src" I used arc liberate to generate the needed files. My config is now:
{
"project.name" : "arc_libs",
"phabricator.uri" : "https://phabricator.xxx.xxx.net/",
"unit.engine" : "PhpunitTestEngine",
"load" : ["arc_libs/src"]
}
The libary DOES get loaded because I can run arc unit
[matthijs#xx xxx]$ arc unit
No tests to run.
But as you can see there are no tests to run. We keep our tests in "project_root/tests" and as far as I understand the documentation I should create a __tests__ dir in "the module" (probably my arc_libs dir ?)
However I want to run my existing PHPunit test files, not new tests I need to create. I tried using a symlink etc but I cannot get it to work. Arcanist doesn't detect my tests.
So my question: How can I automatically run my EXISTING PHPunit tests with arcanist?
(note we use arc diff that should run arc unit automatically)
The documentation you linked won't be very useful - it's aimed at Phabricator developers who want to test their libraries. There is some user-facing documentation for customising unit test tasks, but it's not great. Fortunately, it's quite easy to get Arcanist to run your project's unit tests using the included PhpunitTestEngine:
Prepare a phpunit.xml file in your project root. This should be a standard PHPUnit configuration file. You can test this by running phpunit -c phpunit.xml.
Add a phpunit_config option to your .arcconfig:
{
"phabricator.uri": "https://phabricator.xxx.xxx.net/",
"unit.engine": "PhpunitTestEngine",
"phpunit_config": "phpunit.xml"
}
Run arc unit to test it out.
Although user documentation is thin on the ground, the source code for PhpunitTestEngine has some comments and is fairly concise. If you run into problems, reading through the test engine code can help you track it down.
$ arc unit --help
unit [options] [paths]
unit [options] --rev [rev]
Supports: git, svn, hg
Run unit tests that cover specified paths. If no paths are specified,
unit tests covering all modified files will be run.
By default, arc lint and arc unit are meant to be used as part of a process of making changes, so by default it only acts on changed files. Odds are, you don't have any changed files. You probably want to specify some paths, or run arc unit --everything to run all tests.
I need something like make i.e. dependencies + executing shell commands where failing command stops make execution.
But more deeply integrated with shell i.e. now in make each line is executed in separate context so it is not easy to set variable in one line and use it in following line (I do not want escape char at end of line because it is not readable).
I want simple syntax (no XML) with control flow and functions (what is missing in make).
It does not have to have support for compilation. I have to just bind together several components built using autotools, package them, trigger test and publish results.
I looked at: make, ant, maven, scons, waf, nant, rake, cons, cmake, jam and they do not fit my needs.
take a look at doit
you can use shell commands or python functions to define tasks (builds).
very easy to use. write scripts in python. "no api" (you dont need to import anything in your script)
it has good support to track dependencies and targets
Have a look at fabricate.
If that does not fulfill your needs or if you would rather not write your build script in Python, you could also use a combination of shell scripting and fabricate. Write the script as you would to build your project manually, but prepend build calls with "fabricate.py" so build dependencies are managed automatically.
Simple example:
#!/bin/bash
EXE="myapp"
CC="fabricate.py gcc" # let fabricate handle dependencies
FILES="file1.c file2.c file3.c"
OBJS=""
# build link
for F in $FILES; do
echo $CC -c $F
if [ $? -ne 0 ]; then
echo "Build failed while compiling $F" >2
exit $?
fi
OBJS="$OBJS ${F/.c/.o}"
done
# link
$CC -o $EXE $OBJS
Given that you want control flow, functions, everything operating in the same environment and no XML, it sounds like you want to use the available shell script languages (sh/bash/ksh/zsh), or Perl (insert your own favourite scripting language here!).
I note you've not looked at a-a-p. I'm not familiar with this, other than it's a make system from the people who brought us vim. So you may want to look over that.
A mix of makefile and a scripting language to choose which makefile to run at a time could do it.
I have had the same needs. My current solution is to use makefiles to accurately represent the graph dependency (you have to read "Recursive make considered harmful"). Those makefiles trigger bash scripts that take makefiles variables as parameters. This way you have not to deal with the problem of shell context and you get a clear separation between the dependencies and the actions.
I'm currently considering waf as it seems well designed and fast enough.
You might want to look at SCons; it's a Make-replacement written in Python.