I am using ScalaTest's AsyncFlatSpec style.
Is there any way I can ensure Second and Third test case only starts execution once First is completed?
Here's the code sample :-
class SomeAsyncSpec extends AsyncFlatSpec with Matchers {
var mutableVariable = 0
"First test case" should "perform foo asynchronously" in {
println("First TEST CASE")
performFooFutureCall(mutableVariable)
//assert something
}
"Second test case" should "perform bar asynchronously" in {
println("Second TEST CASE")
performBarFutureCall(mutableVariable)
//assert something
}
"Third test case" should "perform baz asynchronously" in {
println("Second TEST CASE")
performBazFutureCall(mutableVariable)
//assert something
}
}
From the code it's obvious that the reason to execute tests in this suite one after another is to not affect shared mutable state.
In that case it'll help from the line on AsyncFlatSpec's documentation :-
https://github.com/scalatest/scalatest/blob/78e506537ed06d4438e1d365e18c160d46d2bf18/scalatest/src/main/scala/org/scalatest/AsyncFlatSpec.scala#L233
Because ScalaTest's default execution context on the JVM confines execution of Future transformations
* and call backs to a single thread, you need not (by default) worry about synchronizing access to mutable state
* in your asynchronous-style tests.
Have a look at documentation from the link provided for more info.
Related
Our code defines some "rules" in a List<Rule> collection. Each rule contains some logic in the form of a string. The rules are all passed to a "rule engine" along with some data. The rule engine sequentially evaluates the data against rules until it finds a rule which evaluates as true, then it returns that Rule.
We want to make to automatically test every rule. The tests will actually be integration tests, rather than unit tests, because they'll test the combination of the rule engine and the rules.
How do I write a test that says "make sure each rule evaluates as true in at least one unit test"?
I've figured out a way to run some fixture teardown code after all the tests have run (see https://xunit.github.io/docs/shared-context.html#class-fixture), and by using a static variable to record evaluated rules I can check during the teardown whether all the rules have been returned during unit tests. But this approach has the undesirable effect that it causes individual test to report as failed (in teardown) which didn't actually fail.
TLDR: control the test sequence. Since these are integration tests, that's not the big no-no it would be if they were unit tests.
I found that by using a static variable to record the rules that evaluated as true, and ensuring the have-all-rules-evaluated-as-true test last, as per the advice here https://hamidmosalla.com/2018/08/16/xunit-control-the-test-execution-order/, it becomes trivially easy to achieve what I needed.
In case the link dies, here's how to implement this approach
First, create an AlphabeticalOrderer implementing xUnit's ITestCaseOrderer:
public class AlphabeticalOrderer : ITestCaseOrderer
{
public IEnumerable<TTestCase> OrderTestCases<TTestCase>(IEnumerable<TTestCase> testCases)
where TTestCase : ITestCase
{
var result = testCases.ToList();
result.Sort((x, y) => StringComparer.OrdinalIgnoreCase.Compare(x.TestMethod.Method.Name, y.TestMethod.Method.Name));
return result;
}
}
Then use the newly-created attribute on your test class, and create a static variable to record progress:
[TestCaseOrderer("MyCompany.AlphabeticalOrderer", "MyTestsDLL")]
public class RulesTests
{
private static List<Rule> _untestedRules;
public RulesTests()
{
if (_untestedRules == null) // this will run once per [Fact]
{
_untestedRules = <some way of getting all the rules> // this will only run first time around
}
}
Each time a rule triggers, record which rule it was:
private void MarkRuleAsTested(LendingRule testedRule)
{
var rulesToRemove = _untestedRules.Where(r => r.Logic == testedRule.Logic).ToList();
foreach (var ruleToRemove in rulesToRemove)
{
_untestedRules.Remove(ruleToRemove);
}
}
Then the last test to run should check the collection:
[Fact]
public void ZZ_check_all_rules_have_been_triggered_by_other_tests()
{
// This test must run last because it validates that the
// OTHER tests taken together have resulted in each rule
// being evaluated as true at least once.
if (!_untestedRules.Any())
{
return;
}
var separator = "\r\n------------------------\r\n";
var untestedRules = string.Join(separator, _untestedRules.Select(ur => $"Logic: {ur.Logic}"));
throw new SomeRulesUntestedException($"The following rules have not been tested to resolve as True: {separator}{untestedRules}{separator}");
}
This results in a nicely-readable test failure.
You could equally start with an empty static collection of _testedRules and at the end compare that set to the full set of rules.
I have the following code:
private function registerShutdownFunction(): void
{
register_shutdown_function(function () {
$this->dropDatabasesAndUsersIfExist();
});
}
And this code:
private function dropDatabasesAndUsersIfExist(): void
{
// some code for deletion of the databases...
foreach ($connections as $connection) {
$this->assertNotContains($connection, $databases);
}
}
But dropDatabasesAndUsersIfExist is not a "test..." method. And phpunit ignores assertions outside of the test methods.
And seems there are problems may occur, because this shutdown function running directly before the die of the script...
You can use PHPUnit's Assert class outside of test cases if that is really what you want to do:
PHPUnit\Framework\Assert::assertNotContains($connection, $databases);
Edit: After reading your question one more time I'm not really sure if my answer helps you. If I got you right, you are already using the assertion but it did not behave as you'd expect it. My guess is that you want the whole test run to fail if any of the assertions in dropDatabasesAndUsersIfExist was not met.
One solution could be to move the checks you are doing in dropDatabasesAndUsersIfExist to a separate test class that should be executed last. You can achieve this by appending another test suite with the new class right after your test suite(s).
When providing a setting value:
mySetting := {
new Thread() {
override def run() = {
Thread.sleep(1000)
// Need to call a task from here
myTask.value
}
}
"some value"
}
I need to call a task after some time. Of course, in my real scenario, it is something more complex than a simple delay.
I cannot use myTask.value because it would perform the task at the wrong time.
How can I call a task when I wish to?
The solution is to call the implementation of the task.
But SBT, isn't aware that the originating task hasn't ended yet, even though it is scheduled for later the 2nd task.
I have a question about cflow or cflowbelow in AspectJ.
cflow(Pointcut)
Picks out all join points in the control flow of the
join points picked out by the pointcut, including pointcut's join
points themselves.
cflowbelow(Pointcut)
Picks out all join points in the control flow
below the join points picked out by the pointcut.
For both, I could find only the definition and that:
When defining pointcut using cflow or cflowbelow, we need to ensure
that the pointcut does not capture the calls that are made from the
same aspect, otherwise it will invoke recursive method calls and we
will get StackOverflowError.
This happens because cflow() will also
capture the method calls from the aspect itself and try to apply the
advice to it. This can be avoided by using within() construct.
within() takes a type(class or interface) name as argument and
captures all join points the are defined in that type.
But no explanation about how actually cflow() or cflowbelow() lead to an infinite recursion when they are used without within or together with an && expression like:
pointcut aPointcut(): execution(void Test.foo()) && !cflowbelow(execution(void Test.foo()));
Which will match e.g. the first execution of Test.foo() only ignoring any bubbling if inside Test.foo() another call to Test.foo() is made or a call to a foo() method of a class which extends Test is made.
My question is: why cflow actually leads to an infinite recursion when it is used e.g. without within? How does the weaving with cflow happens so that it leads to such a recursion?
A simple example:
Driver application:
package de.scrum_master.app;
public class Application {
public static void sayHelloTo(String name) {
System.out.println("Hello " + name + "!");
}
public static void main(String[] args) {
sayHelloTo("world");
}
}
Aspect:
package de.scrum_master.aspect;
import de.scrum_master.app.Application;
public aspect CflowRecursionDemo {
// Attention, StackOverflowError!
/*
before() : cflow(execution(* Application.sayHelloTo(..))) {
System.out.println(thisJoinPoint);
}
*/
before() : !within(CflowRecursionDemo) && cflow(execution(* Application.sayHelloTo(..))) {
System.out.println(thisJoinPoint);
}
}
The commented-out advice leads to a StackOverflowError while the active one does not.
Console output:
execution(void de.scrum_master.app.Application.sayHelloTo(String))
get(PrintStream java.lang.System.out)
call(java.lang.StringBuilder(String))
call(StringBuilder java.lang.StringBuilder.append(String))
call(StringBuilder java.lang.StringBuilder.append(String))
call(String java.lang.StringBuilder.toString())
call(void java.io.PrintStream.println(String))
Hello world!
Explanation: As you can see there are a number of joinpoints within the control flow of the active advice. Each one of these would again trigger the active advice because each of them again matches the pointcut cflow(execution(* Application.sayHelloTo(..))). The advice is a method like any other, though, it just happens to be inside of an aspect. Anyway, it is in the control flow of its own pointcut which again triggers the advice which again is in its own pointcut's control flow and so forth. Infinite recursion!
I'm trying to use callLater with FlexUnit v0.9:
public function testCallLater():void {
Application.application.callLater( addAsync(function():void {
assertTrue(true);
}, 1000));
}
but when it runs I get this error:
ArgumentError: Error #1063: Argument count mismatch on flexunit.framework::AsyncTestHelper/handleEvent(). Expected 1, got 0.
at Function/http://adobe.com/AS3/2006/builtin::apply()
at mx.core::UIComponent/callLaterDispatcher2()[C:\autobuild\3.2.0\frameworks\projects\framework\src\mx\core\UIComponent.as:8628]
at mx.core::UIComponent/callLaterDispatcher()[C:\autobuild\3.2.0\frameworks\projects\framework\src\mx\core\UIComponent.as:8568]
I'm not sure what the problem is. Is callLater incompatible with FlexUnit?
First, you should really consider migrating to FlexUnit 4.0: http://blogs.digitalprimates.net/codeSlinger/index.cfm/2009/5/3/FlexUnit-4-in-360-seconds
Second, callLater is meant to be used to delay processing until the next frame in visual classes. Your test case class is not a visual class extending UIComponent, therefore you should not try to use callLater.
Third, addAsync is use to test the results of an asynchronous operation. This is typically used in testing the results of a network request, of a file read, of a timer event, etc. That is why normally you see an "event" as a parameter in the addAsync test function (because asynchronous requests use events to process results). In your case, you're not responding to an asynchronous operation with your addAsync call, and therefore you shouldn't be looking for an event in your test function. Remove the event:Event parameter and the error will go away.
However, perhaps you can re-phrase this question to state what you're trying to accomplish? The code sample that you've indicated is not really doing anything useful. If you can be a little more specific we can help you write a better test case.
For help with using addAsync with older versions of FlexUnit, see this tutorial: http://life.neophi.com/danielr/2007/03/asynchronous_testing_with_flex.html
It looks like you are expecting an event, but not getting one. I imagine the following code would work.
public function testCallLater():void {
Application.application.callLater( addAsync(function(/*removed event declaration*/):void {
assertTrue(true);
}, 1000));
}
Just in case someone needs it, this works :
private function testCallLater():void {
Application.application.callLater(doCallLater, [ addAsync(funcUnderTest, 1000) ]);
}
private function doCallLater(testFunc:Function):void {
testFunc(null); // Dummy arg necessary because of addAsync expecting one arg
}
private function funcUnderTest(e:Object = null):void {
assertTrue(true);
}