When I run PHPUnit 7 with the --order-by=defects flag should I expect my tests to be sorted by:
all-time defectiveness (e.g., I've had a lot of failures in module Foo, so even if I'm working in Bar today it'll run Foo first)
recent defectiveness (e.g. I just wrote a red test in new module Baz, so it'll run Baz first, even before the legendarily failure-prone Foo)
Related
I am trying to learn robot & facing two confusions(m a total noob) .Pretty sure I am making a rookie mistake & would appreciate some help .
1)Every time I execute the tests using the "pabot --testlevelsplit Tests/demo.robot" command, the log.html doesn't show the failed test case (in this case: Fill the login form and select "Teacher" from the User option) even though the terminal output shows it as failed.
The same test case however passes when I run them individually and log.html shows 3 passed test cases.
code[Log showing only the 2 passed cases
terminal showing only 2 passed test cases
all the test cases passed and showing up in logs when executed using the ".robot /Test/demo.robot" command
I ran
mpiexec -n $nprocs julia --project myfile.jl
on a cluster, where myfile.jl has the following form
using Distributed; using Dates; using JLD2; using LaTeXStrings
#everywhere begin
using SharedArrays; using QuantumOptics; using LinearAlgebra; using Plots; using Statistics; using DifferentialEquations; using StaticArrays
#Defining some other functions and SharedArrays to be used later e.g.
MySharedArray=SharedArray{SVector{Nt,Float64}}(Np,Np)
end
#sync #distributed for pp in 1:Np^2
for jj in 1:Nj
#do some stuff with local variables
for tt in 1:Nt
#do some stuff with local variables
end
end
MySharedArray[pp]=... #using linear indexing
println("$pp finished")
end
timestr=Dates.format(Dates.now(), "yyyy-mm-dd-HH:MM:SS")
filename="MyName"*timestr
#save filename*".jld2"
#later on, some other small stuff like making and saving a figure. (This does give an error "no method matching heatmap_edges(::Surface{Array{Float64,2}}, ::Symbol)" but I think that this is a technical thing about Plots so not very related to the bigger issue here)
However, when looking at the output, there are a few issues that make me conclude that something is wrong
The "$pp finished" output is repeated many times for each value of pp. It seems that this amount is actually equal to 32=$nprocs
Despite the code not being finished, "MyName" files are generated. It should be one, but I get a dozen of them with different timestr component
EDIT: two more things that I can add
the output of the different "MyName" files is not identical, but this is expected since random numbers are used in the inner loops. There are 28 of them, a number that I don't easily recognize except that its again close to the 32 $nprocs
earlier, I wrote that the walltime was exceeded, but this turns out not to be true. The .o file ends with "BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES ... EXIT CODE :9", pretty shortly after the last output file.
$nprocs is obtained in the pbs script through
#PBS -l select=1:ncpus=32:mpiprocs=32
nprocs= `cat $PBS_NODEFILE|wc -l`
As pointed out by adamslc on the Julia discourse, the proper way to use Julia on a cluster is to either
Start a session with one core from the job script, add more with addprocs() in the Julia script itself
Use more specialized Julia packages
https://discourse.julialang.org/t/julia-distributed-redundant-iterations-appearing/57682/3
I have robot files in a folder (tests) as shown below:
tests
1_robotfile1.robot
2_robotfile2.robot
3_robotfile3.robot
4_robotfile4.robot
5_robotfile5.robot
6_robotfile6.robot
7_robotfile7.robot
8_robotfile8.robot
9_robotfile9.robot
10_robotfile10.robot
11_robotfile11.robot
Now if I execute '/root/users1/power$ pybot root/user1/tests' command, robot files are running in following order:
tests
1_robotfile1.robot
10_robotfile10.robot
11_robotfile11.robot
2_robotfile2.robot
3_robotfile3.robot
4_robotfile4.robot
5_robotfile5.robot
6_robotfile6.robot
7_robotfile7.robot
8_robotfile8.robot
9_robotfile9.robot
I want to force robot_framework to pick robot files in sequential order, like 1,2,3,4,5....
Do we have any option for this?
If you have the option of renaming your files, you just need to make sure that the prefix is sortable. For numbers, that means they should all have the same number of digits.
I recommend renaming your test cases to have three or four digits for the prefix:
001_robotfile1.robot
002_robotfile2.robot
003_robotfile3.robot
004_robotfile4.robot
005_robotfile5.robot
006_robotfile6.robot
007_robotfile7.robot
008_robotfile8.robot
009_robotfile9.robot
010_robotfile10.robot
011_robotfile11.robot
...
With that, they will sort in the order that you expect.
Following #Emna answer, RF docs ( http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#execution-order ) provides some solution.
So what could you do:
rename all the files to have consecutive and computer numbering (001-test.robot instead of 1-test.robot). This may break any internal references to other files (resources), hard to add test in-between,error prone when execution order needs to be changed
you can tag it as Emna
idea from RF docs - write a script to create argument file which will keep ordering in proper way and use it as argument to robot execution. For 1000+ files it should not take longer than few seconds.
try to design tests to not be dependent from execution order, use suite setup instead.
good luck ;)
Tag the tests as foo and bar so you can run each test separately:
pybot -i foo tests
or
pybot -i bar tests
and decide the order you want
pybot -i bar tests || pybot -i foo tests
I have looked at documentation of GNU Makefile Conditional Statements, but I couldn't find any information on how to build both the target and its dependencies conditionally.
EDIT: When I say conditionally, I mean dependent on values of variable set in command line while calling make. I apologize for not making it clear earlier.
To add to the above edit. If some variables set in the commandline take a particular value, even if the targets and their dependencies need to be updated, I want to refrain from doing this.
GNU Makefile Conditionals
Thanks!
What about something like:
ALL_TARGETS = foo bar
ifndef SOME_VARIABLE
ALL_TARGETS += biz baz
endif
all: $(ALL_TARGETS)
Now if you run make it will build foo, bar, biz, and baz, but if you run make SOME_VARIABLE=true it will only build foo and bar.
With the very generic, specifics-free question that's about all we can suggest.
I'm using vanilla R on Linux, and when R is computing something and I press keyboard buttons, they get printed to the terminal.
For example, if I type "foo" while R is sleeping, the following happens:
Sys.sleep(1) #Typing "foo" now before the sleep ends...
foo>foo #"foo" is both registered and shown before the sleep ends
This of course may be wanted in some cases, but there are cases in which I'd like the terminal to be as clean as possible. In this case both the following options would be good:
Ignore any keypress that is registered before the completion of the previous command;
Do not ignore keypresses, but simply do not show them before the completion of the previous command.
To clarify, this is what should happen in those two cases if you write "foo" while R is sleeping:
1:
Sys.sleep(1) #Typing "foo" now before the sleep ends...
> #Nothing is registered
2:
Sys.sleep(1) #Typing "foo" now before the sleep ends...
>foo #"foo" is registered but not shown before the sleep ends
Is there any way I can accomplish one of the two?