If I want to run a test with just one data from the data provider how would I do it?
I tried the solution on this thread and did not work
Cannot run single test with data provider in PHPUnit
I found out that just adding #n would just work for numerically indexed data providers.
phpunit --filter ClassName::testName#datasetNumber
Add #name for associatively indexed named data providers.
phpunit --filter ClassName::testName#datasetName
Or you can use regular expressions. See the documentation for more examples.
If data sets have names containing spaces, not only quoting will work, you can also use "\ " (at least in a shell like sh or bash)
phpunit --filter ClassName::testCase#data\ set\ name
If data sets have names containing spaces, this works (note the "):
phpunit --filter "ClassName::testCase#data set name"
Related
For an example, I have created some feature for automation testing using Behave login_account.feature and choose_product.feature. To run on a single feature, I use this command on the terminal behave -f behave_html_formatter:HTMLFormatter -i login_account.feature.
I want to run on multiple feature login_account.feature and choose_product.feature in one command on the terminal. Anyone, can you give me the example command to run multiple feature using Behave in one command on the terminal?
Thank you in advance.
I think there might be two issues:
I'm not sure if that's the way to run with that formatter. See behave-html-formatter docs for config and example. I can't trial myself as I don't want/need this formatter installed. I would suggest you worry about this after you understand how to run multiple features, which is where issue #2 comes in.
-i expects a single argument, which is a regular expression pattern.
Assuming you are in your features folder, just list the two features with space separator after the behave command:
behave login_account.feature choose_product.feature
Better still, you could use tags. This makes way more sense if you want to run more than the two files in your example. Add a tag at the top of each of the feature files you want to execute e.g. #runme.
Then execute only the ones having the preferred tag:
behave -t #runme
I implemented test cases for my application and decided to run it everyday. The problem is the result of the previous test will be overwritten by the latest test result. I need to keep them both so I came up with a solution that include the test date and time in the report name, for example; report-202111181704.html (use time in 24-hour format).
I searched through the internet and did not found any solution yet. Anybody here know the solution? or any alternative solution will be fine.
It depends on where you execute your tests. From command line you can save the date to variable. Then use this variable to change the name of generated outputs. For example
date=$(date '+%Y-%m-%d_%H:%M:%S')
robot --output ${date}output.xml --log ${date}log.html --report ${date}report.html test.robot
I found the solution. Instead of setting .html file name, I create a folder and put the result there.
To do this, add --outputdir in pabot command so it's gonna look like this
pabot --pabotlibport $PABOT_PORT --pabotlib --resourcefile ./DeviceSet.dat --processes $thread --verbose --outputdir ./result/$OUTPUT_DIR $ENV
where
$OUTPUT_DIR=`date + "%Y%m%d-%H%M"`
The output folder gonna be like ./result/20220301-2052
Basically 2 issues:
1. I plan to execute multiple test cases from argument file. The structure would look like that:
SOME_PATH/
-test_cases/
-some_keywords/
-argumentfile.txt
How should i define a suite setup and teardown for all those test cases executed from file (-A file)?
From what i know:
a) I could execute it in file with 1st and last test case, but the order of test cases may change so it is not desired.
b) provide it in init.robot and put it somewhere without test cases only to get the setup and teardown. This is because if I execute:
robot -i SOME_TAG -A argumentfile /path/to/init
and the init is in test_case folder it will execute the test_cases with a specific tag + those in a folder twice.
Is there any better way? Provide it, for example, in argumentfile?
2 How to provide PATH variable in argumentfiles in robotframework?
I know there is possibility to do:
--variable PATH:some/path/to/files
but is it not for test suite env?
How to get that variable to be visible in the file itself: ${PATH}/test_case_1.robot
For your 2nd question, you could create a temporary environment variable that you'd then use. Depending on the OS you're using, the way you'll do this will be different:
Windows:
set TESTS_PATH=some/path/here
robot -t %TESTS_PATH%/test_case_1.robot
Unix:
export TESTS_PATH="some/path/here"
robot -t $TESTS_PATH/test_case_1.robot
PS: you might want to avoid asking multiple, different questions in the same thread
I have a configuration file that I am reading into my robot test cases. This configuration file contains the following variables:
${DATABASE_IP} 127.0.0.1
${ORACLE_SYSTEM_ID} xe
${ORACLE_DATABASE_URL} jdbc:oracle:thin:#${DATABASE_IP}:1521:${ORACLE_SYSTEM_ID}
${ORACLE_DATABASE_USER} cooluser
${ORACLE_DATABASE_PASSWORD} coolpassword
${ORACLE_DATABASE_DRIVER} oracle.jdbc.driver.OracleDriver
One thing I'd like to be able to do is change some of these properties, depending on where the script is executed from. Example: jenkins
A simple way to look at this, is say as follows:
I have a test file called database_test.robot.
If I invoke this file on my local machine, I'd like to pass in an argument to ensure ${DATABASE_IP} equates to 127.0.0.1 . When Jenkins does it, I want that value to point somewhere else.
Something like this already exists with maven, where you can specify a profile at runtime. Ex: mvn verify -Plocal-config ; mvn verify -Pjenkins-config
I have looked through the robot framework documentation, but cannot seem to implement something similar. The only way to swap out properties that I see is to remove the old and replace in the new. Note : I have hundreds of properties that will differ, and several other environments aside form Jenkins and local that would take different values.
Robot gives you at least three ways to solve this: argument files, variable files, and resource files. In each of the cases, you can specify which environment settings to use with a command line argument.
Argument files
Argument files are, as the name implies, files from which robot can read arguments. They are a convenient way to specify a group of command line arguments.
For example, you could create a "environments" folder that contains argument files for each of your environments (production.args, staging.args, local.args) and within the file you would set the values for all of the variables.
For example, you could create a file named local.args with the following contents:
--variable DATABASE_IP:127.0.0.1
--variable ORACLE_SYSTEM_ID:xe
--variable ORACLE_DATABASE_URL:jdbc:oracle:thin:#127.0.0.1:1521:xe
--variable ORACLE_DATABASE_USER:cooluser
--variable ORACLE_DATABASE_PASSWORD:coolpassword
--variable ORACLE_DATABASE_DRIVER:oracle.jdbc.driver.OracleDriver
Then, to run with this configuration you would use the -A or --argumentfile option:
robot --argumentfile environments/local.args ...
The advantage to using argument files is that you can override single values on the command line for times when you need to change just one value:
robot --argumentfile environments/local.args --variable ORACLE_DATABASE_USER:anotheruser
Also, with argument files you can also specify any other command line arguments. For example, if you always want to ignore tests on your CI server that are known to be broken, you could include something like --exclude known-broken (where known-broken is a tag you've applied to one or more tests)
One downside to argument files is that you can't define variables based on the value of previous variables (ie: you can't do --variable FOOBAR=${FOO}bar). I've not found that to be much of a problem.
Variable files
Variable files work in a similar way, but let you define the variables with python. The advantage to variable files is that you can do anything that python lets you do. For example, you could automatically determine the IP of the local database, or selectively turn features on or off based on runtime conditions.
The simplest way to define a variable file is to simply create python variables, which robot will find by importing your file.
For example, the variable file for your variables might look like this:
DATABASE_IP = "127.0.0.1"
ORACLE_SYSTEM_ID = "xe"
ORACLE_DATABASE_URL = " jdbc:oracle:thin:#%s:1521:%s % (DATABASE_IP, ORACLE_SYSTEM_ID)
ORACLE_DATABASE_USER} = "cooluser"
ORACLE_DATABASE_PASSWORD} = "coolpassword"
ORACLE_DATABASE_DRIVER} = "oracle.jdbc.driver.OracleDriver"
Resource Files
Much like the other two solutions, you can have separate resource files for each environment. Since robot allows you to use variables in resource file paths within a suite, you can use a variable to define which resource file to use.
For example, you could import a resource file like this:
# some_tests.robot
*** Settings ***
Resource config/${environment}.robot
You would then create a config file for each environment like you normally would (eg: config/local.robot, config/staging.robot, etc). Then, when you run robot you can tell it which resource file to use:
$ robot --variable environment:local ...
I tried the third option with Resource files but given above command line argument statement:
$ robot --variable environment=local
Didn't work for me. After looking at the robot help file, came to know that variable values should be passed through : and not with =.
So I tried with:
$ robot --variable environment:local
And it worked for me.
The correct way to specify the Resource path for a subdirectory is:
Resource ../config/${environment}.robot
if config is a subdirectory.
I have robot files in a folder (tests) as shown below:
tests
1_robotfile1.robot
2_robotfile2.robot
3_robotfile3.robot
4_robotfile4.robot
5_robotfile5.robot
6_robotfile6.robot
7_robotfile7.robot
8_robotfile8.robot
9_robotfile9.robot
10_robotfile10.robot
11_robotfile11.robot
Now if I execute '/root/users1/power$ pybot root/user1/tests' command, robot files are running in following order:
tests
1_robotfile1.robot
10_robotfile10.robot
11_robotfile11.robot
2_robotfile2.robot
3_robotfile3.robot
4_robotfile4.robot
5_robotfile5.robot
6_robotfile6.robot
7_robotfile7.robot
8_robotfile8.robot
9_robotfile9.robot
I want to force robot_framework to pick robot files in sequential order, like 1,2,3,4,5....
Do we have any option for this?
If you have the option of renaming your files, you just need to make sure that the prefix is sortable. For numbers, that means they should all have the same number of digits.
I recommend renaming your test cases to have three or four digits for the prefix:
001_robotfile1.robot
002_robotfile2.robot
003_robotfile3.robot
004_robotfile4.robot
005_robotfile5.robot
006_robotfile6.robot
007_robotfile7.robot
008_robotfile8.robot
009_robotfile9.robot
010_robotfile10.robot
011_robotfile11.robot
...
With that, they will sort in the order that you expect.
Following #Emna answer, RF docs ( http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#execution-order ) provides some solution.
So what could you do:
rename all the files to have consecutive and computer numbering (001-test.robot instead of 1-test.robot). This may break any internal references to other files (resources), hard to add test in-between,error prone when execution order needs to be changed
you can tag it as Emna
idea from RF docs - write a script to create argument file which will keep ordering in proper way and use it as argument to robot execution. For 1000+ files it should not take longer than few seconds.
try to design tests to not be dependent from execution order, use suite setup instead.
good luck ;)
Tag the tests as foo and bar so you can run each test separately:
pybot -i foo tests
or
pybot -i bar tests
and decide the order you want
pybot -i bar tests || pybot -i foo tests