I would like to move all my output files to a custom location, to a Run directory created based on Date time during Run time. The output folder by datetime is created in the TestSetup
I have function "Process_Output_files" which will move the files to the Run folder(Run1,Run2,Run3 Folders).
I have tried using the argument-d and used the function "Process_Output_files" as suite tear down to move the output files to the respective Run directory.
But I get the following error "The process cannot access the file because it is being used by another process". I know this is because the Robot Framework (Ride) is currently using this.
If I dont use the -d argument, the output files are getting saved in temp folders.
c:\users\<user>\appdata\local\temp\RIDEfmbr9x.d\output.xml
c:\users\<user>\appdata\local\temp\RIDEfmbr9x.d\log.html
c:\users\<user>\appdata\local\temp\RIDEfmbr9x.d\report.html
My question is, Is there a way to get move the files to custom location during run time with in Robot Framework.
You can use the following syntax in RIDE (Arguments:) to create the output in newfolders dynamically
--outputdir C:/AutomationLogs/%date:~-4,4%%date:~-10,2%%date:~-7,2% --timestampoutputs
The above syntax gives you the output in below folder:
Output: C:\AutomationLogs\20151125\output-20151125-155017.xml
Log: C:\AutomationLogs\20151125\log-20151125-155017.html
Report: C:\AutomationLogs\20151125\report-20151125-155017.html
Hope this helps :)
I understand the end result you want is to have your output files in their custom folders. If this is your desire, it can be accomplished at runtime and you won't have to move them as part of your post processing. This will not work in RIDE, unfortunately, since the folder structure is created dynamically. I have two options for you.
Option 1: Use a script to kick off your tests
RIDE is awesome, but in my humble opinion, one shouldn't be using it to run ones tests, only to build and debug ones tests. Scripts are far more powerful and flexible.
Assuming you have a test, test2.txt, you wish to run, the script you use to do this could be something like:
from time import gmtime, strftime
import os
#strftime returns string representations of a date-time tuple.
#gmtime returns the date-time tuple representing greenwich mean time
dts=strftime("%Y.%m.%d.%H.%M.%S", gmtime())
cmd="pybot -d Run%s test2"%(dts,)
os.system(cmd)
As an aside, if you do intend to do post processing of your files using rebot, be aware you may not need to create intermediate log and report files. The output.xml files contain everything you need, so if you don't want to create superfluous files, use --log NONE --report NONE
Option 2: Use a listener to do post processing
A listener is a program you write that responds to events (x_start, x_end, etc). The close() event is akin to the teardown function and is the last thing called. So, assuming you have a function moveFiles() you simply need to create a listener class (myListener), define the close() method to call your moveFiles() function, and alert your test that it should report to a listener with the argument --listener myListener.
This option should be compatible with RIDE though I admit I have never tried to use listeners with the IDE.
At least you can write a custom run script that handles the moving of files after the test case execution. In this case the files are no longer used by pybot.
Related
I'm using jupyter and pandas read_sql, this works fine but looks ugly.
for instance I have a query:
SELECT *
FROM table_a AS a
LIMIT 10;
I could show it nicely in a markdown cell as so:
``` mysql
SELECT *
FROM table_a AS a
LIMIT 10;
```
and I could execute it in a code cell as so:
pd.read_sql('SELECT * FROM table_a AS a LIMIT 10;', conn)
this involves copy/paste and displaying the query twice (not too good if I want to simply export my notebook to a pdf report)
is there a way to avoid the duplication by reading the markdown text into a string python variable, or any other way?
The cellmagic answer cited by #Micah Kornfield in the question comments may be a good fit for many situations. In the question however it is said that it is desirable to avoid duplicates. Let's imagine that the SQL is huge and we don't want see the same query more than once.
Unfortunatelly right now in 2021 there's no easy solution for this. In a jupyter notebook there are two worlds, the backend which is the kernel and in our case runs python, and the frontend which runs javascript. Only javascript sees the markdown cells. It is possible to make the backend and frontend communicate with each other, those methods are usually a little hacky, but anyway we will rely on some of them.
I have written a script that does our job in two different ways, which will probably bring similar results. I will call those methods the file read method and the javascript method.
First, please save the following file markdown.py in the same folder as the ipython (we are using a separate file because you specified that your notebook willl eventually go to a report and it is undesirable to have this script together with the notebook):
from IPython.display import Javascript
from urllib.parse import unquote
from json import loads as jsonloads
def markread(cellnumber,notebookname=None,callbackvar=None):
try:
if type(cellnumber) is int:# maybe check if (varname in globals()):
if callbackvar is not None and type(callbackvar) is str:
return Javascript("const mdtjs = Jupyter.notebook.get_cells().filter(c=>c.cell_type==\"markdown\")["+str(cellnumber)+"].get_text(); IPython.notebook.kernel.execute(\"mdtp = unquote('\"+encodeURI(mdtjs)+\"');mdtp=mdtp[mdtp.find('\\\\n',mdtp.find('```'))+1:min(mdtp.rfind('\\\\n'),mdtp.rfind('```'))].strip();"+callbackvar+"=mdtp;del mdtp\");")
if notebookname is not None and type (notebookname) is str:
if not notebookname.endswith('.ipynb'):
notebookname += '.ipynb'
with open(notebookname) as f:
j = jsonloads(f.read())
mdts = [''.join(c['source'][1:]).strip().strip('`').strip() for c in j['cells'] if c['cell_type']=='markdown']
return mdts[cellnumber]
except:
return None
return None
Now back to the notebook, to load the script, you have to import it:
from markdown import markread, unquote
The unquote is needed to use the javascript method, otherwhise you can skip it.
1. File read method:
Usage:
marktext = markread(2, notebookname='mynotebookname')
Here marktext will get the value from the third markdown cell in the mynotebookname (third because we live in a zero-indexed world, so 2 means third; if you skip '.ipynb' extension in the notebookname as in this case it will be automatically appended). Important - this method reads the notebook file writen on disk and not the hot state of things. If you changed anything since last save, things may go wrong.
2. Javascript method:
Usage:
markread(1, callbackvar='marktext')
Here we write the value of our second markdown cell to a variable called marktext. Javascript method is trickier - it is async, so we have to send the name of the variable that we want to write to (must be a string representing its name, not the variable itself). Is is important to know also that markread must be the last command in the cell due to a limitation in javascript invoking.
How it works
Internally, the file read method just reads the notebook file which is json, picks the value from 'cells' and filters out the ones which are markdown.
The javascript method however is more complex. It invokes JS because JS has access to the cells including markdown, so JS reads cells values (from the Jupyter.notebook.get_cells), filters the markdown ones, invoke python back and send back those markdown cells - url enconded. Those encoded cells are decoded back and assigned to the callbackvar. In both methods I made some assumptions that may not be correct about trimming the start and the end of the cell value (the ``` and whitespaces).
There are ways to improve the code, for example making it auto detect the notebook name for the file read method, but it involves even more hacks, relying again on javascript to get the notebook name or making an call to the api on port 8888, but having to deal with session password. I believe the most important is covered already by our script. If one method does not work, you will probably still have the other.
From the documentation, config-default.xml must be presented in the workflow workspace.
- /workflow.xml
- /config-default.xml
|
- /lib/ (*.jar;*.so)
The problem
I've created a custom Oozie action and try to add default values for retry-max and retry-interval to all the custom actions.
So my workflow.xml will look like this:
<workflow-app xmlns="uri:oozie:workflow:0.3" name="wf-name">
<action name="custom-action" retry-max="${default_retry_max}" retry-interval="${default_retry_interval}">
</action>
config-default.xml file contains the values of default_retry_max and default_retry_interval.
What I've tried
Putting config-default.xml to every workflow workspace. This works, but the problem is there will be this file everywhere.
Setting oozie.service.LiteWorkflowStoreService.user.retry.max and oozie.service.LiteWorkflowStoreService.user.retry.inteval also works, but it would affect all action types.
I've also looked at Global Configurations, but it doesn't solve this problem.
I think there should be a way to put config-default.xml to oozie.libpath and only those workflows that use this libpath will be affected.
AFAIK, there is unfortunately no clean way to do it.
You might be interested in this recently created feature request: https://issues.apache.org/jira/browse/OOZIE-3179
The only thing that worked for me was to begin the workflow with a shell step that uses a script stored in hdfs. This script holds the centralized configuration. The script would look like this:
#!/bin/sh
echo "oozie.use.system.libpath=true"
echo "hbase_zookeeper_quorum=localhost"
.. etc other system or custom variables ..
Yes, the script simply prints the variables to the stdout.
Let's say the shell step action is called "global_config". All following steps are able to get the variables using following syntax:
${wf:actionData('global_config')['hbase_zookeeper_quorum']}
HTH...
When I run test cases from RIDE the reports are saved in the below path.
C:\Windows\Temp\RIDExf4xla.d
I want save reports in specific path. Can I do this from RIDE? Is there any setting to change the reports location?
Can anyone please suggest the way to do it.
Thanks
Look at the --outputdir command within the Robot Framework Documentation:
Here is what I use:
--outputdir C:/Robot/AutomationLogs/etc/etc --timestampoutputs
You use this one liner on the "Arguments" Field, right on the top of RIDE within the run tab.
From Wamans comment you can add formats to the end of the argument, to also change the dir name dynamically. See the 2nd answer within that SO question. This should be enough for you to get what you're asking for.
There is no way to set this within a UI.
Just set it by pasting that argument option within the "Arguments" Field at the top.
use below code in command line
C:\Tests\> robot -d C:\Test_results Test.robot
I'm trying to write my first robot test; I'd like to use ride as advertized in http://developer.plone.org/reference_manuals/external/plone.app.robotframework/happy.html#install-robot-tools
I added
initialization =
import os
os.environ['PATH'] = os.environ['PATH'] + os.pathsep + '${buildout:directory}/bin'
to my [robot] section to make it possible to run the tests clicking "Start" in ride.
It works, but the second time I run the tests I still see the content created by the first test run.
How do I tell robot-server to go back to a just-initialized state?
Easily (and you should throw me into pool for not documenting this yet in plone.app.robotframework's documentation – I thought that RIDE is too difficult to get running until it works on wxPython 2.9).
In RIDE
select Run-tab
change Execution Profile to custom script
click browse to select for bin/robot from your buildout as the Script to run tests
Click Start.
Technically bin/robot is a shortcut for bin/pybot --listener plone.app.robotframework.RobotListener (I keep repeating bin/, because it's important that plone.app.robotframework is available in sys.path). Robot Framework Listener -interface is specified in Robot Framework User Guide.
Our listener calls bin/robot-server (using XML-RPC) before every test to testSetUp-methods for the current test layer and after every test testTearDown-methods. This resets the fixture and isolates functional tests.
I am trying to write a file watcher job in autosys that would watch out for a particular file. The file name format would be filename_ddmmyyyy.
The requirement is that the file comes at 7.15am everyday and the file watcher job starts running at 6.50am and the runs till 8am. If the file is received by then, job is successful else an alert is raised.
Now what I am trying to do is to watch out the file filename_ddmmyyyy for a particular day. e.g. if today is 22nd Feb 2013, the file name will be filename_22022013 and this is the file that I am looking for. If I use wildcards like filename_*, it would look for all possible files which I don't want.
I am not sure how to do this in Windows.
Any help would be much appreciated.
Let me know in case of questions.
You will need to use the profile job attribute to initalize variables when the job starts. One of these variables will need to be the date pattern you are looking for (you'll need another process that outputs that dynamically). Then once you set it to a variable in your profile script, you can refer to that variable name from within the watch_file attribute.
Create global variable as variable with date and us that variable :
example:filename_$${GV_DATE}
GV_DATE: ddmmyyyy
Pretty late to answer, but here is an answer without using global variable. You can use formatted system date variable in the file name.
File_to_watch: filename_%date:~10,4%%date:~4,2%%date:~7,2%