Is it possible to dynamically create control-M jobs.
Here's what I want to do:
I want to create two jobs. First one I call a discovery job, the second one I call a template job.
The discovery job runs against some database and comes back with an array of parameters. I then want to start the template job for each element in the returned array passing in that element as a parameter. So if the discovery job returned [a1,a2,a3] I want to start the template job 3 times, first one with parameter a1, second with parameter a2 and third one with parameter a3.
Only when each of the template jobs finish successfully should the discovery job show as completed successfully. If one of the template job instances fails I should be able to manually retry that one instance and when it succeeds the Discovery job should become successful.
Is this possible ? And if so, how should this be done ?
Between the various components of Control-M this is possible.
The originating job will have an On/Do tab - this can perform subsequent actions based on the output of the first job. This can be set to work in various ways but it basically works on the principle of "do x if y happens". The 'y' can be job status (ok or not) exit code (0 or not) or text string in standard output (e.g. "system wants you to run 3 more jobs"). The 'x' can be a whole list of things too - demand in a job, add a specific condition, set variables.
You should check out the Auto Edit variables (I think they've changed the name of these in the latest versions) but these are your user defined variables (use the ctmvar utility to define/alter these). The variables can be defined for a specific job only or across your whole system.
If you don't get the degree of control you want then the next step would be to use the ctmcreate utility - this allows full on-the-fly job definition.
You can do it and the way I found that worked was to loop through a create script which then plugs in your variable name from your look-up. You can then do the same for the job number by using a counter to generate a job name such as adhoc0001, adhoc0002, etc. What I have done is to create n number of adhoc jobs as required by the query, order them into a new group and then once the group is complete send the downstream conditions on. If one fails then you can re-run it as normal. I use ctmcreate -input_file . Which works a treat.
Related
Is it possible to retrieve multiple values from XCOM, pushed by a single task but with different keys?
I think I've seen examples to this:
# pulls one value
pulled_value = ti.xcom_pull(key=None, task_ids='my_task_id')
and to this:
# pulls multiple values but from multiple tasks
pulled_value_1, pulled_value_2 = ti.xcom_pull(key=None, task_ids=['my_task_id_1', 'my_task_id_2'])
What I need is would possibly look like this:
# pulls multiple values but from a single
pulled_value_1, pulled_value_2 = ti.xcom_pull(key=['my_key_1', 'my_key_2'], task_ids='my_task_id')
I can't find this in the documentation.
Is this even possible?
If yes, it makes a single database query in the background, or just repeats a single query multiple times?
If not, how could I get similar behavior?
Answering your questions:
There is no such feature.
xcom_pull accepts task_ids: Optional[Union[str, Iterable[str]]] but with the same key. You can open a PR to Airflow for adding the functionality you seek.
As for number of queries: I assume that by "repeats a single query" you are asking if it execute a query per task_id. The answer is No. Airflow did this optimization in PR.
In the source code you can see that xcom_pull() uses Xcom.get_many() and get_many creates a filter using IN which allows to compare against multiple values. You can see the relevant code lines here.
We have a control-m jobs out condition to trigger the next successor jobs.
But, we could see the some numbers are getting added at that. What would be the check in the job settings need to add to get rid of number at the end.
Apart from ordering the control-m folder with the check "Order adependent Flow"
This can happen in 2 different scenarios;
When manually ordering a set of jobs and you specify "Order as Independent Flow" then Control-M adds these suffixes to prevent interference between the manually ordered jobs and pre-existing jobs.
In the Planning Domain/Links Setting - there is a setting called Create unique names for conditions - this will add a random number the the end of a condition where that condition already exists. However, if this option is disabled, and a condition with the same name is created, a single condition is linked to multiple destinations.
If you have access to the BMC documentation you can see this info here -
https://documents.bmc.com/supportu/9.0.18/help/Main_help/en-US/index.htm#11880.htm
I would like to count the number of agents that exits the Sink and save that number continuously to my variable "Loads" while i run the simulation.
!https://imgur.com/rAUQ52n
Anylogic suggets the following.. but i can't seem to get it right.
long count() - returns the number of agents exited via this Sink block.
hope some of you can help me out
It should be easy... you don't even need a variable since sink.count() has the information you need always.
But if you insist in creating one, You can also create a variable called counter of type int and on the sink action just use counter++;
I want to synchronize data for actual work from a web-based application of my company with MS Project. I am currently developing an Add-In with JavaScript in order to achieve this:
The red circle in my screenshot shows the data that I want to set programmatically. However, I have no idea how to achieve this.
I understand that I can get Task GUIDs and then set task fields using the task GUID and the field ID. This way I can save the cumulative actual work, but not per day like in my screenshot.
The API Docs on the MS Office Website are rather hard to read and navigate. Any help would be apprechiated!
Let's first separate the language from the operation.
Operationally, based on your circle, you want to set work for a task to happen on individual days? This is done using timeScaleData, see https://learn.microsoft.com/en-us/previous-versions/office/developer/office-2003/aa206255(v=office.11) . When I did something similar (in VBA), I had to (1) get an array of time scale values, then (2) walk/iterate through that array and set work to those days:
set timeScaleValsArry = myTask.Assignments(1).TimeScaleData(startDay, endDay, pjAssignmentTimeScaledWork, daily)
for a = 1 to timeScaleValsArry.Count
timeScaleValsArry[a].value = hoursToWorkThatDay
next
Breaking down the elements above:
myTask is the task (of type task) I want to manipulate.
Assignments is an array representing each resource assigned to the task; for my purposes, I only ever had 1 resource assigned, hence the index of (1).
TimeScaleData is the function that returns the the array starting on the day startDay (whatever you want that to be), endDay, pjAssignmentTimeScaledWork which tells this function what data we want to work with (being work, but there are alternates ), and daily which is the frequency you want to work with (for instance you can go down to minutes, or up to years).
Then the returned array timeScaleValsArry is walked, and inside the loop the daily assignment for each value is manipulated. You'd need to customize this part to meet your needs; alternatively, you don't even need to loop if you always had three days: just hard code the array indices.
As far as language, clearly this is do-able in VBA. Doing this in C# as a VSTO addin has very similar syntax. I'd presume for JavaScript (what are you using, ScriptLab?) would also have similar syntax.
Suppose we have some 400 jobs in different boxes then I want to put on hold the daily running jobs at 9-10 pm pm only?
Do you use WCC or command line?
In WCC you can just use a comma-separated list of jobs to see only the jobs you want. You can filter by status and select the jobs you want to take action on, then select 'change status' to do a sendevent but check off the 'future' box. Set it up so you send an 'on-hold' event at 9pm and again for an 'off-hold' at 10pm.
If you use command line you'll want to do something like below. Do all of your boxes have some naming conventions in common? If so you can run the command only once using the string that returns your boxes. In the AutoSys instance I work in we use a prefix structure...
To get the list of running jobs:
autorep -J prefix% | [find for windows or egrep for unix] " RU "
... Where you need the spaces between the double quotes and the two-letter status otherwise it would return lines where the item name contains those two characters.
To do a future sendevent use the usual sendevent syntax and just append the switches to indicate the time you want the action taken.
Will this accomplish what you're looking to do? If not please let us know if you're using windows or Unix as well as any additional information that can help us understand the specifics of your scenario.