Is it possible to retrieve multiple values from XCOM, pushed by a single task but with different keys?
I think I've seen examples to this:
# pulls one value
pulled_value = ti.xcom_pull(key=None, task_ids='my_task_id')
and to this:
# pulls multiple values but from multiple tasks
pulled_value_1, pulled_value_2 = ti.xcom_pull(key=None, task_ids=['my_task_id_1', 'my_task_id_2'])
What I need is would possibly look like this:
# pulls multiple values but from a single
pulled_value_1, pulled_value_2 = ti.xcom_pull(key=['my_key_1', 'my_key_2'], task_ids='my_task_id')
I can't find this in the documentation.
Is this even possible?
If yes, it makes a single database query in the background, or just repeats a single query multiple times?
If not, how could I get similar behavior?
Answering your questions:
There is no such feature.
xcom_pull accepts task_ids: Optional[Union[str, Iterable[str]]] but with the same key. You can open a PR to Airflow for adding the functionality you seek.
As for number of queries: I assume that by "repeats a single query" you are asking if it execute a query per task_id. The answer is No. Airflow did this optimization in PR.
In the source code you can see that xcom_pull() uses Xcom.get_many() and get_many creates a filter using IN which allows to compare against multiple values. You can see the relevant code lines here.
Related
I have about 3 collections and i want to write into an excel and send them in mail separately, I tried to use multi choice to finalize one after the other but it doesn't work
any other idea how to do this?
I will put screen shots below for more illustration
It's likely not functioning the way you intended because each of the three Data Items you're attempting to compare to a blank string ("") aren't Text-typed - rather, they're Collections. Collections themselves can't be compared to strings (see: apples and oranges), but the data contained within them can.
What you're likely attempting to do is to compare the value within the current row of each of those collections - it's not clear what the field name is from your screenshot (the field name itself is cut off on the right edge) but your comparisons should look something like the following:
[No Amount.Politic]<>""
[Pending difference.Field]<>""
[Ready to print.Field]<>""
I want to synchronize data for actual work from a web-based application of my company with MS Project. I am currently developing an Add-In with JavaScript in order to achieve this:
The red circle in my screenshot shows the data that I want to set programmatically. However, I have no idea how to achieve this.
I understand that I can get Task GUIDs and then set task fields using the task GUID and the field ID. This way I can save the cumulative actual work, but not per day like in my screenshot.
The API Docs on the MS Office Website are rather hard to read and navigate. Any help would be apprechiated!
Let's first separate the language from the operation.
Operationally, based on your circle, you want to set work for a task to happen on individual days? This is done using timeScaleData, see https://learn.microsoft.com/en-us/previous-versions/office/developer/office-2003/aa206255(v=office.11) . When I did something similar (in VBA), I had to (1) get an array of time scale values, then (2) walk/iterate through that array and set work to those days:
set timeScaleValsArry = myTask.Assignments(1).TimeScaleData(startDay, endDay, pjAssignmentTimeScaledWork, daily)
for a = 1 to timeScaleValsArry.Count
timeScaleValsArry[a].value = hoursToWorkThatDay
next
Breaking down the elements above:
myTask is the task (of type task) I want to manipulate.
Assignments is an array representing each resource assigned to the task; for my purposes, I only ever had 1 resource assigned, hence the index of (1).
TimeScaleData is the function that returns the the array starting on the day startDay (whatever you want that to be), endDay, pjAssignmentTimeScaledWork which tells this function what data we want to work with (being work, but there are alternates ), and daily which is the frequency you want to work with (for instance you can go down to minutes, or up to years).
Then the returned array timeScaleValsArry is walked, and inside the loop the daily assignment for each value is manipulated. You'd need to customize this part to meet your needs; alternatively, you don't even need to loop if you always had three days: just hard code the array indices.
As far as language, clearly this is do-able in VBA. Doing this in C# as a VSTO addin has very similar syntax. I'd presume for JavaScript (what are you using, ScriptLab?) would also have similar syntax.
I want to know the query for kibana, i tried terms and agg, but didnt get right output, so need to filter the data based on distinct query in kibana.
I want to apply query in following input data in elasticsearch
Rows and columns
CELLID|MCC|MNC|
1222|405|861|
1222|405|861|
1222|405|122|
1233|406|861|
1233|406|861|
1224|407|777|
1224|407|777|
need to apply query such a way, it will remove the same CELL ID with different MNC, so expecting output like this
CELLID|MCC|MNC|
1233|406|861|
1233|406|861|
1224|407|777|
1224|407|777|
As you know, it's impossible to have such row vs column infrastructure within Kibana graphs as of now. Cos this feature is yet be made to the new versions as an enhancement.
But then if you're simply trying to print out the count|sum or let it be whatever the aggregation you need, you can have a Data Table visualization with a metric of count and then within your buckets you could define multiple terms-aggregation. In your case, you should have CELLID|MCC|MNC being split by terms-aggregation which should do the job for you. Hope this helps!
I have a table (key=username, value=male or female) and an index on the values.
After I add an item to the table, I want to update the counts of males and females. However, after a successful write, as the index is a Global Secondary Index, the count query is not consistent.
Is there a way (dynamo db Streams, Lambda, ...) to monitor when the index is up to date?
Note that Im not looking for a solution that involves something else (keep count of increments in redis or ...), what I describe here is a simplified problem to especially ask a question about how can I monitor an index in dynamo.
Thanks!
I am not sure if there is any mechanism currently provided to check this but, you can easily solve this problem by adding a single line of code to your query.
ConsistentRead = True
DynamoDB has a parameter when set as true will make sure that you get latest updated value.
Now, when you add/update the item and then query the data add ConsistentRead option in it, this will ensure that you will have latest count value.
Here is the reference link.
If you are able to accomplish using other technique then please do share it.
Hope that helps.
Is it possible to dynamically create control-M jobs.
Here's what I want to do:
I want to create two jobs. First one I call a discovery job, the second one I call a template job.
The discovery job runs against some database and comes back with an array of parameters. I then want to start the template job for each element in the returned array passing in that element as a parameter. So if the discovery job returned [a1,a2,a3] I want to start the template job 3 times, first one with parameter a1, second with parameter a2 and third one with parameter a3.
Only when each of the template jobs finish successfully should the discovery job show as completed successfully. If one of the template job instances fails I should be able to manually retry that one instance and when it succeeds the Discovery job should become successful.
Is this possible ? And if so, how should this be done ?
Between the various components of Control-M this is possible.
The originating job will have an On/Do tab - this can perform subsequent actions based on the output of the first job. This can be set to work in various ways but it basically works on the principle of "do x if y happens". The 'y' can be job status (ok or not) exit code (0 or not) or text string in standard output (e.g. "system wants you to run 3 more jobs"). The 'x' can be a whole list of things too - demand in a job, add a specific condition, set variables.
You should check out the Auto Edit variables (I think they've changed the name of these in the latest versions) but these are your user defined variables (use the ctmvar utility to define/alter these). The variables can be defined for a specific job only or across your whole system.
If you don't get the degree of control you want then the next step would be to use the ctmcreate utility - this allows full on-the-fly job definition.
You can do it and the way I found that worked was to loop through a create script which then plugs in your variable name from your look-up. You can then do the same for the job number by using a counter to generate a job name such as adhoc0001, adhoc0002, etc. What I have done is to create n number of adhoc jobs as required by the query, order them into a new group and then once the group is complete send the downstream conditions on. If one fails then you can re-run it as normal. I use ctmcreate -input_file . Which works a treat.