I'm new with cloudify (4.2) and trying to exercise sheduling workflows. On Cloudify Roadmap i found this feature:
Scheduled Workflow Execution: The ability to schedule a workflow execution at a future time, such as scaling the number of web server VMs at a certain time of the day
But unfortunately i can't find that in the documentation nor a small example on how to do this, what services, policies should i use.
Any hints and suggestions will be greatly appreciated.
Keep in mind that this feature is in development, which is the reason you found it on the Roadmap page, and it will not be found anywhere in the docs either. We don't yet know to which upcoming release this will be added.
If you have any other questions, please head over and ask on the User Group.
Related
I have a questions regarding differences between Apache Airflow and Metaflow(https://docs.metaflow.org/). As far as I understood Apache airflow is just a job scheduler, that runs tasks. Metaflow from Netflix is as a dataflow library, which creates machine learning pipeline(dataflow is available) in forms of DAGs. Basically it means, that Metaflow can be executed on the Apache Airflow?
is my understanding correct?
If yes, is it possible to convert Metaflow DAG into Apache Airflow DAG?
Honestly, I haven't worked with Metaflow and thank you for introducing it to me! There is a nice introduction video you can find on Youtube.
Airflow is a framework for creating scheduled pipelines. A pipeline is a set of tasks, linked between each other that represent an Directed Acyclic Graph. Pipeline can be scheduled, you can tell how often or when it should run, you can tell when it should've ran in the past and what time period it should backfill. You can run the whole Airflow as one single docker container or you can have multi-node cluster, it has bunch of already existing operators to integrate with 3rd party services. I recommend to look into Airflow Architecture and concepts.
Metaflow looks like something similar, but created specifically for data-scientists. I can be wrong here, but looking at the Metaflow Basics it looks like I can the same way create a scheduled pipeline similar to Airflow.
I would look in specific tools you want to integrate with and which one of both integrates better. As mentioned, Airflow has lots of already made connectors and operators, as well as, powerful scheduler with backfill and Jinja template language to design your DB queries for enter link description here.
Hope that is somewhat helpful.
Here is also some nice article with feature comparison.
We are evaluating and looking for a workflow engine which support .NET-Core and I'd really appreciate the community input. I would like to hear on the suggestion based on your guys implementation stories.
My main evaluation criteria, so far, are below;
open source and OEM friendly license
production installations (success stories are a great help)
technical support available
open standards support - BPMN
dynamic creation/assembly of the workflow based on input
embeddable
Currently I am evaluating Elsa, Workflow Core, Argo, and Airflow. Elsa seems like a good candidate as well but never used it.
Do you guys have any successful deployments on Elsa workflow engine?
Full disclosure: I am the project lead of Elsa, but I will try and be as objective as I can.
Elsa does not currently support BPMN, so if this is a hard requirement then Elsa might not be suitable for your project. At least not until it implements BPMN in the future.
As for technical support, there is no official paid support available as of yet, but the community is very friendly & helpful, though still relatively small.
Dynamic creation based on input is possible since you can programmatically define workflows. But you cannot update workflows while it executes (which would be more or less similar to being able to update your C# program statements as your program runs). Not sure if this is what you are looking for or not?
Other than that, Elsa is OEM friendly, runs in production successfully at several companies that I know of and is embeddable.
We have seen flyway framework for database migration. It is pretty wonderful. In our project, we are using GuassDB as database which flyway not support yet. Could you please let me kown how to make it work or will GuassDB be supported in future ? Thank you!
There are a few examples if you go through http://github.com/flyway/flyway/pulls - and if you don't mind waiting a day or so we will be adding more detailed instructions to the Flyway website with the 6.2 release. You will be able to find it here:
https://flywaydb.org/documentation/contribute/contributingDatabaseSupport
We generally check DB-Engines ranking to get a feel for the level of support for each database platform before we commit to supporting it, as well as an ability to run our test suite against it.
We came across choice to work on Scheduler named Azkaban with good UI benefits and dependencies resolution.
Read through the document https://azkaban.github.io/azkaban/docs/latest/ and started using.
Struggle to mark it stable for long run of scheduled jobs.
So, what is the azkaban stability point of view in terms of:
any challenges faces?
critical issues?
Response from community support for Azkaban?
Does Windows Workflow Foundation 4.0+ and more specifically its State Machine workflow have the concept of shared events/transitions across all states within a workflow? Also known as a Hierarchical State Machine.
This was possible in WF 3.5 as proven here, if you scroll to the bottom.
I really can't find anyway to implement this kind of behavior, and I'm looking to upgrade an exiting WF 3.5 system.
I found a similar question here although its got a larger scope, and no answers.
Looking at your example I can see that plenty of classes and interfaces used in that article are obsolete and you should not use them in your future projects:
EventDrivenActivity Class
HandleExternalEventActivity
WebServiceInputActivity
....
Below are the WF4 alternatives for your problem:
Arguments
Bookmarks : allows you to do event-driven communication to an activity within a workflow from outside by the help of WF runtime as the channel of communication.
Resuming a pending workflow is now managed using a Bookmark object