Does apache airflow has rule editor, where we can define multiple rules like "(x>y) && (z==a)" etc.. and later we can integrate these rules in the workflow steps similar to Drools workbench editor?
No, depending on what you need to implement you can use variables to store some data. You can mainly store strings or a json structure.
Using those you can define some kind of structure that will be read inside the DAG. You can even create different tasks dynamically from the value of a Variable.
Related
I currently have a Kotlin-Exposed project that I would like to add Flyway to. The problem I am having is that most documentation and answers online indicate that the best way to add Flwyay to an existing schema is to just have the first script be a data definition script. This usually would work, but since I'm dynamically generating my SQL with an ORM, this doesn't really make sense. Are there ways around this?
I really just want to use Flyway to add/delete persistent data that I will always need in certain tables. I don't want to insert it at the ORM level because if the application is run multiple times, then it can insert the data each time it's run (as opposed to Flyway where it will just migrate the database to the newest constructed state).
I think another way to word this question is: "Can I use Flyway for static data only, and not schema?"
Yes, you can.
Some info:
You are not required to have a first script containing the data definition / "baseline" of the schema. You can simply skip that.
When running Flyway against a non-empty database for the first time, you will still need to run the baseline command. In your case this will simply indicate to Flyway that it can assume the baseline schema is present and that it's safe to run migrations. (Your baseline schema was deployed by the ORM instead of a baseline script -- that's totally fine, Flyway won't check/doesn't care.)
You could choose to write your scripts that insert static data in a way that they are idempotent / use a guard clause so that they don't insert the data twice. That way it would be safe to use at the ORM level if you choose.
I can use the powershell command: New-AzureRmResourceGroupDeployment, with a relevant template file to create an instance of all the items within the template. In fact, this is how I initially created some of the components in the current resource group I am working on. Since creating my Azure components I have tweaked things quite a lot, and I would like to create a template based on the setup in my resource group as it stands now, this would allow me to run the above command and to recreate the components if I ever need to.
Can anyone tell me if there is a way to create a template based on an existing resource group?
Looks like there are some MS docs detailing the process here:
https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-export-template
In Flyway is there any way to mark certain scripts for inclusion or exclusion? What I am looking for is something similar to Liquibase's 'contexts' feature, or Dbmaintains 'qualifiers'. My primary use case is the test one that is outlined on the Liquibase site, a 'test' context where only scripts that are related to test data will run.
Yes. Put those test scripts in a second folder, and selectively configure flyway.locations to include it.
What would be a proper way to handle global "settings" in my sailsjs application? The user will want to change those settings via the web front of my app.
I imagine I could use a new model "GlobalSettings" with only one item, but I don't really know if it's a good "MVC" practice.
Since it is based on user input, it has to be stored in a database and therefore storing it in model seems like a right choice to me.
Having just 1 row/collection is completely ok in my opinion, especially in the no-SQL field. But for more reusability and scalability, you might want to consider to actually store each setting in invididual row, that might give you space to expand the usability of it in the future.
In my own opinion, I always find as a web app develops, you will start to realize there are more and more fields that you want the user to setup as their preference, as a good practice to relax the application.
For me I usually setup a meta_data model with name, value, criteria, and some other fields.
For example, when viewing your web page, 'Alice' may want a background color of black, 'Bob' may want a background color of green. Then you can let them modify or insert row into this meta_data collection. Then in your database, you will have
name value criteria
background_color black user_name='Alice'
background_color green user_name='Bob'
and it can be all kinds of values.
of course if you just have one value that can be changed by all of your users, it is probably a good idea to know who updated them. For this you would want to create a trigger, (if you are using a sql database)see trigger in mysql, so that every update on the table will trigger a function that stores what was changed and who changed it in another table
So yes, to answer your question, it is totally ok to have a model to store a value, and don't worry about only have one row, you will have more as you develop your app.
The config/globals.js file seems to be a good place to place a global configuration setting.
For convenience, Sails exposes a handful of global variables. By
default, your app's models, services, and the global sails object are
all available on the global scope; meaning you can refer to them by
name anywhere in your backend code (as long as Sails has been loaded).
Nothing in Sails core relies on these global variables - each and
every global exposed in Sails may be disabled in sails.config.globals
(conventionally configured in config/globals.js.)
Sailsjs.org Documentation - Globals
Alternatively you can use the sails.config object.
Custom Configuration
Sails recognizes many different settings, namespaced under different top level keys (e.g. sails.config.sockets and
sails.config.blueprints). However you can also use sails.config for
your own custom configuration (e.g.
sails.config.someProprietaryAPI.secret).
From the docs
There is also services which is global by default.
Overview
Services can be thought of as libraries which contain functions that you might want to use in many places of your application. For example,
you might have an EmailService which wraps some default email message
boilerplate code that you would want to use in many parts of your
application. The main benefit of using services in Sails is that they
are globalized--you don't have to use require() to access them.
It really depends on what kind of global you are wanting.
Has any one used WF Rules engine outside workflow activities WITHOUT using Rules Editor or CodeDOM?
Scenario
I am trying to use the Rule engine that ships with Workflow foundation classes with .Net Framework for a web based application. We evaluate hundreds of Validation Rules before proceeding towards a calculation engine for a complex calculation.
I have gone through many blogs which state how to use Rules engine without having to have Workflow Activities including https://github.com/geersch/WorkflowRulesEngine.
However they all eventually end up using a Rule Set Editor for defining rules. I want to declaratively define rules in an XML file
What I am looking for
I need to be able to specify Rules in simple XML without dealing with CODEDOM. CodeDOM seems to be very complex to write.
I should not be dependent on RuleSet Editor for defining rules.
Any thoughts any one ?
Don't know what kind of rules you are evaluating, but if you search a good open source rule engine, here is one.
Have a look at the following article for a strategy (not a solution) to approach the problem.
http://code.msdn.microsoft.com/windowsdesktop/Creating-Rules-Using-the-23c5d561
The serialization format for WF Rules is CodeDom.
It does not have another serialization format built-in.
If you want an alternate XML representation, you would need to create a tool to map between the representations.