Is it Possible to read Nifi flowfile data by the help of nifi expression language? - bigdata

Basically I want to read flowfile data and call another processor based pre-processor data.
example:- kafka consumer read data (processor) ---> flowfile(collection id) --> Getmongo (processor) get data from mongo based on pre processor data (collection id). Is it possible?

you can use UpdateAttribute processor to updates the Attributes for a FlowFile by using the Attribute Expression Language and/or deletes the attributes based on a regular expression. ExtractText processor, adding a custom attribute with a value set to the regular expression "(.*)". create a new attribute with value as the content of the flowfile

Related

Serialize and Deserialize JSON with different property names for different use cases

I have data stored in a database with short field names to save space. When I load these into my .NET Core app, I convert the data to its C# model. Then, when I pass this to the fronted I want to serialize the JSON to have full field names. So instead of "fN", "firstName".
I know I can use [JsonProperty(PropertyName = "SomeName")] to change serialization/deserialization. What I'm asking is whether there's a way to specify multiple property names for a single field, ideally with some additional "profile" variable so that when I serialize/deserialize from the DB I use the DB profile, and when sending to the client I use the other profile. This way I can either use the short or long names when convenient.
Thanks!

Can you save a result (Given) to a variable in a Gherkin feature file, and then compare the variable with another result (Then)? (Cucumber for Java)

I am new to Cucumber for Java and trying to automate testing of a SpringBoot server backed by a MS SQL Server.
I have an endpoint "Get All Employees".
Writing the traditional feature file, I will have to list all the Employees in the #Then clause.
This is not possible with thousands of employees.
So I just want to get a row count of the Employee table in the database, and then compare with the number of objects returned from the "Get All Employees" endpoint.
Compare
SELECT count(*) from EMPLOYEE
with size of the list returned from
List<Employee> getAllEmployees()
But how does one save the rowcount in a variable in the feature file and then pass it into the stepdefs Java method?
I have not found any way that Gherkin allows this.
After writing a few scenario and feature files, I understood this about Cucumber and fixed the issue.
Gherkin/Cucumber is not a programming language. It is just a specification language. When keywords like Given, Then are reached by the interpreter, the matching methods in Java code are called. So they are just triggers.
These methods are part of a Java glue class. Data is not passed out of the Java class and into the gherkin feature file. The class is instantiated at the beginning and retained until the end. Because of this, it can store state.
So from my example in the question above, the Then response from a Spring endpoint call will be stored in a member variable in the glue class. The next Then invocation to verify the result will call the corresponding glue method which will access the data in the member variable to perform the comparison.
So Gherkin cannot do this, but Java at a lower level in the glue class, can.
You could create a package for example named dataRun (with the corresponding classes in the package) and save there the details during the test via setters.
During the execution of the step "And I get the count of employees from the data base" you set this count via the corresponding setter, during the step "And I get all employees" you set the number via the dedicated setter.
Then during the step "And I verify the number of employees is the same as the one in the data base" you get the two numbers via the getters and compare them.
Btw it's possible to compare the names of the employees (not just the count) if you put them into list and compare the lists.

How to store only node specific off-ledger custom data in corda?

I created custom table in corda using QueryableState. e.g. IOUStates table.
I can able to see the custom information getting stored in this kind of table.
but i observed that if party A and Party B is doing the transaction then this
custom information gets stored at both the places , e.g. IOUStates
table getting created at nodeA ledger as well as nodeB's ledger.
and custom information is stored in partyA's and PartyB's ledger.
My Question is :-
If some Transaction is getting processed from PartyA's node , then
I want to store part of the transaction's data i.e. custom data ONLY at partyA's Ledger.* level . i.e. off-Ledger of partA only.
It should not be shared with partyB.
In simple case , how to store Only node specific off ledger custom data ?
Awaiting for some reply...
Thanks.
There's a number of ways to achieve this:
Don't use Corda at all! If the data is truly off-ledger then why are you using Corda? Instead, store it in a separate database. Of course you can "JOIN" it with on-ledger data if required, as the on-ledger data is stored in a SQL database.
Similar to point one except you can use the jdbcSession() functionality of the ServiceHub to create a custom table in the node's database. This table can easily be accessed from within your flows.
Create a ContractState object that only has one participant: the node that wants to store the data. I call this a "unilateral" state, i.e. a state that only one party ever stores.
Most importantly, if you don't want to share some data with a counter-party then it should never be disclosed inside a corda state object or attachment that another party might see. Instead:
inside your flows, you can use the data encapsulated within the shared state object (e.g. the IOU) to derive the private data
alternatively if the data is supplied when the flow begins then store the private data locally using one of the methods above

How to model Not In query in Couch DB [duplicate]

Folks, I was wondering what is the best way to model document and/or map functions that allows me "Not Equals" queries.
For example, my documents are:
1. { name : 'George' }
2. { name : 'Carlin' }
I want to trigger a query that returns every documents where name not equals 'John'.
Note: I don't have all possible names before hand. So the parameters in query can be any random text like 'John' in my example.
In short: there is no easy solution.
You have four options:
sending a multi range query
filter the view response with a server-side list function
using a CouchDB plugin
use the mango query language
sending a multi range query
You can request the view with two ranges defined by startkey and endkey. You have to choose the range so, that the key John is not requested.
Unfortunately you have to find the commit request that somewhere exists and compile your CouchDB with it. Its not included in the official source.
filter the view response with a server-side list function
Its not recommended but you can use a list function and ignore the row with the key John in your response. Its like you will do it with a JavaScript array.
using a CouchDB plugin
Create an additional index with e.g. couchdb-lucene. The lucene server has such query capabilities.
use the "mango" query language
Its included in the CouchDB 2.0 developer preview. Not ready for production but will be definitely included in the stable release.

Modality work list - Which items are returned for C-FIND request of a sequence?

My question is a really basic question. Consider to query a modality work list to get some work items by a C-FIND query. Consider using a sequence (SQ) as Return Key attribute for the C-FIND query, for example: [0040,0100] (Scheduled Procedure Step) and universal matching.
What should I expect in the SCP's C-FIND response? Or, better say, what should I expect to find with regards of the scheduled procedure step for a specific work item? All the mandatory items that Modality Work List Information Model declare as encapsulated in the sequence? Should I instead explicitly issue a C-FIND request for those keys I want the SCP return in the response?
For example: if I want the SCP return the Scheduled Procedure Step Start Time and Scheduled Procedure Start Date, do I need to issue a specific C-FIND request with those keys or querying for Scheduled Procedure Step key is enough to force the SCP to send all items related to the Scheduled Procedure Step itself?
Yes, you should include the Scheduled Procedure Step Start Time / Date Tags into the 0040,0100 sequence.
See also Service Class Specifications (K6.1.2.2)
This will not ensure you will retrieve this information, because it depends on the Modality Worklist Provider, which information will be returned.
You could also request a Dicom Conformance Statement from the Modality Provider to know the necessary tags for request/retrieve.
As for table K.6-1, you can consider it as showing only the requirement of the SCP side or what SCP is required to use for matching key (i.e. query filter) and additional required attribute values to return (i.e. Return Key) with successful match. It is up to SCP’s implementation to support matching against required key but you can always expect SCP to use the values in matching key for query filter.
Also note that, SCP is only required to return values for attributes that are present in the C-FIND Request. One exception is the sequence matching and there you have the universal matching like mechanism where you can pass a zero length ITEM to retrieve entire sequence. So as stated in PS 3.4 section C.2.2.2.6, you can just include an empty ITEM (FFFE, E000) element with VR of SQ under Scheduled Procedure Step Sequence (0040, 0100) for universal matching.

Resources