I'm working with Corda, in case of querying data from Vault of Node. In basic way, I used the Schema to add some criteria for this querying. For example:
val quantityIndex = SchemaV1.Persistent::value.greaterThanOrEqual(4)
So, there are anyway to query the State in Vault without using Schema? Because I believe that If I am able to use the class of State instead of Schema class, I can query State with better criteria rule.
As of Corda 3, there is no way to do this. This is because other than the properties that are extracted as part of the schema, the state is simply stored as a Java blob.
You'll have to extract the states then filter them in memory:
List<StateAndRef<ArtState>> artStateAndRefs = getServiceHub().getVaultService().queryBy(ArtState.class).getStates();
StateAndRef<ArtState> inputArtStateAndRef = artStateAndRefs
.stream().filter(artStateAndRef -> {
ArtState artState = artStateAndRef.getState().getData();
return artState.getArtist().equals(artist) && artState.getTitle().equals(title);
})
.findAny()
.orElseThrow(() -> new IllegalArgumentException("The piece of art was not found."));
If there are a large number of states, you'll have to filter across each vault page individually then collate the results.
Related
I'm new to Android, and rather new to SQL in general.
I have a data model where I have a Text that consists of TextMetadata as well as a long string, which is the text content itself. So
Text {
metadata: {
author: string,
title: string
// more associated metadata
},
textContent: long string, or potentially array of lines or paragraphs
}
I'd like to load a list of the metadata for all texts on the App's landing page, without incurring the cost of reading all the long strings (or having operations be slowed down because the table has a column with a long string?).
What is the proper pattern here? Should I use two tables, and related them? Or can I use one table/one #Entity, with embedded metadata, and do some fancy stuff in the DAO to just list/sort/operate on the embedded metadata?
Most of my background is with NoSQL databases, so I could be thinking about this entirely wrong. Advice on the general best practices here would be helpful, but I guess I have two core questions:
Does having a long/very long string/TEXT column cause performance considerations when operating on that specific table/row?
Is there a clean way using Kotlin annotations to express embedded metadata that would make it easy to fetch in the DAO, without having use a long SELECT for each individual column?
This is a good question that is also relevant to other environments.
The Core Issue: How to store large data without effecting your database?
As a rule of thumb you should avoid storing information in your database that is not queryable. Large strings, images, or event metadata which you will never query - does not belong in your db. I was surprised when I realized how many design patterns there are regarding to mongo db (which are relevant to other noSQL databases as well)
So, we know that this data should NOT be stored in the DB. But, because the alternative (file system) is WAY worse than that (unless you would like to implement your own secured file-system-based store) - we should at least try to minimize its footprint.
Our Strategy: save large data chunks in a different table without defining it as an entity (there is no need to wrap it as entity anyway)
How Are We Going To Do That?
Well, thankfully, android room has a direct access to sqLite and it can be used directly (read the docs). This is the place to remind us that android room is built on-top of sqLite - which is (in my own opinion) a fascinating database. I enjoy working with it very much and it's just getting better as the time goes by (personal opinion). Advantages? we are still using android APIs while storing large data in a performant, unified and secure way. yay
Steps we are going to perform:
Initiate a class which will manage a new database - for storing large data only
Define a command that will create our table - constructed of 2 columns
key (primary key) - the id of the item
value - the item itself
In original db for the Text entity - define a column that will hold the id (key) of the large text stored
Whenever you save an item to your large items table - get the id and store it in your entity
You can of course use only 1 table for this.. but.. I know that sqLite requires a certain amount of understanding and it is NOT as easy as android room so.. it's your choice whenever to use 1 or 2 tables in your solution
Below is a code that demonstrates the main principal of my proposal
object LargeDataContract {
// all tables for handling large data will be defined here
object TextEntry : BaseColumns {
const val TABLE_NAME = "text_entry"
const val COLUMN_NAME_KEY = "key"
const val COLUMN_NAME_VALUE = "value"
}
}
// in the future - append this "create statement" whenever you add more tables to your database
private const val SQL_CREATE_ENTRIES =
"CREATE TABLE ${TextEntry.TABLE_NAME} (" +
"${TextEntry.COLUMN_NAME_KEY} INTEGER PRIMARY KEY," +
"${TextEntry.COLUMN_NAME_VALUE} TEXT)"
// create a helper that will assist you to initiate your database properly
class LargeDataDbHelper(context: Context) : SQLiteOpenHelper(context, DATABASE_NAME, null, DATABASE_VERSION) {
override fun onCreate(db: SQLiteDatabase) {
db.execSQL(SQL_CREATE_ENTRIES)
}
companion object {
// If you change the database schema, you must increment the database version. Also - please read `sqLite` documentation to better understand versioning ,upgrade and downgrade operations
const val DATABASE_VERSION = 1
const val DATABASE_NAME = "LargeData.db"
}
}
// create an instance and connect to your database
val dbHelper = LargeDataDbHelper(context)
// write an item to your database
val db = dbHelper.writableDatabase
val values = ContentValues().apply {
put(TextEntry.COLUMN_NAME_VALUE, "some long value goes here")
}
val key = db?.insert(TextEntry.TABLE_NAME, null, values)
// now take the key variable and store it in you entity. this is the only reference you should need
Bottom Line: This approach will assist you to gain as much performance as possible while using android APIs. Sure thing, not the most "intuitive" solution, but - this is how we gain performance and making great apps as well as educating ourselves and upgrading our knowledge and skillset. Cheers
I am new to coding. I appreciate the help.
I already create a DynamoDB table, API Gateway, child resource, child method(POST), integration type: AWS service proxy. I also created an IAM role to allow access to DynamoDB putitem.
I tried to add something to the table with postman and it worked well. However, now I want to add an item to the table with boto3 and I'm finding it hard to identify the method that I should be using.
dydbsr = boto3.resource('dynamodb')
dbtable = 'myTable'
table = dydbsr.Table(dbtable)
If you have a partition key only:
partion key is: mypartitionkey
json_dictionary = {"mypartitionkey": "myvalue"}
table.put_item(Item=json_dictionary)
If you have a partition key and sort key. If sort key is defined it MUST be included
partion key is: mypartitionkey
sort key is: mysortkey
json_dictionary = {"mypartitionkey": "myvalue", "mysortkey": "myothervalue"}
table.put_item(Item=json_dictionary)
After adding partition and sort keys, then your attributes are just key:value pairs in the same json dictionary
Refer to the AWS documentation here for examples to create, read, update, and delete an item with Python.
Consider Below is my sample json.
{
"servletname": "cofaxEmail",
"servlet-class": "org.cofax.cds.EmailServlet",
"init-param": {
"mailHost": "mail1",
"mailHostOverride": "mail2"
}
i have chosen "servletname" as my primary key as i will receive it in every request plus few 1000 server names are there it could be the best PK.
My Question is, to make the partition key work for me.
Do i have to specify the partition key option seperately like below
ItemResponse<ServerDto> ServerDtoResponse = await this.container.ReadItemAsync<ServerDto>(bocServerDto.mailHost, new PartitionKey(bocServerDto.servletname));
or
Including the partition key in the select query itself , without adding seperate new PartitionKey(), like
select * from r where r.servletname='cofaxEmail' and r.mailHost='mail1';
Crux of the question is: By passing partitionKey object in where condition of select query is it enough to utilize the partition key feature?
Thanks
For any crud operation you would pass in the value for the partition key. For example, on a point read.
ItemResponse<ServerDto> ServerDtoResponse = await this.container.ReadItemAsync<ServerDto>(bocServerDto.mailHost, new PartitionKey("cofaxEmail"));
For a query, you can either pass it in the queryRequest options or use it in the query as the first filter predicate. Here is an example of using the queryRequest options.
thanks.
First of all, I have table structure like this,
Users:{
UserId
Name
Email
SubTable1:[{
Column-111
Column-112
},
{
Column-121
Column-122
}]
SubTable2:[{
Column-211
Column-212
},
{
Column-221
Column-222
}]
}
As I am new to DynamoDB, so I have couple of questions regarding this as follows:
1. Can I create structure like this?
2. Can we set primary key for subtables?
3. Luckily, I found DynamoDB helper class to do some operations into my DB.
https://www.gopiportal.in/2018/12/aws-dynamodb-helper-class-c-and-net-core.html
But, don't know how to fetch only perticular subtable
4. Can we fetch only specific columns from my main table? Also need suggestion for subtables
Note: I am using .net core c# language to communicate with DynamoDB.
Can I create structure like this?
Yes
Can we set primary key for subtables?
No, hash key can be set on top level scalar attributes only (String, Number etc.)
Luckily, I found DynamoDB helper class to do some operations into my DB.
https://www.gopiportal.in/2018/12/aws-dynamodb-helper-class-c-and-net-core.html
But, don't know how to fetch only perticular subtable
When you say subtables, I assume that you are referring to Array datatype in the above sample table. In order to fetch the data from DynamoDB table, you need hash key to use Query API. If you don't have hash key, you can use Scan API which scans the entire table. The Scan API is a costly operation.
GSI (Global Secondary Index) can be created to avoid scan operation. However, it can be created on scalar attributes only. GSI can't be created on Array attribute.
Other option is to redesign the table accordingly to match your Query Access Pattern.
Can we fetch only specific columns from my main table? Also need suggestion for subtables
Yes, you can fetch specific columns using ProjectionExpression. This way you get only the required attributes in the result set
Is it possible to retrieve an entity from gae datastore using a property and not using the key?
I could see I can retrieve entities with key using the below syntax.
quote = mgr.getObjectById(Students.class, id);
Is there an alternative that enables us to use a property instead of key?
Or please suggest any other ways to achieve the requirement.
Thanks,
Karthick.
Of course this is possible. Think of the key of an entity being like the primary key of an SQL row (but please, don't stretch the analogy too far - the point is it's a primary key - the implementations of these two data storage systems are very different and it causes people trouble when they don't keep this in mind).
You should look either here (JDO) to read about JDO queries or here (JPA) to read about JPA queries, depending what kind of mgr your post refers to. For JDO, you would do something like this:
// begin building a new query on the Cat-kind entities (given a properly annotated
// entity model class "Cat" somewhere in your code)
Query q = pm.newQuery(Cat.class);
// set filter on species property to == param
q.setFilter("species == speciesParam");
// set ordering for query results by age property descending
q.setOrdering("age desc");
// declare the parameters for this query (format is "<Type> <name>")
// as referenced above in filter statement
q.declareParameters("String speciesParam");
// run the query
List<Cat> results = (List<Cat>) q.execute ("siamese");
For JPA, you would use JPQL strings to run your queries.