We have a queryablestate for storing some information when the system is initialized, these states are issued once and never consumed
class DataState implements LinearState, QueryableState {
Party partyA;
Party partyB;
String partyAId;
String partyBId;
String sharedIdentifierNumber;
DataState(Party partyA, Party partyB, String partyAId, String partyBId, String sharedIdentifierNumber) {
this.partyA = partyA;
this.partyB = partyB;
this.partyAId = partyAId;
this.partyBId = partyBId;
this.sharedIdentifierNumber = sharedIdentifierNumber;
}
}
partyA and partyAId must be related to entity A (same for partyB)
some example instances:
new DataState(party1, party2, "00001", "12345", "0001")
new DataState(party3, party1, "05432", "00001", "0022")
new DataState(party2, party1, "12345", "00001", "0123")
we want to have methods that work like a map:
String retrievePartyId(Party party){}
assertTrue(retrievePartyId(party1).equals("00001"))
Party retrieveParty(String partyId){}
assertTrue(retrieveParty("12345").equals(party2))
we have already done this by querying all the states with custom field criteria and comparing through iteration on the
List<StateAndRef<DataState>>
We would like to know if there is some efficient way of doing this, maybe with some custom querycriteria in Corda. We think this is related to sql column projections. The query interface returns list of 'states(.getStates())' OR aggregation results '(.getOtherResults())'. We were wondering if it’s possible (or planned) to get a single column from the db and then filter the duplicates through the vault interface, currently we’re doing that in java.
If your states are QueryableStates, they can be efficiently queried based on their schema's attributes.
For example, suppose in the CorDapp Example (https://github.com/corda/cordapp-example), I wanted to query the vault for IOUStates based on their value and whether I am the lender. I could do this as follows:
try {
Field value = IOUSchemaV1.PersistentIOU.class.getDeclaredField("value");
Field lender = IOUSchemaV1.PersistentIOU.class.getDeclaredField("lender");
QueryCriteria valueOver50Criteria = new QueryCriteria.VaultCustomQueryCriteria(greaterThan(value, 50));
QueryCriteria lenderIsMeCriteria = new QueryCriteria.VaultCustomQueryCriteria(equal(lender, getOurIdentity().getName().toString()));
QueryCriteria criteria = valueOver50Criteria.and(lenderIsMeCriteria);
Vault.Page<IOUState> results = getServiceHub().getVaultService().queryBy(IOUState.class, criteria);
List<StateAndRef<IOUState>> matchingStates = results.getStates();
} catch (NoSuchFieldException e) {
e.printStackTrace();
}
In your case, you would be querying based on the party's ID or the party's name. You would then take the first result and use it to map the party's name to the party's ID, or vice-versa.
Related
I have function from my RPC client that pulls the consumed history of a state. If I pass a custom criteria, it seem to ignore the general criteria with Vault StateStatus.
My general query looks like this:
QueryCriteria generalCriteria = new QueryCriteria.VaultQueryCriteria(Vault.StateStatus.CONSUMED);
My custom criteria like this:
customCriteria = generalCriteria;
if (requestJSON.containsKey("linearId")) {
String linearId = requestJSON.get("linearId").toString();
FieldInfo fieldInfo = getField("linearId", POSchemaV1.POSchemaPersistence.class);
criteriaExpression = Builder.equal(fieldInfo, linearId);
customCriteria = new QueryCriteria.VaultCustomQueryCriteria(criteriaExpression);
customCriteria = generalCriteria.and(customCriteria);
}
The vault query looks as below:
Vault.Page<PurchaseOrder> orderPage = proxy.vaultQueryByCriteria(customCriteria, PurchaseOrder.class);
List<StateAndRef<PurchaseOrder>> orderStateAndRefList = orderPage.getStates();
The result seem to have unconsumed states if I pass linearId, if I ignore linearId then I get all consumed states correctly. pls let me know if I am missing something.
There is a more simple (out of the box) approach to query linear states:
QueryCriteria queryCriteria = new QueryCriteria.LinearStateQueryCriteria()
.withStatus(Vault.StateStatus.CONSUMED)
.withUuid(Collections.singletonList(linearId));
What if you don't have the linearId at hand; but have other custom attributes. It just returns UNCOSUMED states. If it is a linearState you may have to query twice, once to get UNCONSUMED state and then use that linearID to get the CONSUMED states.
How will it work if you have states that are consumed and you want to query for certain customCriteria and don't have the linearID? I don't want to query all the all states ( there can me lots of them ) and look for specific ones that match from all records that were retrieved. I assumed, VaultCustomQueryCriteria was supposed to do that.
How can I get items count for a particular partition key using .net core preferably using Object Persistence Interface or Document Interfaces?
Since I do not see any docs any where, currently I get the number of items count by retrieve all the item and get its count, but it is very expensive to do the reads.
What is the best practices for such item count request? Thank you.
dynamodb is mostly a document oriented key-value db; so its not optimized for functionality of the common relation db functions (like item count).
to minimize the data that is transmitted and to improve speed you may want to do the following:
Create Lambda Function that returns Item Count
To avoid transmitting data outside of AWS; which is slow and expensive.
query options
use only keys in your projection-expression,
reducing the data that is transmitted from db
max page-size, reducing number of calls needed
Stream Option
Streams could also be used for keeping counts; e.g. as described in
https://medium.com/signiant-engineering/real-time-aggregation-with-dynamodb-streams-f93547cfb244
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-gsi-aggregation.html
Related SO Question
Complexity of finding total records count with partition key in nosql dynamodb table?
I just realized that using low level interface in QueryRequest one can set Select = "COUNT" then when calling QueryAsync() orQuery() will return the count only as a integer only. Please refer to code sample below.
private static QueryRequest getStockRecordCountQueryRequest(string tickerSymbol, string prefix)
{
string partitionName = ":v_PartitionKeyName";
string sortKeyPrefix = ":v_sortKeyPrefix";
var request = new QueryRequest
{
TableName = Constants.TableName,
ReturnConsumedCapacity = ReturnConsumedCapacity.TOTAL,
Select = "COUNT",
KeyConditionExpression = $"{Constants.PartitionKeyName} = {partitionName} and begins_with({Constants.SortKeyName},{sortKeyPrefix})",
ExpressionAttributeValues = new Dictionary<string, AttributeValue>
{
{ $"{partitionName}", new AttributeValue {
S = tickerSymbol
}},
{ $"{sortKeyPrefix}", new AttributeValue {
S = prefix
}}
},
// Optional parameter.
ConsistentRead = false,
ExclusiveStartKey = null,
};
return request;
}
but I would like to point out that this still will consumed the same read units as retrieving all the item and get its count by yourself. but since it is only returning the count as an integer, it is a lot more efficient then transmitting the entire items list cross the wire.
I think using DynamoDB Streams in a more proper way to get the counts for large project. It is just a lot more complicated to implement.
I want to suppress the CosmosDB information in the following resultset, how can that be done?
{
"id": null,
"_rid": null,
"_self": null,
"_ts": 0,
"_etag": null,
"topLevelCategory": "Shorts,Skirt"
},
This is an extract of course but I dont want to show the ID etc as they serve no purpose in this result but I cannot figure out how to suppress that info.
I expect the following
{
"topLevelCategory": "Shorts,Skirt"
},
Query looks as follows
$"SELECT DISTINCT locales.categories[0] AS topLevelCategory " +
$"FROM c JOIN locales in c.locales " +
$"WHERE locales.country = '{apiInputObject.Locale}' " +
$"AND locales.language = '{apiInputObject.Language}'";
Interesting thing is if I cast the result as a JOBJECT I dont get the system data, I only get it if I createDOcumentQuery as DOcument, so a workaround would be as follows
IQueryable<JObject> queryResultSet = client.CreateDocumentQuery<JObject>(UriFactory.CreateDocumentCollectionUri(databaseName, databaseCollection), parsedQueryObject.SqlStatement, queryOptions);
but that has other async issues but the above does not show the system generate IDs but the below one does
var query = client.CreateDocumentQuery<Document>(UriFactory.CreateDocumentCollectionUri(databaseName, databaseCollection), parsedQueryObject.SqlStatement, queryOptions).AsDocumentQuery();
var result = await query.ExecuteNextAsync<Document>();
These are system-generated properties of items in Cosmos DB.
Surely,you could filter them in the sql: select c.topLevelCategory from c, don't mention them or use select * from c. Filtering in sql is the best method, better than secondary processing of result set.
Update Answer:
Your situation is executing the exact same query the JOBJECT does not show the system data but the Document does.
My explanation as below:
Document Class is a self-contained base class of Document DB .NET package.It has these generate properties:
SDK will try to map the result data one by one to the entity class which you defined in the CreateDocumentQuery<T>.
So actually,you already find the solution.You could define your custom pojo to receive the result data. Just contain the properties you want in that pojo inside like:
class Pojo : Document
{
public string id { get; set; }
public string name { get; set; }
}
That would have both business implications and no more redundant fields.Hope i'm clear on this.
To get state I can use Vault, but what about transactions? How I can get them, for example, by txHash? Is it possible to do this by CordaRPCOps, there is internalVerifiedTransactionsSnapshot method, but it is deprecated now.
First, note that as of Corda 3, there are no stability guarantees regarding the behaviour of any method to retrieve a transaction or its dependencies. In particular, we cannot guarantee that the set of transactions retrieved will not change across Corda versions.
This is because in future versions of Corda, nodes will likely only exchange transaction chains in SGX-encrypted form. These transaction chains will then be verified inside an SGX enclave on the node. This will prevent nodes from seeing the contents of the transactions they are verifying (see the blogpost here: https://www.corda.net/2017/06/corda-sgx-privacy-update/). This may even go so far as to only allow nodes to see certain parts of the transactions they are signing.
Ways to retrieve transactions as of Corda 3
1. Using CordaRPCOps.internalVerifiedTransactionsSnapshot
If you are interacting with the node via RPC, CordaRPCOps.internalVerifiedTransactionsSnapshot returns a list of all recorded transactions.
If you only wanted to get a single transaction and you knew its hash, you could write:
val transactions = cordaRPCOps.internalVerifiedTransactionsSnapshot()
val signedTransaction = transactions
.find { it.id == transactionHash }
?: throw IllegalArgumentException("Unknown transaction hash.")
Note that the transactions returned are of type SignedTransaction. This form does not contain the transaction's attachments or inputs (only the attachment hashes and input state references).
To retrieve a transaction's attachments via RPC, you could write:
val transactions = cordaRPCOps.internalVerifiedTransactionsSnapshot()
val signedTransaction = transactions
.find { it.id == transactionHash }
?: throw IllegalArgumentException("Unknown transaction hash.")
val attachmentHashes = signedTransaction.tx.attachments
val attachmentStreams = attachmentHashes.map { hash -> cordaRPCOps.openAttachment(hash) }
And to retrieve a transaction's inputs via RPC, you could write:
val transactions = cordaRPCOps.internalVerifiedTransactionsSnapshot()
val signedTransaction = transactions
.find { it.id == transactionHash }
?: throw IllegalArgumentException("Unknown transaction hash.")
val inputStateRefs = signedTransaction.inputs
val inputStates = inputStateRefs.map { stateRef ->
val transaction = transactions.find { it.id == stateRef.txhash }
?: throw IllegalArgumentException("Unknown transaction hash.")
transaction.tx.outputStates[stateRef.index]
}
2. Using the ServiceHub
If you are in a situation where you have access to the node's ServiceHub (e.g. within a flow or a Corda service), you can use serviceHub.validatedTransactions.track().snapshot to get all transactions, and serviceHub.validatedTransactions.getTransaction(transactionHash) to get a specific transaction by hash.
Note that the transactions returned are of type SignedTransaction. This form does not contain the transaction's attachments or inputs (only the attachment hashes and input state references).
To convert the SignedTransaction to a LedgerTransaction (where the attachments and inputs are resolved), you could write:
val signedTransaction = serviceHub.validatedTransactions.getTransaction(transactionHash)
val ledgerTransaction = signedTransaction.toLedgerTransaction(serviceHub)
3. By connecting to the node's database
You can connect directly to the SQL database backing the node, and retrieve the transactions using an SQL query.
That's right, although please note that the ServiceHub and SQL approaches are basically the same thing as the deprecated RPC and may also stop working in future (or not, depending on how we manage the transition to an encrypted ledger).
There are other approaches you can use. For instance you could aggregate the bits of history you care about up into the latest version of the state. This also lets you restrict the view of the history once SGX lands.
The first solution (Using CordaRPCOps.internalVerifiedTransactionsSnapshot) is really slow.
It is exist one more way to get transaction history and it is pretty effective.
You can do it by using rpcOps.vaultQueryBy
fun transaction(transactionId: String): List<Vault.Page<ContractState>> {
// get jdbc connection (you may simplify it within cordapp)
val jt = jt()
// get all states of transaction
val output_indexes = jt.queryForList("SELECT OUTPUT_INDEX FROM VAULT_STATES WHERE transaction_id = '$transactionId'", Int::class.java)
val transactionHash = SecureHash.parse(transactionId)
// get Rpc connection
val rpcOps = initialiseNodeRPCConnection()
val transactionStates = output_indexes.map {
val constraintTypeCriteria = QueryCriteria.VaultQueryCriteria(status = Vault.StateStatus.ALL, stateRefs = listOf(StateRef(transactionHash, it)))
rpcOps.vaultQueryBy<ContractState>(constraintTypeCriteria)
}
return transactionStates
}
In RavenDB I can store objects of type Products and Categories and they will automatically be located in different collections. This is fine.
But what if I have 2 logically completely different types of products but they use the same class? Or instead of 2 I could have a generic number of different types of products. Would it then be possible to tell Raven to split the product documents up in collections, lets say based on a string property available on the Product class?
Thankyou in advance.
EDIT:
I Have created and registered the following StoreListener that changes the collection for the documents to be stored on runtime. This results in the documents correctly being stored in different collections and thus making a nice, logically grouping of the documents.
public class DynamicCollectionDefinerStoreListener : IDocumentStoreListener
{
public bool BeforeStore(string key, object entityInstance, RavenJObject metadata)
{
var entity = entityInstance as EntityData;
if(entity == null)
throw new Exception("Cannot handle object of type " + EntityInstance.GetType());
metadata["Raven-Entity-Name"] = RavenJToken.FromObject(entity.TypeId);
return true;
}
public void AfterStore(string key, object entityInstance, RavenJObject metadata)
{
}
}
However, it seems I have to adjust my queries too in order to be able to get the objects back. My typical query of mine used to look like this:
session => session.Query<EntityData>().Where(e => e.TypeId == typeId)
With the 'typeId' being the name of the new raven collections (and the name of the entity type saved as a seperate field on the EntityData-object too).
How would I go about quering back my objects? I can't find the spot where I can define my collection at runtime prioring to executing my query.
Do I have to execute some raw lucene queries? Or can I maybe implement a query listener?
EDIT:
I found a way of storing, querying and deleting objects using dynamically defined collections, but I'm not sure this is the right way to do it:
Document store listener:
(I use the class defined above)
Method resolving index names:
private string GetIndexName(string typeId)
{
return "dynamic/" + typeId;
}
Store/Query/Delete:
// Storing
session.Store(entity);
// Query
var someResults = session.Query<EntityData>(GetIndexName(entity.TypeId)).Where(e => e.EntityId == entity.EntityId)
var someMoreResults = session.Advanced.LuceneQuery<EntityData>(GetIndexName(entityTypeId)).Where("TypeId:Colors AND Range.Basic.ColorCode:Yellow)
// Deleting
var loadedEntity = session.Query<EntityData>(GetIndexName(entity.TypeId)).Where(e =>
e.EntityId == entity.EntityId).SingleOrDefault();
if (loadedEntity != null)
{
session.Delete<EntityData>(loadedEntity);
}
I have the feeling its getting a little dirty, but is this the way to store/query/delete when specifying the collection names runtime? Or do I trap myself this way?
Stephan,
You can provide the logic for deciding on the collection name using:
store.Conventions.FindTypeTagName
This is handled statically, using the generic type.
If you want to make that decision at runtime, you can provide it using a DocumentStoreListner