How the legal prose attached with contract can be retrieved from the receiving node side
Assuming you have attached a file to a transaction that you want to open on the side of the receiving node, you can use the following code:
val ids: List<AttachmentId> = serviceHub.attachments.queryAttachments(
AttachmentQueryCriteria.AttachmentsQueryCriteria(
filenameCondition = Builder.equal("some name")
)
)
val attachments: List<Attachment?> = ids.map { id -> serviceHub.attachments.openAttachment(id) }
Related
I'm now implement a program to migrate large amount of data to ADX base on Ingest from Storage feature of ADX and I'm need to check that status of each ingestion request each time the request finish but I'm facing an issue
Base on MS document in here
If I set the persistDetails = true for example with the command below it must save the ingestion status but currently this setting seem not work (with or without it)
.ingest async into table MigrateTable
(
h'correct blob url link'
)
with (
jsonMappingReference = 'table_mapping',
format = 'json',
persistDetails = true
)
Above command will return an OperationId and when I using it to check export status when the ingest task finish I always get this error message :
Error An admin command cannot be executed due to an invalid state: State='Operation 'DataIngestPull' does not persist its operation results' clientRequestId: KustoWebV2;
Can someone clarify for me what is the root cause relate to this? With me it seem like a bug relate to ADX
Ingesting data directly against the Data Engine, by running .ingest commands, is usually not recommended, compared to using Queued Ingestion (motivation included in the link). Using Kusto's ingestion client library allows you to track the ingestion status.
Some tools/services already do that for you, and you can consider using them directly. e.g. LightIngest, Azure Data Factory
If you don't follow option 1, you can still look for the state/status of your command using the operation ID you get when using the async keyword, by using .show operations
You can also use the client request ID to filter the result set of .show commands to view the state/status of your command.
If you're interested in looking specifically at failures, .show ingestion failures is also available for you.
The persistDetails option you specified in your .ingest command actually has no effect - as mentioned in the docs:
Not all control commands persist their results, and those that do usually do so by default on asynchronous executions only (using the async keyword). Please search the documentation for the specific command and check if it does (see, for example data export).
============ Update sample code follow suggestion from Yoni ========
Turn out, other member in my team mess up with access right with adx, after fixing it everything work fine
I just have one concern relate to PartiallySucceeded that need clarify from #yoni or someone have better knowledge relate to that
try
{
var ingestProps = new KustoQueuedIngestionProperties(model.DatabaseName, model.IngestTableName)
{
ReportLevel = IngestionReportLevel.FailuresAndSuccesses,
ReportMethod = IngestionReportMethod.Table,
FlushImmediately = true,
JSONMappingReference = model.IngestMappingName,
AdditionalProperties = new Dictionary<string, string>
{
{"jsonMappingReference",$"{model.IngestMappingName}" },
{ "format","json"}
}
};
var sourceId = Guid.NewGuid();
var clientResult = await IngestClient.IngestFromStorageAsync(model.FileBlobUrl, ingestProps, new StorageSourceOptions
{
DeleteSourceOnSuccess = true,
SourceId = sourceId
});
var ingestionStatus = clientResult.GetIngestionStatusBySourceId(sourceId);
while (ingestionStatus.Status == Status.Pending)
{
await Task.Delay(WaitingInterval);
ingestionStatus = clientResult.GetIngestionStatusBySourceId(sourceId);
}
if (ingestionStatus.Status == Status.Succeeded)
{
return true;
}
LogUtils.TraceError(_logger, $"Error when ingest blob file events, error: {ingestionStatus.ErrorCode.FastGetDescription()}");
return false;
}
catch (Exception e)
{
return false;
}
I want to suppress the CosmosDB information in the following resultset, how can that be done?
{
"id": null,
"_rid": null,
"_self": null,
"_ts": 0,
"_etag": null,
"topLevelCategory": "Shorts,Skirt"
},
This is an extract of course but I dont want to show the ID etc as they serve no purpose in this result but I cannot figure out how to suppress that info.
I expect the following
{
"topLevelCategory": "Shorts,Skirt"
},
Query looks as follows
$"SELECT DISTINCT locales.categories[0] AS topLevelCategory " +
$"FROM c JOIN locales in c.locales " +
$"WHERE locales.country = '{apiInputObject.Locale}' " +
$"AND locales.language = '{apiInputObject.Language}'";
Interesting thing is if I cast the result as a JOBJECT I dont get the system data, I only get it if I createDOcumentQuery as DOcument, so a workaround would be as follows
IQueryable<JObject> queryResultSet = client.CreateDocumentQuery<JObject>(UriFactory.CreateDocumentCollectionUri(databaseName, databaseCollection), parsedQueryObject.SqlStatement, queryOptions);
but that has other async issues but the above does not show the system generate IDs but the below one does
var query = client.CreateDocumentQuery<Document>(UriFactory.CreateDocumentCollectionUri(databaseName, databaseCollection), parsedQueryObject.SqlStatement, queryOptions).AsDocumentQuery();
var result = await query.ExecuteNextAsync<Document>();
These are system-generated properties of items in Cosmos DB.
Surely,you could filter them in the sql: select c.topLevelCategory from c, don't mention them or use select * from c. Filtering in sql is the best method, better than secondary processing of result set.
Update Answer:
Your situation is executing the exact same query the JOBJECT does not show the system data but the Document does.
My explanation as below:
Document Class is a self-contained base class of Document DB .NET package.It has these generate properties:
SDK will try to map the result data one by one to the entity class which you defined in the CreateDocumentQuery<T>.
So actually,you already find the solution.You could define your custom pojo to receive the result data. Just contain the properties you want in that pojo inside like:
class Pojo : Document
{
public string id { get; set; }
public string name { get; set; }
}
That would have both business implications and no more redundant fields.Hope i'm clear on this.
Assume a node is given a filtered transaction containing a few states, while some have been excluded out. How can the node run the smart contract verify function on the states that included in the transaction? Am trying to achieve something similar to ledgerTransaction.verify()
As of Corda 3, you cannot run the remaining states' verify methods, since the verify method requires a LedgerTransaction.
Instead, you would have to retrieve the states from the FilteredTransaction, and provide your own checking logic. For example:
val inputStateRefs = filteredTransaction.inputs
val inputStateAndRefs = inputStateRefs.map { inputStateRef ->
serviceHub.toStateAndRef<TemplateState>(inputStateRef)
}
inputStateAndRefs.forEach { inputStateAndRef ->
val state = inputStateAndRef.state
// TODO: Checking...
}
To get state I can use Vault, but what about transactions? How I can get them, for example, by txHash? Is it possible to do this by CordaRPCOps, there is internalVerifiedTransactionsSnapshot method, but it is deprecated now.
First, note that as of Corda 3, there are no stability guarantees regarding the behaviour of any method to retrieve a transaction or its dependencies. In particular, we cannot guarantee that the set of transactions retrieved will not change across Corda versions.
This is because in future versions of Corda, nodes will likely only exchange transaction chains in SGX-encrypted form. These transaction chains will then be verified inside an SGX enclave on the node. This will prevent nodes from seeing the contents of the transactions they are verifying (see the blogpost here: https://www.corda.net/2017/06/corda-sgx-privacy-update/). This may even go so far as to only allow nodes to see certain parts of the transactions they are signing.
Ways to retrieve transactions as of Corda 3
1. Using CordaRPCOps.internalVerifiedTransactionsSnapshot
If you are interacting with the node via RPC, CordaRPCOps.internalVerifiedTransactionsSnapshot returns a list of all recorded transactions.
If you only wanted to get a single transaction and you knew its hash, you could write:
val transactions = cordaRPCOps.internalVerifiedTransactionsSnapshot()
val signedTransaction = transactions
.find { it.id == transactionHash }
?: throw IllegalArgumentException("Unknown transaction hash.")
Note that the transactions returned are of type SignedTransaction. This form does not contain the transaction's attachments or inputs (only the attachment hashes and input state references).
To retrieve a transaction's attachments via RPC, you could write:
val transactions = cordaRPCOps.internalVerifiedTransactionsSnapshot()
val signedTransaction = transactions
.find { it.id == transactionHash }
?: throw IllegalArgumentException("Unknown transaction hash.")
val attachmentHashes = signedTransaction.tx.attachments
val attachmentStreams = attachmentHashes.map { hash -> cordaRPCOps.openAttachment(hash) }
And to retrieve a transaction's inputs via RPC, you could write:
val transactions = cordaRPCOps.internalVerifiedTransactionsSnapshot()
val signedTransaction = transactions
.find { it.id == transactionHash }
?: throw IllegalArgumentException("Unknown transaction hash.")
val inputStateRefs = signedTransaction.inputs
val inputStates = inputStateRefs.map { stateRef ->
val transaction = transactions.find { it.id == stateRef.txhash }
?: throw IllegalArgumentException("Unknown transaction hash.")
transaction.tx.outputStates[stateRef.index]
}
2. Using the ServiceHub
If you are in a situation where you have access to the node's ServiceHub (e.g. within a flow or a Corda service), you can use serviceHub.validatedTransactions.track().snapshot to get all transactions, and serviceHub.validatedTransactions.getTransaction(transactionHash) to get a specific transaction by hash.
Note that the transactions returned are of type SignedTransaction. This form does not contain the transaction's attachments or inputs (only the attachment hashes and input state references).
To convert the SignedTransaction to a LedgerTransaction (where the attachments and inputs are resolved), you could write:
val signedTransaction = serviceHub.validatedTransactions.getTransaction(transactionHash)
val ledgerTransaction = signedTransaction.toLedgerTransaction(serviceHub)
3. By connecting to the node's database
You can connect directly to the SQL database backing the node, and retrieve the transactions using an SQL query.
That's right, although please note that the ServiceHub and SQL approaches are basically the same thing as the deprecated RPC and may also stop working in future (or not, depending on how we manage the transition to an encrypted ledger).
There are other approaches you can use. For instance you could aggregate the bits of history you care about up into the latest version of the state. This also lets you restrict the view of the history once SGX lands.
The first solution (Using CordaRPCOps.internalVerifiedTransactionsSnapshot) is really slow.
It is exist one more way to get transaction history and it is pretty effective.
You can do it by using rpcOps.vaultQueryBy
fun transaction(transactionId: String): List<Vault.Page<ContractState>> {
// get jdbc connection (you may simplify it within cordapp)
val jt = jt()
// get all states of transaction
val output_indexes = jt.queryForList("SELECT OUTPUT_INDEX FROM VAULT_STATES WHERE transaction_id = '$transactionId'", Int::class.java)
val transactionHash = SecureHash.parse(transactionId)
// get Rpc connection
val rpcOps = initialiseNodeRPCConnection()
val transactionStates = output_indexes.map {
val constraintTypeCriteria = QueryCriteria.VaultQueryCriteria(status = Vault.StateStatus.ALL, stateRefs = listOf(StateRef(transactionHash, it)))
rpcOps.vaultQueryBy<ContractState>(constraintTypeCriteria)
}
return transactionStates
}
In Corda, is there a way to refer to an unspent state in a transaction without spending it? The aim is to allow the contract to use some of the information in the state being referred to as part of the verify method.
Reference states will be added to Corda Version 4 which will be released later this year. This solves the problem above. See:
https://github.com/corda/corda/blob/master/docs/source/design/reference-states/design.md
There is no built-in support for this pattern in Corda at this time, but it will be added in Corda 4 (see Roger's answer below). For now, you have several options:
Writing the states' contracts to allow this behaviour:
You can add a command to the contract which enforces the requirement that there is a matching output for each input of the state type you wish to reference. This guarantees that the transaction is only referencing the state, and not modifying it. Here is an example:
class MyContract : Contract {
override fun verify(tx: LedgerTransaction) {
val command = tx.commands.requireSingleCommand<MyContract.Commands>()
when (command.value) {
is Commands.Reference -> requireThat {
val inputs = tx.inputsOfType<MyState>()
val outputs = tx.outputsOfType<MyState>()
// Assuming `MyState.equals` has been overridden appropriately.
"There must be a matching output state for each input state" using
(inputs.toSet() == outputs.toSet())
}
}
}
interface Commands : CommandData {
class Reference: Commands
}
}
Referring the state as a field in an input state, output state or command:
You can include the reference state in the transaction as a field on an input state, output state or command. A command is likely to be the best fit:
interface Commands : CommandData {
class Reference(val referenceState: MyState): Commands
}
You can then check the contents of this state within the contract's verify method. For example:
class MyContract : Contract {
override fun verify(tx: LedgerTransaction) {
val command = tx.commands.requireSingleCommand<MyContract.Commands>()
when (command.value) {
is Commands.Reference -> requireThat {
val commandData = command.value as Commands.Reference
val referenceState = commandData.referenceStateAndRef.state.data
val inputs = tx.inputsOfType<MyState>()
"The input state contents must match the reference data" using
(inputs.all { it.contents == referenceState.contents })
}
}
}
interface Commands : CommandData {
class Reference(val referenceStateAndRef: StateAndRef<MyState>): Commands
}
}
With this approach, you also have to check in the flow that the reference state is identical to the state actually on the ledger (i.e. that the transaction's proposer hasn't added a fake state object as a reference). For example:
val referenceCommand = ledgerTransaction.commandsOfType<Reference>().single()
val referenceStateAndRef = referenceCommand.value.referenceStateAndRef
val actualStateAndRefFromVault = serviceHub.toStateAndRef<MyState>(referenceStateRef)
if (referenceStateAndRef != actualStateAndRefFromVault) {
throw FlowException("Referenced state does not match state in the vault.")
}