When we create a key in objectify from a web safe string, is Objectify hitting the datastore in anyway. For example when we do ...
Key.create(webSafeString):
Is Objectify using some idempotent algorithm to create the key, or is it dependent on what is stored in the datastore, or what entities have been registered? What exceptions can we expect to be thrown?
Also, if it is reading the datastore, what is the cost?
No. Creating a Key from a websafe string is just protobuf manipulation, and should not involve an RPC to Google's service layer.
Related
I am new to Rebus and am trying to get up to speed with some patterns we currently use in Azure Logic Apps. The current target implementation would use Azure Service Bus with Saga storage preferably in Cosmos DB (still investigating that sample implementation). Maybe even use Rebus Mongo DB with Cosmos DB using the Mongo DB API (not sure if that is possible though).
One major use case we have is an event/timeout pattern, and after doing some reading of samples/forums/Stack Overflow this is not uncommon. The tricky part is that our Sagas would behave more as a Finite State Machine vs. a Directed Acyclic Graph. This mainly happens because dates are externally changed and therefore timeouts for events change.
The Defer() method does not return a timeout identifier, which we assume is an implementation restriction (Azure Service Bus returns a long). Since we must ignore timeouts that had been scheduled for an event which has now shifted in time, we see a way of having those timeouts "ignored" (since they cannot be cancelled) as follows:
Use a Dictionary<string, Guid> in our own SagaData-derived base class, where the key is some derivative of the timeout message type, and the Guid is the identifier given to the timeout message when it was created. I don't believe this needs to be a concurrent dictionary but that is why I am here...
On receipt of the event message, remove the corresponding timeout message type key from the above dictionary;
On receipt of the timeout message:
Ignore if it's timeout message type key is not present or the Guid does not match the dictionary key/value; else
Process. We could also remove the dictionary key at this point as well.
When event rescheduling occurs, simply add the timeout message type/Guid dictionary entry, or update the Guid with the new timeout message Guid.
Is this on the right track, or is there a more 'correct' way of handling defunct timeout (deferred) messages?
You are on the right track 🙂
I don't believe this needs to be a concurrent dictionary but that is why I am here...
Rebus lets your saga handler work on its own copy of the saga data (using optimistic concurrency), so you're free to model the saga data as if it's being only being accessed by one at a time.
What happens if a transaction fails and the application crashes for other reasons and the transaction is not rolled back?
Also, what happens and how should rollback failures be treated?
You don't have to worry about the impact of your app's crashes on transaction rollbacks (or any other stateful datastore operation).
The application just sends RPC requests for the operations. The actual operation steps/sequence execution, happens on the datastore backend side, not inside your application.
From Life of a Datastore Write:
We'll dive into a bit more detail in terms of what new data is placed
in the datastore as part of write operations such as inserts,
deletions, updates, and transactions. The focus is on the backend work
that is common to all of the runtimes.
...
When we call put or makePersistent, several things happen behind
the scenes before the call returns and sets the entity's key:
The my_todo object is converted into a protocol buffer.
The appserver makes an RPC call to the datastore server, sending the entity data in a protocol buffer.
If a key name is not provided, a unique ID is determined for this entity's key. The entity key is composed of app ID | ancestor keys |
kind name | key name or ID.
The datastore server processes the request in two phases that are executed in order: commit, then apply. In each phase, the datastore
server identifies the Bigtable tablet servers that should receive
the data.
Now, depending on the client library you use, transaction rollback could be entirely automatic (in the ndb python client library, for example) or could be your app's responsibility. But even if it is your app's responsibility, it's a best-effort attempt anyways. Crashing without requesting a rollback would simply mean that some potentially pending operations on the backend side will eventually time out instead of being actively ended. See also related GAE: How to rollback a transaction?
Does creating DbContext per query in Asp.net make EF only read the data from its cache, or does it query DB for the whole sets every time? I know about metadata caching per AppDomain, but what about just the data?
Context: data acquisition and visualisation application with MVC4 + Web API frontend, wouldn't call that "high volume", but lots of queries return the same sets of data in some shorter time frame.
Entity Framework doesn't have a data cache per AppDomain, only a cache per context instance.
If you create a new context per request or query you start with an empty cache and EF will fetch the data from the database.
Moreover, the term "cache per context instance" can be misleading as it doesn't mean that EF won't run queries to the database if the entities are already loaded in the context cache. The way how this cache works and how you can leverage it (or not) is the following:
Every LINQ-to-Entities query on a DbSet<T> or generally on an IQueryable<T> will run a database query, no matter if the entities already exist in the context or not. But if an entity with the same key as a queried entity already exists in the context EF will throw the result of that query away and return the cached entity instance back to the caller.
It does this check if the entity with the same key exists after it has run the query. (For complex queries - for example queries that contain an Include - it can't do this check before because it cannot know which entities and key values will be returned.)
That's the default behaviour (MergeOption is AppendOnly). You can change this behaviour to OverwriteChanges and other options, I believe, but none of them will avoid that LINQ queries always issue database queries.
For querying an entity just by its key you can use GetObjectByKey or Find (with DbContext) which will check first if the entity with that key is already cached in the context and then return this cached object. If not it will run a database query to load it.
You can query EF's ChangeTracker, it's especially well supported with DbContext where you have access to the context cache via the DbSet<T>.Local collection.
The problem here is that there is no logic to query the database automatically if a query on Local does not return a result. You have to write this logic manually. The even bigger problem is that a query on Local is LINQ-to-Objects and not LINQ-to-Entities (Local doesn't implement IQueryable<T>, only IEnumerable<T>), so you often have to rewrite your queries to act on Local - for example you can't use Include here, your can't use any EntityFunctions, you will get different behaviour for string comparisons regarding case sensitivity, etc., etc.
In reading an article on N-Tiered Applications, I came across information regarding concurrency tokens and change tracking information:
Another important concept to understand is that while the
default-generated entities support serialization, their
change-tracking information is stored in the ObjectStateManager (a
part of the ObjectContext), which does not support serialization.
My question is three-fold:
Is there the same thing when using DbContext?
If the only interaction with the database is in a Repository class within a using statement, does closing the database connection when the program leaves the using statement get rid of any option for change tracking?
Can this be leveraged as/with a Concurrency Token?
Yes. DbContext is just wrapper around ObjectContext and it exposes change tracking information through ChangeTracker property (returns DbChangeTracker) and for particular entity through calling Entry method (returns DbEntityEntry<T>).
Yes. Closing context will remove all change tracking information.
Concurrency token and change tracking are two completely different concepts. Change tracking tells context what operations it has to execute on database when you call SaveChanges. It tracks changes you did on your entities since you loaded them into current context instance. Concurrency token resolves optimistic concurrency in the database => it validates that another process / thread / user / context instance didn't change the same record your context is going to modify during SaveChanges.
I am using LINQ object in my webservice and consumed by iphone and android developers.
Now i have added some new fields in a table which in turn will be used linq object.
I want to know some way by which i can mark some linq properties as optional i.e. if iphone and android deve. doesnt pass that properties then no error will show up.
Thanks
Gaurav
First, it is not recommended to embed your LINQ objects directly in a web service. Create data objects specifically for the web service that captures the relevant parameters for each web service call. This will protect your service consumers from changes to the underlying data storage and allow you to more tightly control what is known to the consumers. You can then use the adapter pattern to convert objects in your web service to LINQ objects for submitting to database.
Secondly, regarding your question about optional parameters. With custom data objects, this will be easily done with nullable properties (for primitive types), or just null for object types. If you must use the LINQ objects, making the fields nullable in the database will also cause the properties to be nullable, and thus optional to service consumers.