I'm looking for a solution to a design problem. This will take a bit of explaining. I would post code, but that woul make this even longer.
I have a custom generic collection I use to hold Business Objects as needed. For easy of reference, call the business objects BO and the generic collection GC. Inside GC I have a private collection of those business objects that have been flagged for deletion. Call this private collection PDC.
I can have an arbitrary number of GC's, each with their own PDC, at any one time, plus other BOs that aren't in any collection.
When I save changes I loop over all BO and GC and have each one save their changes. This happens wrapped in a TransactionScope so I rollback database changes if anything fails to save properly.
When a GC saves I have a problem with the state of its PDC. The GC first saves all BOs with updates, then deletes records associated with the BOs in the PDC, then clears the PDC of all BOs. I do this so the state of the GC correctly reflects the new state of the database.
Now suppose a BO or GC fails to save after one or more of the GC has successsfully saved . The TransactionScope performs a rollback. The records deleted from the database are restored, but the some/all of the PDC's have been cleared and that state information lost.
So here is my quandry: How do I keep the PDC information around until after the commit has occured, then guarantee the appriopriate collections are cleared?
TransactionScope has no event I can catch to let me know when it the changes have been committed. There are potentially MANY BOs and GC affected by any given transaction, so I can't restrict the transaction to handle one GC at a time.
Any suggestions?
The TransactionStarted is raised by the transaction manager, and the TransactionCompleted by the transaction itself.
You may want the GC to implement IEnlistmentNotification; there is an example that may help you.
You may also want to check DNR TV show 113 and 114.
Related
I am implementing an auction process in Firestore and so far, everything works fine. Each auction item has a deadline and until the time runs out, user are able to bid on the item. The user with the highest bid wins.
However, I wonder how I can handle multiple users bidding at the same time for an item. At the moment I have a collection ("auction_item") which saves the highest bid and the user accordingly. When a user bids in the app, I make sure that the bid is higher than the current highest bid and then save the user and the latest bid.
I fear that multiple users bidding at the same time will cause multiple bids with the same amount in the collection. How do I prevent that in Firestore?
Thank you!
What you're looking for is to support an atomic operation to ensure that the writing of the bid from any of the users is done always with up-to-date and consistent data.
Firestore supports this kind of operation through transactions:
Cloud Firestore supports atomic operations for reading and writing data. In a set of atomic operations, either all of the operations succeed, or none of them are applied.
Transactions: a transaction is a set of read and write operations on one or more documents.
Please note that when using transactions the following applies:
Read operations must come before write operations.
A function calling a transaction (transaction function) might run more than once if a concurrent edit affects a document that the transaction reads.
Transaction functions should not directly modify application state.
Transactions will fail when the client is offline.
Depending on the scenario, a transaction might have a performance impact, as the function might run more than once in case of concurrent edits.
I have around 200 tables in an SQLite database with hundreds to millions rows. These tables are queried by many concurrent processes of an OLTP application.
Each table needs to be occasionally updated by a full refresh -- delete all rows and then insert different ones, all within a transaction. For largest tables this takes almost one minute, but such refresh only happens few times a day for a table.
I need to make sure that the readers do not need to wait for the refresh to finish -- should use old version of table data until the transaction completes, then use the new version. Any waits, if need be, should be in terms of milliseconds rather than seconds or minutes.
Can this be achieved, i.e. avoid any database locks or table locks blocking the readers? I am not concerned about writers, they can wait and serialize.
It is a Python application with pysqlite.
Use WAL mode:
WAL provides more concurrency as readers do not block writers and a writer does not block readers. Reading and writing can proceed concurrently.
A transaction in an Oracle db makes changes in the db, and the changes are committed. Is it possible that other transactions see the performed changes only after several seconds, not immediately?
Background:
We have an application that performs db changes, commits them, and THEN (immediately) it reads the changed data back from the database. However, sometimes it happens that it finds no changes. When the same read is repeated later (by executing the same select manually from SQL Developer), the changed data are returned correctly. The db is standalone, not clustered.
The application does not communicate with the database directly, that would be easy, several layers (including MQ messaging) are involved. We've already eliminated other potential causes of the behaviour (like incorrect parameters, caching etc.). Now I'd like to eliminate an unexpected behaviour of the Oracle db as the cause.
Edit:
At first, I'd like to emphasize that I'm NOT asking whether uncommited changes can be visible to other sessions.
At second, the Oracle COMMIT statement has several modifiers, like WRITE BATCH or NOWAIT. I don't know whether these modifiers can have any influence on the answer to my questions, but we are not using them anyway.
Assuming that your sessions are all using a read committed isolation level, changes would be visible to any query that starts after the data was committed. It was possible in early versions of RAC to have a small delay between when a change was committed on one node and when it was visible on another node but that has been eliminated for a while and you're not using RAC so that's presumably not it.
If your transactions are using the serializable isolation level and the insert happens in a different session than the select, the change would only be visible to other sessions whose transactions began after the change was committed. If sessions A & B both start serializable transactions at time 0, A inserts a row at time 1, and B queries the data at time 2, B would see the state of the data at time 0 and wouldn't see the data that was inserted at time 1 until after it committed its transaction. Note that this would only apply if the two statements are in different sessions-- session A would see the row because it was inserted in A's transaction.
Barring an isolation level issue, I would expect that the SELECT wasn't actually running after the INSERT committed.
I am using Unit of work and Repository pattern along with EF6 in my asp.net web application. DbContext object is getting created and destroyed on every request.
I am thinking that it is costly creating the new dbcontext on every request(I have not done any performance bench marking).
Is this cost of creating DbContext on every request can be ignored ? Does anybody done some bench marking?
Creating a new context is ridiculously cheap, on the order of about 137 ticks on average (0.0000137 seconds) in my application.
Hanging onto a context, on the other hand, can be incredibly expensive, so dispose of it often.
The more objects you query, the more entities end up being tracked in the context. Since entities are POCOS, entity framework has absolutely no way of knowing which ones you've modified except to examine every single one of them in the context and mark it accordingly.
Sure, once they're marked, it will only make database calls for the ones that need updated, but it's determining which ones need updated that is expensive when there are lots of entities being tracked, because it has to check all the POCOS against known values to see if they've changed.
This change tracking when calling save changes is so expensive, that if you're just reading and updating one record at a time, you're better off disposing of the context after each record and creating a new one. The alternative is hanging onto the context, such that every record you read results in a new entity in the context, and every time you call save changes it's slower by one entity.
And yes, it really is slower. If you're updating 10,000 entities for example, loading one at a time into the same context, the first save will only take about 30 ticks, but every subsequent one will take longer to the point where the last one will take over 30,000 ticks. In contrast, creating a new context each time will result in a consistent 30 ticks per update. In the end, because of the cumulative slow-down of hanging onto the context and all the tracked entities, disposing of and recreating the context before each commit ends up taking only 20% as long (1/5 the total time)!
That's why you should really only call save changes once on a context, ever, then dispose of it. If you're calling save changes more than once with a lot of entities in the context, you may not be using it correctly. The exceptional case, obviously, is when you're doing something transactional.
If you need to perform some transactional operation, then you need to manually open your own SqlConnection and either begin a transaction on it, or you need to open it within a TransactionScope. Then, you can create your DbContext by passing it that same open connection. You can do that over and over, disposing of the DbContext object each time while leaving the connection open. Usually, DbContext handles opening and closing the context for you, but if you pass it an open connection, it won't try to close it automatically.
That way, you treat the DbContext as just a helper for tracking object changes on an open connection. You create and destroy it as many times as you like on the same connection, where you can run your transaction. It's very important to understand what's going on under the hood.
Entity Framework is not thread safe, meaning, you cannot use a context across more than one thread. IIS uses a thread for each request sent to the server. Given this, you have to have a context per request. Else, you run a major risks of unexplained and seemingly random exceptions and potentially incorrect data being saved to the database.
Lastly, the context creation is not that expensive of an operation. If you are experiencing a slow application experience (not on first start, but after using the site), your issue probably lies somewhere else.
I have a routine which is updating my business entity. The update involves about 6 different tables. All the commands are being executed within a transaction.
Recently, I needed to add some code into the routine which accesses a lookup table from the database. The lookup code already existed in another business object so I used that business object. For example:
Using tr As DbTransaction = myConnection.BeginTransaction()
ExecuteCommand1(tr)
ExecuteCommand2(tr)
If myLookupTable.GetLookupTable().FindById(id).HasFlagSet Then
ExecuteCommand3(tr)
End If
End Using
However, the lookup table business object hangs/deadlocks. I think this is because it doesn't have a reference to the transaction being used by the original routine.
After doing some research, I attempted to put the lookup table logic in its own transaction, setting the IsolationLevel to ReadUncommitted. This gave me the results I desired. However, after further research, I'm now second-guessing if I've implemented this correctly.
Assuming a reference to the active transaction is unavailable to my lookup table object, is what I've described considered best practice? I feel like I might be missing something.
If you're doing a read in the middle of your transaction then you should do it under the transaction context, not using a different transaction and dirty reads. Luckily there is an easy solution: instead of using the ADO.Net transaction objects use the .Net TransactionScope object. The ADO.Net code is sensible to it and will enlist all your operations in this transaction, including your other business component reads. Just make sure your business object does not open a different connection, this will result in attempting to escalate the existing transaction into a distributed transaction and enlist the new connection into it.
The alternative is to pass down your SqlConnection/SqlTransaction pair on each call, but that propagates horribly ugly everywhere in your code.
If it were me I would rewrite the logic so I do not have to do an uncommitted read.
The golden rule to avoid deadlocks is to always take table locks in the same order in every transaction. So look at the code in the other transactions to see what order they take table locks; then make sure you use the same order in your transaction.
Apparently your look up is attempting to access a row(s) that is exclusively locked by transaction tr. If you use a readuncommitted transaction or alternatively use WITH(NOLOCK) in your lookup query, you will see all uncommitted changes by transactions that might be occurring and effecting your lookup logic. So I am not so sure how desirable this would be.
I think it is best to find a way to ensure that your lookup query participates in the current transaction if you need to do the lookup during that transaction. If all of these operations are to be executed in the same thread, one thing that you can do is to store the transaction object in thread local storage when you create one and have GetLookupTable method check the thread local storage for a transaction object and if there is a transaction set, you can get the connection from that transaction object. Otherwise, you create a new connection. This way your lookup will become part of that transaction and it should run its logic without getting blocked by the current transaction and in turn blocking the current transaction and thus leading to a deadlock.