what is the difference between Long Running transaction type and None transaction type, set for an orchestration or a scope.
One difference that I know is that Long Running transaction can have compensation block.
A scope with no transaction is usually used as a try/catch block just to handle exceptions. A long running transaction scope is, well, a transaction, which as you point out can have compensation blocks associated with it to "undo" its work.
Most commonly, a long running transaction isn't used in isolation... it will contain nested transactions (both atomic and long running on their own), with the outer transaction coordinating through compensations the undoing of any nested transactions not automatically rolled back.
Related
Hi i am trying to understand if BEGIN EXCLUSIVE, BEGIN IMMEDIATE wait for the duration "busy_timeout", meaning they keep trying to lock the database until that amount of time has passed and only then return a failure of some kind.
I found this documentation for busy_timeout https://www.sqlite.org/c3ref/busy_timeout.html where it mentions that "This routine sets a busy handler that sleeps for a specified amount of time when a table is locked". So this is for table and not for a database?
I basically have different threads doing stuff on the database, so if one thread did a EXCLUSIVE LOCK on the database and a new thread does BEGIN EXCLUSIVE I don't want it fail immediately but keep tyring for a duration of time before throwing error.
Any links to officail documentation which deals with this issue are appreciated.
I’m aware there is a hard limit of 25 items per transaction. However, I’m sure there is a way of implementing transactions for more items from scratch. How might I go about it?
I’m thinking something like, keep a version number on every item. Fetch all the items up front, during insert verify version number is the same. Ie optimistic locking. If the condition fails, revert all failed items. Naturally, I can imagine that the revert could fail and I need to do optimistic locking on the revert and end up in a deadlock of reverts.
The solution I found in the end was to implement pessimistic locking. It supports an arbitrary number of writes and reads and guarantees transactional consistency. The catch is if you're not careful, it's easy to run into deadlocks.
The idea is that you
Create a lock table. Each row refers to a specific lock. The primary key of the lock table should be a string which I'll refer to as the lock-key. Often you'll want to lock a specific entity so this is a reasonable format for the lock-key {table_name}#{primary_key} but it might be more arbitrary so any string will do. Rows in the lock table should also auto-delete after a certain time period as per a ttl field ie TimeToLiveSpecification.
Before starting the transaction, acquire the lock. You do this by creating the row with your arbitrary lock-key and with a conditional check that the row doesn't already exist. If it does exist the row creation should fail which means another process has already acquired the lock. You then need to poll, trying to recreate the lock row until the lock has been released.
Once you have acquired the lock you need to keep the lock alive with a heartbeat to prevent other tasks from executing. The hearbeat process should increment a heartbeat property on the lock row which reflects the last-active time of the lock. The ttl of the row should be greater than the heartbeat interval. Normally about double, so that the lock is not auto-purged erroneously. If your process dies, the lock will be naturally released by the auto-deletion of the ttl.
If your task completes successfully it should delete the lock row freeing it up for other tasks.
I am using typedpolling with stored procedures using ambient transaction. PollingAvailable statement gives "all" rows count. Polling statement gives only a subset of valid rows, and sometimes none.
In the polling procedure I need to be able to exclude some results based on some business logic. Sometimes there are no rows returned from the procedure, which leads to the WCF-SQL adapter doing rollback on the transaction. And, polling process gets looped until at least one valid row is returned.
I don't like to disable ambient transaction because of the "valid" results I'd like to return, if something happens later in the process.
Stored procedure transaction directives (used as my "standard"):
SET XACT_ABORT ON;
SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
Any ideas of a workaround?
Committing an inner transaction I guess will be rolled back anyway?
So, the 'correct' solution is to craft the PollingDataAvailableStatement to return the proper value 0 or >0 depending on the expected output of the polling statement.
I situations where it's mattered, because it doesn't always matter :), I've added a flag to the SP to indicate it's called from PollingDataAvailableStatement so the SP can so some of it's logic to determine if data is available and return that signal.
Of course, you can use a separate SP, but I'd prefer to keep the logic in one place for maintenance reasons.
I'm investigating asynchronous transaction in order to improve performances.
Could you please explain me what is the behaviour of a transaction in a replicated asynchronous cache?
If I have a transaction composed by operations, in which every operation depends by previous operation (i.e. order of execution of operations is important).
For instance, consider a transaction T that performs a read of data1 needed to build data2, data2 is then written on cache.
TRANSACTION T {
// 1° operation
data1 = get(key1);
// 2° operation
data2 = elaborate(data1);
// 3° operation
put(data2);
}
In other words, I need that "entire transactions" are asynchronously executed, but I need that operations performed inside a transaction remain synchronous.
Is it possible? If yes, how I have to configure infinispan?
Many Thanks
Francesco Sclano
I guess you've wrapped those operations into tm.begin(); try { ... tm.commit(); } catch (...) { tm.rollback(); }.
There are few other option, therefore, I assume you use the default - optimistic transactions and 2-phase commit. I'd also recommend enabling write skew check - this should be default in Infinispan 7 but in prior versions you had to enable it explicitly, and some operations behave weird without that.
As for the get, this is always synchronous - you have to wait for the response.
Before commit, the put does basically just another get (in order to return the value) and records that the transaction should do the write during commit.
Then, during commit, the PrepareCommand which locks all updated entries is sent asynchronously - therefore, you can't know whether it succeeded or failed (usually due to timeout, but also due to write skew check or changed value in conditional command).
Second phase, the CommitCommand which overwrites the entries and unlocks the locks is sent synchronously after that - so you wait until the response is received. However, if you have <transaction useSynchronization="true" /> (default), even if this command fails somewhere, the commit succeeds.
In order to send the CommitCommand (or RollbackCommand) asynchronously, you need to configure <transaction syncCommitPhase="false" syncRollbackPhase="false" />.
I hope I've interpreted the sources correctly - and don't ask me why is it done this way. Regrettably, I am not sure whether you can configure the semantics "commit or rollback this transaction reliably, but don't report the result and let me continue" out of the box.
EDIT: The commit in asynchronous mode should be 1-phase as you can't check the result anyway. Therefore, it's possible that concurrent writes get reordered on different nodes and you'll get the cluster inconsistent, but you are not waiting for the transaction to be completed.
Anyway, if you want to execute the whole block atomically and asynchronously, there's nothing easier than to wrap the code into Runnable and execute it in your own threadpool. And with synchronous cache.
Good day, I receive data from a communication channel and display it. Parallel, I serialize it into a SQLite database (using normal SQL INSERT statements). After my application exit I do a .commit on the sqlite object.
What happens if my application is terminated brutally in the middle? Will the latest (reasonably - not say 100 microsec ago, but at least a sec ago) data be safely in the database even without a .commit is made? Or should I have periodic commit? What are best patterns for doing these things?
I tried autocommit on (sqlite's option) and this slows code a lot by a factor ~55 (autocommit vs. just one commit at end). Doing commit every 100 inserts brings performance within 20% of the optimal mode. So autocommit is very slow for me.
My application pumps lots data into DB - what can I do to make it work well?
You should be performing this within a transaction, and consequently performing a commit at appropriate points in the process. A transaction will guarantee that this operation is atomic - that is, it either works or doesn't work.
Atomicity states that database
modifications must follow an “all or
nothing” rule. Each transaction is
said to be “atomic” if when one part
of the transaction fails, the entire
transaction fails. It is critical that
the database management system
maintain the atomic nature of
transactions in spite of any DBMS,
operating system or hardware failure.
If you've not committed, then the inserts won't be visible (and be rolled back) when your process is terminated.
When do you perform these commits ? When your inserts represent something consistent and complete. e.g.. if you have to insert 2 pieces of information for each message, then commit after you've inserted both pieces of info. Don't commit after each one, since your info won't be consistent or complete.
The data is not permanent in the database without a commit. Use an occasional commit to balance the speed of performing many inserts in a transaction (the more frequent the commit, the slower) with the safety of having more frequent commits.
You should do a COMMIT every time you complete a logical change.
One reason for transaction is to prevent uncommitted data from a transaction to be visible from outside. That is important because sometimes a single logical change can translate into multiple INSERT or UPDATE statements. If one of the latter queries of the transaction fails, the transaction can be cancelled with ROLLBACK and no change at all is recorded.
Generally speaking, no change performed in a transaction is recorded in the database until COMMIT succeeds.
does not this slow down considerably my code? – zaharpopov
Frequent commits, might slow down your code, and as an optimization you could try grouping several logical changes in a single transaction. But this is a departure from the correct use of transactions and you should only do this after measuring that this significantly improves performance.