sqlalchemy: How to make sqlite transactions immediate? - sqlite

sqlite transactions cat be "deferred", "immediate" or "exclusive". The default is "deferred" which means, not starting the transaction until absolutely necessary. This can cause transaction aborts if concurrent transactions start with reading and the proceed to writing. Such aborts can be avoided by using immediate transactions (at the cost of degrading performance).
sqlalchemy abstracts sql dialects including sqlite. It also has a model for writing transactions:
with engine.begin() as connection:
do_something_with_connection
How does one tell sqlalchemy that such a transaction should be immediate. Alternatively, how does one tell sqlalchemy that all sqlite transactions should be immediate?

Core events https://docs.sqlalchemy.org/en/latest/core/events.html can be used to intercept connection events and rewrite the BEGIN statement issued at the start of the transaction to achieve what you want.
See the section of the sqlalchemy documentation on the sqlite dialect for more details https://docs.sqlalchemy.org/en/latest/dialects/sqlite.html.
The example code below is copied directly from the documentation other than for changing BEGIN to BEGIN IMMEDIATE.
from sqlalchemy import create_engine, event
engine = create_engine("sqlite:///myfile.db")
#event.listens_for(engine, "connect")
def do_connect(dbapi_connection, connection_record):
# disable pysqlite's emitting of the BEGIN statement entirely.
# also stops it from emitting COMMIT before any DDL.
dbapi_connection.isolation_level = None
#event.listens_for(engine, "begin")
def do_begin(conn):
# emit our own BEGIN
conn.execute("BEGIN IMMEDIATE")

Related

Python3: Does a call to pymysql.rollback() require a commit?

I'm using python3.6 and pymysql 0.7.11.
The call to rollback rolls a transaction back, according to the documentation.
A call to commit finalises a transaction.
My question is whether after a call to rollback it is prudent to call commit ... ?
this is not really a python question but about mysql transactions.
you can read about it at http://www.mysqltutorial.org/mysql-transaction.aspx for example.
in short: you need to start a transaction to rollback or commit it.
usually autocommit is enabled if you don't want to use transactions manually .. in that case mysql creates a transaction for every statement.

Does transaction assures dirty reads do not happen?

I've been reading a lot lately and I am now very confused how transactions and locking are working and how are related to each other.
When working with SQLite, imagine the following flow:
begin SQLite transaction
run a select statement on a SQLite connection to return values
if condition is met on the returned values, go to step #4, otherwise go to step #5
do the update
commit SQLite transaction
If two threads run same code, is there a possibility that in one of the threads could get what is called a "dirty read" meaning, between the step #3 and step #4, the other thread could run the update (step #4)?
Yes, it's called isolation level: https://www.sqlite.org/isolation.html

SQLite SAVEPOINTS: how does ROLLBACK works in conjunction with RELEASE?

I probably misundertood something about SAVEPOINTS in SQLite.
I am using C API in an application, where we run a bunch of inserts, and in case something fails, we should give up on all the inserts alltogether.
I am aware I could BEGIN TRANSACTION for such a simple scenario, but I fear that the scenario could get a more complicated, and nesting might become a requirement, that's why I went for SAVEPOINTS.
Anyway, here is an extract of SQL statements I run:
SQL> SAVEPOINT SAVEPOINT_20170524_172706;
SQL> INSERT, SELECT STATEMENT (no COMMIT or END TRANSACTION)
SQL> ROLLBACK TO SAVEPOINT SAVEPOINT_20170524_172706;
SQL> RELEASE SAVEPOINT_20170524_172706;
Basically I create a new savepoint based on the timestamp, before I start inserting and selecting data from the database.
Then one operation fails and I need to bail out, so I rollback to the savepoint I just created.
In the end I want to get rid of the savepoint I wont need anymore, since I dont want to clutter the database with useless savepoints, hence I ran RELEASE . In this case I find myself with the database filled with all the data inserted by statements that were supposed to be rolled back.
If I dont execute the RELEASE statement, then the database looks just fine, but I wonder what happens with the abandoned SAVEPOINT which will never be referenced anymore.
Which wrong assumption am I making? What happens to SAVEPOINTS if I dont release them, are they going to be 'dropped' as I close the 'connection' to the DB file?
I had PRAGMA journal_mode = OFF

Spring JDBC: Oracle transaction errors out after 120 seconds

For a particular requirement, I will have to iterate through a list of 50000 records and insert them into database. The requirement is that if any one of the 50000 records fail, all the other records should be rollback. And hence we did not involve any commit in the processing. But this resulted in the following error:
[2/1/16 14:01:47:939 CST] 000000be SystemOut O ERROR
org.springframework.jdbc.UncategorizedSQLException:
PreparedStatementCallback; uncategorized SQLException for SQL [INSERT
INTO ...) VALUES (...)]; SQL state [null]; error code [0]; Current
thread has not commited in more than [120] seconds and may incur
unwanted blocking locks. Please refactor code to commit more
frequently.
Now, when we implemented batching - We are using PreparedStatement.executeBatch() method to insert data in batches of 100, the above error doesn't arise. The autoCommit is set to true by default for the batching, so a commit happens after every batch execution.
Could anyone suggest how we can handle the rollback mechanism in the above case. If 50th batch execution fails, then we want all the previous 49 batch executions to be reverted. We are using Spring Data/JDBC, Oracle 11g database, WebSphere application server. I have read somewhere that the above 120 seconds timeout for commit can also be configured in the JTA settings of WebSphere. Is it so? Please suggest any alternatives or other possible solutions.
Thank you in advance.
You must set autocommit to false and only commit at the end if all your batches executed successfully.

Error handling and RollBack Transaction in SQLITE from SQL Statement

I'm Altering multiple sqlite tables with SQL script by calling ExecuteNonQuery. I want to do this operation in transaction and want to roll it back when anything fails.
I looked at BEGIN TRANSACTION and its clear that I have to call ROLLBACK TRANSACTION when anything goes wrong. But I don't know how could TRY...CATCH (Transact-SQL) kind of thing here.
NOTE: Whole of Sql Script file (which contains many other statements apart from these few statements which needs to be fired in one transaction) is read by .ReadToEnd() and then executed in one go as of now. I want to handle this in sql script file itself and don't want to change the code.
Please take a look at SQLite on conflict clause
Start with BEGIN TRANSACTION
You have to add ON CONFLICT ROLLBACK on your actions
And COMMIT TRANSACTION at the end ;-)

Resources