I'm new to SQLite.
Say a row in your table has a column status that is persisting a state-machine. A typical operation on such a row would be to read the current state, and set the next state depending on that, the code having some side-effects.
In a transaction:
val state = db.execute("SELECT state FROM ...")
if (state == "READY_TO_SEND") {
.... do some side-effecting stuff, like sending an email ...
db.execute("UPDATE ... SET state = "MAIL_SENT")
}
In Postgres I could pessimistically lock the row using "SELECT ... FOR UPDATE" to synchronize in this case.
In case of SQLite with WAL mode enabled, if I understood correctly, the read of the current state could happen concurrently by two transactions. Both threads would create the side-effects. Then both transaction would want to upgrade to write. One of them will succeed at an upgrade, the other not, but it's too late to synchronize the side-effects.
Did I understand the SQLite isolation mechanism correctly?
How would you go about synchronization in this case?
Thanks!
I believe that in your example there would be no issue.
If both reads were concurrent and read the same row(s) both would want to update and set the state from "READY_TO_SEND" to "MAIL_SENT". Even if the updates ran concurrently the concurrency would be disturbed as only 1 writer can run at a time. Thus both updates would be applied but sequentially (even though only 1 would be required) and then end result would be the same.
However, since there is only one WAL file, there can only be one writer at a time.
SQLite - Write-Ahead-Logging
Perhaps a bigger issue/side affect could be if the update where UPDATE .... SET state = 'MAIL_SENT' WHERE state = 'READY_TO_SEND'; and the result of the update (number of rows updated) where then used to indicate if the email where sent. The 2nd could be 0 as the WHERE clause would then supress the update. Thus the result from the update might lead to an indication that the email was not sent when it has been sent.
However, this would be a design issue as obviously they are separate emails.
Related
I have a query like this:
UPDATE loginlogs SET rxbytes = rxbytes + ?, txbytes = txbytes + ? WHERE logid = ?
from time to time, my DB gets into a deadlock and SELECT * FROM sys.innodb_lock_waits shows pending locks like this:
wait_started wait_age wait_age_secs locked_table locked_index locked_type waiting_trx_id waiting_trx_started waiting_trx_age waiting_trx_rows_locked waiting_trx_rows_modified waiting_pid waiting_query waiting_lock_id waiting_lock_mode blocking_trx_id blocking_pid blocking_query blocking_lock_id blocking_lock_mode blocking_trx_started blocking_trx_age blocking_trx_rows_locked blocking_trx_rows_modified sql_kill_blocking_query sql_kill_blocking_connection
2020-12-27 07:43:32 00:00:04 4 `db`.`loginlogs` PRIMARY RECORD 37075679818 2020-12-27 07:43:32 00:00:04 1 0 19194139 UPDATE loginlogs SET ... HERE logid = 64634225227638257 37075679818:921:15944673:61 X 37075617021 19191704 UPDATE loginlogs SET ... HERE logid = 64634225227638257 37075617021:921:15944673:61 X 2020-12-27 07:43:07 00:00:29 1 0 KILL QUERY 19191704 KILL 19191704
As you can see, the 2 identical queries seem to run at the same time. And the 2nd one is waiting for the first one to complete.
I thought MySQL should handle simple UPDATE queries like this. Do I need to first select the bytes and then do the UPDATE without referencing rxbytes and txbytes in the new values?
BTW this started happening after update from MariaDB 10.4.2 to 10.4.17, therefore I also suspect MariaDB bug and I opened a bug report.
A lock-wait is not a deadlock!
I see this misunderstanding frequently.
What you have shown is a lock-wait. That is, one transaction is waiting for the other. Every UPDATE locks the rows it examines, and any concurrent UPDATE has to wait for the first one to COMMIT to release those row locks. That's a lock-wait. It's normal and common.
A deadlock is different. It's where two transactions get into a mutual lock-wait. UPDATE1 locks some rows, but does not commit yet. Then UPDATE2 tries to update the same rows, and begins waiting. Then the transaction for UPDATE1 tries another locking statement, that needs locks on some rows already held by the transaction for UPDATE2, possibly from a previous statement it ran. Thus both transactions are waiting for the other, and won't COMMIT to release the locks they hold, because they're waiting. The deadlock is so named because there's no way to resolve the mutual wait.
https://dev.mysql.com/doc/refman/8.0/en/innodb-deadlocks.html says:
A deadlock is a situation where different transactions are unable to proceed because each holds a lock that the other needs. Because both transactions are waiting for a resource to become available, neither ever release the locks it holds.
You won't experience a delay from a true deadlock. MySQL watches for these cyclical lock-waits and forces one of them to rollback its transaction. This happens nearly instantly, so there is virtually no waiting.
So why did this start happening to you when you upgraded from MariaDB 10.4.2 to 10.4.17. Apparently something changed that affects either the number of rows locked, or the duration of the transaction, to make it more likely that you have concurrent transactions conflict in this way.
Or else the software did not change anything related to locking or transactions, but it was your traffic that changed, coincidentally, as you upgraded to the new version of MariaDB.
Working on object locking in IBM DB2 LUW, I am using Read stability (RS) level locking on my table. This locks the records I have fetched in the Select query ,unless I am done with the transaction and mention COMMIT explicitly and release the lock. During this process, with another query/procedure want to return the locked row for a different purpose. How to go about this?Also, there is a parameter WAIT_FOR_OUTCOME to set lock time, where do I define this timeout?
Insufficient information: What is the effective setting for the CUR_COMMIT database? Also, the locking behavior can change with specific Db2-registry settings which you have not mentioned such as DB2_SKIP_INSERTED , DB2_SKIP_DELETED among others etc.
If the effective value of the CUR_COMMIT database parameter is ON, and the other connection is using CS isolation for the query, then the other connection will get the currently committed value of a locked row (i.e. the writer is not blocking the reader).
If you wish you use wait for outcome, it is a clause applicable to a SELECT statement, refer to the documentation.
Use a special register to have session-specific wait controls via
SET CURRENT LOCK TIMEOUT WAIT n ; (where n is the number of seconds), otherwise either a client-side timeout (if relevant), or the database level LOCKTIMEOUT setting will apply, whichever is lower. Note that the default for LOCKTIMEOUT is -1 (wait forever) which is nearly always unsuitable, typical values are 30 or 60 for OLTP.
Other connections can try using UR isolation per statement (e.g. for the query to select a locked row value).
Up until v11.5.6, Db2 LUW did not allow to specify the lock wait timeout on the statement level. It had no SQL statement syntax for that.
With v11.5.6 or later you can do something like this:
SELECT *
FROM some_table
WHERE col1 = 'Foo'
FOR READ ONLY
WITH RS USE AND KEEP EXCLUSIVE LOCKS
WAIT FOR OUTCOME WAIT 5
;
The above will create an exclusive row-level lock on the row(s) returned by the query. The new addition is the WAIT 5 which means to wait 5 seconds for the lock, otherwise throw error.
Before v11.5.6 the lock timeout could only be specified at the session level or at the server level. Both of these are in mao's answer.
For more information:
https://www.ibm.com/docs/en/db2/11.5?topic=statement-concurrent-access-resolution-clause
I’m trying to cause a ‘SELECT’ query to fail if the record it is trying to read is locked.
To simulate it I have added a trigger on UPDATE that sleeps for 20 seconds and then in one thread (Java application) I’m updating a record (oid=53) and in another thread I’m performing the following query:
“SET STATEMENT max_statement_time=1 FOR SELECT * FROM Jobs j WHERE j.oid =53”.
(Note: Since my mariadb server version is 10.2 I cannot use the “SELECT … NOWAIT” option and must use “SET STATEMENT max_statement_time=1 FOR ….” instead).
I would expect that the SELECT will fail since the record is in a middle of UPDATE and should be read/write locked, but the SELECT succeeds.
Only if I add ‘for update’ to the SELECT query the query fails. (But this is not a good option for me).
I checked the INNODB_LOCKS table during the this time and it was empty.
In the INNODB_TRX table I saw the transaction with isolation level – REPEATABLE READ, but I don’t know if it is relevant here.
Any thoughts, how can I make the SELECT fail without making it 'for update'?
Normally consistent (and dirty) reads are non-locking, they just read some sort of snapshot, depending on what your transaction isolation level is. If you want to make the read wait for concurrent transaction to finish, you need to set isolation level to SERIALIZABLE and turn off autocommit in the connection that performs the read. Something like
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
SET autocommit = 0;
SET STATEMENT max_statement_time=1 FOR ...
should do it.
Relevant page in MariaDB KB
Side note: my personal preference would be to use innodb_lock_wait_timeout=1 instead of max_statement_time=1. Both will make the statement fail, but innodb_lock_wait_timeout will cause an error code more suitable for the situation.
I am trying to implement a simple counter using SQLite provided with Python. I am using CGI to write simple dynamic web pages. It's the only simple way I can think of to implement a counter. The problem is I need to first read the counter value and then update it. But ideally, every user should read a unique value, and it's possible for two users to read the same counter value if they read simultaneously. Is there a simple way to make the read/write operation atomic? I unfamiliar with SQL so please give specific statements to do so. Thanks in advance.
I use a table with one column and one row to store the counter.
You may try this flow of SQL statements:
BEGIN EXCLUSIVE TRANSACTION;
// update counter here
// retrieve new value for user
COMMIT TRANSACTION;
While you perform updates in trisection, changes can be seen only with connection on which they was performed. In this case we used EXCLUSIVE transactions, which locks database for other clients till it will commit transaction.
You should better not use the EXCLUSIVE keyword in the transaction to make it more efficient. The first select automatically creates a shared lock and the update statement will then turn it into an exclusive. It is only necessary that the SELECT and the UPDATE are both inside an explicit set transaction.
BEGIN TRANSACTION;
// read counter value
// update counter value
COMMIT TRANSACTION;
I am running an END TRANSACTION on my database and occasionally I get error
#1 that "cannot commit - no transaction is active"
Is there a way to determine if a transaction is active before trying a commit? I have been tracking my "BEGIN TRANSACTIONS" by hand but I feel there is a better way.
I am using the C API
You might want to check this:
http://www.sqlite.org/c3ref/get_autocommit.html
According to the page, if you are in a transaction, sqlite3_get_autocommit() will return 0.
The interface you're looking for is actually implemented in this next version of sqlite as you can see in the notes: https://sqlite.org/draft/c3ref/txn_state.html
Determine the transaction state of a database
int sqlite3_txn_state(sqlite3*,const char *zSchema);
The sqlite3_txn_state(D,S) interface returns the current transaction state of schema S in database connection D. If S is NULL, then the highest transaction state of any schema on databse connection D is returned. Transaction states are (in order of lowest to highest):
SQLITE_TXN_NONE
SQLITE_TXN_READ
SQLITE_TXN_WRITE
If the S argument to sqlite3_txn_state(D,S) is not the name of a valid schema, then -1 is returned.
That's weird. I thought sqlite was always in a transaction, either explicitly created by you or implicitly created by sqlite:
http://www.sqlite.org/lang_transaction.html
So I suppose the error means that it's not in a transaction that you initiated ... and if that's what you need to know, it seems OK for sqlite to expect you to keep up with it. Not terribly convenient of course, but I guess that's the cost of a simple API. =/
In SQLite, transactions created using BEGIN TRANSACTION ... END TRANSACTION do not nest.
For nested transactions you need to use the SAVEPOINT and RELEASE commands.
See http://www.sqlite.org/lang_transaction.html
and http://www.sqlite.org/lang_savepoint.html