I’m trying to cause a ‘SELECT’ query to fail if the record it is trying to read is locked.
To simulate it I have added a trigger on UPDATE that sleeps for 20 seconds and then in one thread (Java application) I’m updating a record (oid=53) and in another thread I’m performing the following query:
“SET STATEMENT max_statement_time=1 FOR SELECT * FROM Jobs j WHERE j.oid =53”.
(Note: Since my mariadb server version is 10.2 I cannot use the “SELECT … NOWAIT” option and must use “SET STATEMENT max_statement_time=1 FOR ….” instead).
I would expect that the SELECT will fail since the record is in a middle of UPDATE and should be read/write locked, but the SELECT succeeds.
Only if I add ‘for update’ to the SELECT query the query fails. (But this is not a good option for me).
I checked the INNODB_LOCKS table during the this time and it was empty.
In the INNODB_TRX table I saw the transaction with isolation level – REPEATABLE READ, but I don’t know if it is relevant here.
Any thoughts, how can I make the SELECT fail without making it 'for update'?
Normally consistent (and dirty) reads are non-locking, they just read some sort of snapshot, depending on what your transaction isolation level is. If you want to make the read wait for concurrent transaction to finish, you need to set isolation level to SERIALIZABLE and turn off autocommit in the connection that performs the read. Something like
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
SET autocommit = 0;
SET STATEMENT max_statement_time=1 FOR ...
should do it.
Relevant page in MariaDB KB
Side note: my personal preference would be to use innodb_lock_wait_timeout=1 instead of max_statement_time=1. Both will make the statement fail, but innodb_lock_wait_timeout will cause an error code more suitable for the situation.
Related
I'm new to SQLite.
Say a row in your table has a column status that is persisting a state-machine. A typical operation on such a row would be to read the current state, and set the next state depending on that, the code having some side-effects.
In a transaction:
val state = db.execute("SELECT state FROM ...")
if (state == "READY_TO_SEND") {
.... do some side-effecting stuff, like sending an email ...
db.execute("UPDATE ... SET state = "MAIL_SENT")
}
In Postgres I could pessimistically lock the row using "SELECT ... FOR UPDATE" to synchronize in this case.
In case of SQLite with WAL mode enabled, if I understood correctly, the read of the current state could happen concurrently by two transactions. Both threads would create the side-effects. Then both transaction would want to upgrade to write. One of them will succeed at an upgrade, the other not, but it's too late to synchronize the side-effects.
Did I understand the SQLite isolation mechanism correctly?
How would you go about synchronization in this case?
Thanks!
I believe that in your example there would be no issue.
If both reads were concurrent and read the same row(s) both would want to update and set the state from "READY_TO_SEND" to "MAIL_SENT". Even if the updates ran concurrently the concurrency would be disturbed as only 1 writer can run at a time. Thus both updates would be applied but sequentially (even though only 1 would be required) and then end result would be the same.
However, since there is only one WAL file, there can only be one writer at a time.
SQLite - Write-Ahead-Logging
Perhaps a bigger issue/side affect could be if the update where UPDATE .... SET state = 'MAIL_SENT' WHERE state = 'READY_TO_SEND'; and the result of the update (number of rows updated) where then used to indicate if the email where sent. The 2nd could be 0 as the WHERE clause would then supress the update. Thus the result from the update might lead to an indication that the email was not sent when it has been sent.
However, this would be a design issue as obviously they are separate emails.
I have a query like this:
UPDATE loginlogs SET rxbytes = rxbytes + ?, txbytes = txbytes + ? WHERE logid = ?
from time to time, my DB gets into a deadlock and SELECT * FROM sys.innodb_lock_waits shows pending locks like this:
wait_started wait_age wait_age_secs locked_table locked_index locked_type waiting_trx_id waiting_trx_started waiting_trx_age waiting_trx_rows_locked waiting_trx_rows_modified waiting_pid waiting_query waiting_lock_id waiting_lock_mode blocking_trx_id blocking_pid blocking_query blocking_lock_id blocking_lock_mode blocking_trx_started blocking_trx_age blocking_trx_rows_locked blocking_trx_rows_modified sql_kill_blocking_query sql_kill_blocking_connection
2020-12-27 07:43:32 00:00:04 4 `db`.`loginlogs` PRIMARY RECORD 37075679818 2020-12-27 07:43:32 00:00:04 1 0 19194139 UPDATE loginlogs SET ... HERE logid = 64634225227638257 37075679818:921:15944673:61 X 37075617021 19191704 UPDATE loginlogs SET ... HERE logid = 64634225227638257 37075617021:921:15944673:61 X 2020-12-27 07:43:07 00:00:29 1 0 KILL QUERY 19191704 KILL 19191704
As you can see, the 2 identical queries seem to run at the same time. And the 2nd one is waiting for the first one to complete.
I thought MySQL should handle simple UPDATE queries like this. Do I need to first select the bytes and then do the UPDATE without referencing rxbytes and txbytes in the new values?
BTW this started happening after update from MariaDB 10.4.2 to 10.4.17, therefore I also suspect MariaDB bug and I opened a bug report.
A lock-wait is not a deadlock!
I see this misunderstanding frequently.
What you have shown is a lock-wait. That is, one transaction is waiting for the other. Every UPDATE locks the rows it examines, and any concurrent UPDATE has to wait for the first one to COMMIT to release those row locks. That's a lock-wait. It's normal and common.
A deadlock is different. It's where two transactions get into a mutual lock-wait. UPDATE1 locks some rows, but does not commit yet. Then UPDATE2 tries to update the same rows, and begins waiting. Then the transaction for UPDATE1 tries another locking statement, that needs locks on some rows already held by the transaction for UPDATE2, possibly from a previous statement it ran. Thus both transactions are waiting for the other, and won't COMMIT to release the locks they hold, because they're waiting. The deadlock is so named because there's no way to resolve the mutual wait.
https://dev.mysql.com/doc/refman/8.0/en/innodb-deadlocks.html says:
A deadlock is a situation where different transactions are unable to proceed because each holds a lock that the other needs. Because both transactions are waiting for a resource to become available, neither ever release the locks it holds.
You won't experience a delay from a true deadlock. MySQL watches for these cyclical lock-waits and forces one of them to rollback its transaction. This happens nearly instantly, so there is virtually no waiting.
So why did this start happening to you when you upgraded from MariaDB 10.4.2 to 10.4.17. Apparently something changed that affects either the number of rows locked, or the duration of the transaction, to make it more likely that you have concurrent transactions conflict in this way.
Or else the software did not change anything related to locking or transactions, but it was your traffic that changed, coincidentally, as you upgraded to the new version of MariaDB.
Working on object locking in IBM DB2 LUW, I am using Read stability (RS) level locking on my table. This locks the records I have fetched in the Select query ,unless I am done with the transaction and mention COMMIT explicitly and release the lock. During this process, with another query/procedure want to return the locked row for a different purpose. How to go about this?Also, there is a parameter WAIT_FOR_OUTCOME to set lock time, where do I define this timeout?
Insufficient information: What is the effective setting for the CUR_COMMIT database? Also, the locking behavior can change with specific Db2-registry settings which you have not mentioned such as DB2_SKIP_INSERTED , DB2_SKIP_DELETED among others etc.
If the effective value of the CUR_COMMIT database parameter is ON, and the other connection is using CS isolation for the query, then the other connection will get the currently committed value of a locked row (i.e. the writer is not blocking the reader).
If you wish you use wait for outcome, it is a clause applicable to a SELECT statement, refer to the documentation.
Use a special register to have session-specific wait controls via
SET CURRENT LOCK TIMEOUT WAIT n ; (where n is the number of seconds), otherwise either a client-side timeout (if relevant), or the database level LOCKTIMEOUT setting will apply, whichever is lower. Note that the default for LOCKTIMEOUT is -1 (wait forever) which is nearly always unsuitable, typical values are 30 or 60 for OLTP.
Other connections can try using UR isolation per statement (e.g. for the query to select a locked row value).
Up until v11.5.6, Db2 LUW did not allow to specify the lock wait timeout on the statement level. It had no SQL statement syntax for that.
With v11.5.6 or later you can do something like this:
SELECT *
FROM some_table
WHERE col1 = 'Foo'
FOR READ ONLY
WITH RS USE AND KEEP EXCLUSIVE LOCKS
WAIT FOR OUTCOME WAIT 5
;
The above will create an exclusive row-level lock on the row(s) returned by the query. The new addition is the WAIT 5 which means to wait 5 seconds for the lock, otherwise throw error.
Before v11.5.6 the lock timeout could only be specified at the session level or at the server level. Both of these are in mao's answer.
For more information:
https://www.ibm.com/docs/en/db2/11.5?topic=statement-concurrent-access-resolution-clause
I am running an END TRANSACTION on my database and occasionally I get error
#1 that "cannot commit - no transaction is active"
Is there a way to determine if a transaction is active before trying a commit? I have been tracking my "BEGIN TRANSACTIONS" by hand but I feel there is a better way.
I am using the C API
You might want to check this:
http://www.sqlite.org/c3ref/get_autocommit.html
According to the page, if you are in a transaction, sqlite3_get_autocommit() will return 0.
The interface you're looking for is actually implemented in this next version of sqlite as you can see in the notes: https://sqlite.org/draft/c3ref/txn_state.html
Determine the transaction state of a database
int sqlite3_txn_state(sqlite3*,const char *zSchema);
The sqlite3_txn_state(D,S) interface returns the current transaction state of schema S in database connection D. If S is NULL, then the highest transaction state of any schema on databse connection D is returned. Transaction states are (in order of lowest to highest):
SQLITE_TXN_NONE
SQLITE_TXN_READ
SQLITE_TXN_WRITE
If the S argument to sqlite3_txn_state(D,S) is not the name of a valid schema, then -1 is returned.
That's weird. I thought sqlite was always in a transaction, either explicitly created by you or implicitly created by sqlite:
http://www.sqlite.org/lang_transaction.html
So I suppose the error means that it's not in a transaction that you initiated ... and if that's what you need to know, it seems OK for sqlite to expect you to keep up with it. Not terribly convenient of course, but I guess that's the cost of a simple API. =/
In SQLite, transactions created using BEGIN TRANSACTION ... END TRANSACTION do not nest.
For nested transactions you need to use the SAVEPOINT and RELEASE commands.
See http://www.sqlite.org/lang_transaction.html
and http://www.sqlite.org/lang_savepoint.html
I would like to find out if it is possible to find out which package or procedure in a package is updating a table?
Due to a certain project being handed over (the person who handed over the project has since left) without proper documentation, data that we know we have updated always go back to some strange source point.
We are guessing that this could be a database job or scheduler that is running the update command without our knowledge. I am hoping that there is a way to find out where the source code is calling from that is updating the table and inserting the source as a trigger on that table that we are monitoring.
Any ideas?
Thanks.
UPDATE: I poked around and found out
how to trace a statement back to its
owning PL/SQL object.
In combination with what Tony mentioned, you can create a logging table and a trigger that looks like this:
CREATE TABLE statement_tracker
( SID NUMBER
, serial# NUMBER
, date_run DATE
, program VARCHAR2(48) null
, module VARCHAR2(48) null
, machine VARCHAR2(64) null
, osuser VARCHAR2(30) null
, sql_text CLOB null
, program_id number
);
CREATE OR REPLACE TRIGGER smb_t_t
AFTER UPDATE
ON smb_test
BEGIN
INSERT
INTO statement_tracker
SELECT ss.SID
, ss.serial#
, sysdate
, ss.program
, ss.module
, ss.machine
, ss.osuser
, sq.sql_fulltext
, sq.program_id
FROM v$session ss
, v$sql sq
WHERE ss.sql_address = sq.address
AND ss.SID = USERENV('sid');
END;
/
In order for the trigger above to compile, you'll need to grant the owner of the trigger these permissions, when logged in as the SYS user:
grant select on V_$SESSION to <user>;
grant select on V_$SQL to <user>;
You will likely want to protect the insert statement in the trigger with some condition that only makes it log when the the change you're interested in is occurring - on my test server this statement runs rather slowly (1 second), so I wouldn't want to be logging all these updates. Of course, in that case, you'd need to change the trigger to be a row-level one so that you could inspect the :new or :old values. If you are really concerned about the overhead of the select, you can change it to not join against v$sql, and instead just save the SQL_ADDRESS column, then schedule a job with DBMS_JOB to go off and update the sql_text column with a second update statement, thereby offloading the update into another session and not blocking your original update.
Unfortunately, this will only tell you half the story. The statement you're going to see logged is going to be the most proximal statement - in this case, an update - even if the original statement executed by the process that initiated it is a stored procedure. This is where the program_id column comes in. If the update statement is part of a procedure or trigger, program_id will point to the object_id of the code in question - you can resolve it thusly:
SELECT * FROM all_objects where object_id = <program_id>;
In the case when the update statement was executed directly from the client, I don't know what program_id represents, but you wouldn't need it - you'd have the name of the executable in the "program" column of statement_tracker. If the update was executed from an anonymous PL/SQL block, I'm not how to track it back - you'll need to experiment further.
It may be, though, that the osuser/machine/program/module information may be enough to get you pointed in the right direction.
If it is a scheduled database job then you can find out what scheduled database jobs exist and look into what they do. Other things you can do are:
look at the dependencies views e.g. ALL_DEPENDENCIES to see what packages/triggers etc. use that table. Depending on the size of your system that may return a lot of objects to trawl through.
Search all the database source code for references to the table like this:
select distinct type, name
from all_source
where lower(text) like lower('%mytable%');
Again that may return a lot of objects, and of course there will be some "false positives" where the search string appears but isn't actually a reference to that table. You could even try something more specific like:
select distinct type, name
from all_source
where lower(text) like lower('%insert into mytable%');
but of course that would miss cases where the command was formatted differently.
Additionally, could there be SQL scripts being run through "cron" jobs on the server?
Just write an "after update" trigger and, in this trigger, log the results of "DBMS_UTILITY.FORMAT_CALL_STACK" in a dedicated table.
The purpose of this function is exactly to give you the complete call stack of al the stored procedures and triggers that have been fired to reach your code.
I am writing from the mobile app, so i can't give you more detailed examples, but if you google for it you'll find many of them.
A quick and dirty option if you're working locally, and are only interested in the first thing that's altering the data, is to throw an error in the trigger instead of logging. That way, you get the usual stack trace and it's a lot less typing and you don't need to create a new table:
AFTER UPDATE ON table_of_interest
BEGIN
RAISE_APPLICATION_ERROR(-20001, 'something changed it');
END;
/