I've been reading a lot lately and I am now very confused how transactions and locking are working and how are related to each other.
When working with SQLite, imagine the following flow:
begin SQLite transaction
run a select statement on a SQLite connection to return values
if condition is met on the returned values, go to step #4, otherwise go to step #5
do the update
commit SQLite transaction
If two threads run same code, is there a possibility that in one of the threads could get what is called a "dirty read" meaning, between the step #3 and step #4, the other thread could run the update (step #4)?
Yes, it's called isolation level: https://www.sqlite.org/isolation.html
Related
I probably misundertood something about SAVEPOINTS in SQLite.
I am using C API in an application, where we run a bunch of inserts, and in case something fails, we should give up on all the inserts alltogether.
I am aware I could BEGIN TRANSACTION for such a simple scenario, but I fear that the scenario could get a more complicated, and nesting might become a requirement, that's why I went for SAVEPOINTS.
Anyway, here is an extract of SQL statements I run:
SQL> SAVEPOINT SAVEPOINT_20170524_172706;
SQL> INSERT, SELECT STATEMENT (no COMMIT or END TRANSACTION)
SQL> ROLLBACK TO SAVEPOINT SAVEPOINT_20170524_172706;
SQL> RELEASE SAVEPOINT_20170524_172706;
Basically I create a new savepoint based on the timestamp, before I start inserting and selecting data from the database.
Then one operation fails and I need to bail out, so I rollback to the savepoint I just created.
In the end I want to get rid of the savepoint I wont need anymore, since I dont want to clutter the database with useless savepoints, hence I ran RELEASE . In this case I find myself with the database filled with all the data inserted by statements that were supposed to be rolled back.
If I dont execute the RELEASE statement, then the database looks just fine, but I wonder what happens with the abandoned SAVEPOINT which will never be referenced anymore.
Which wrong assumption am I making? What happens to SAVEPOINTS if I dont release them, are they going to be 'dropped' as I close the 'connection' to the DB file?
I had PRAGMA journal_mode = OFF
On the linux server, where our web app runs, we also have a small app that uses
sqlite (it is written in c).
For performing database actions we use the following commands:
sqlite3_prepare_v2
sqlite3_bind_text or sqlite3_bind_int
sqlite3_step
sqlite3_finalize
Every now and then there was a concurrency situation and I got the following error:
database is locked
So I thought: "This happens when one process writes a certain record and the
other one is trying to read exactly the same record."
So after every step-command, where this collision could be, I checked for this error. When it happended, I waited a few milliseconds and the tried again.
But the sqlite error "database is locked" still occurred.
So I changed every step command and the code lines after it. Somehow I thought that this "database is locked" error could only occur with the step command.
But the error kept coming.
My question is now:
Do I have to check after any sqlite3 command for "error_code ==5" (database is locked)?
Thanks alot in advance
If you're receiving error code 5 (busy) you can limit this by using an immediate transaction. If you're able to begin an immediate transaction, SQLite guarantees that you won't receive a busy error until you commit.
Also note that SQLite doesn't have row-level locking. The entire database is locked. Using a WAL journal, you can one writer and multiple readers. With other journaling methods, you can have either one writer, or multiple readers, but not both simultaneously.
SQLite Documentation on 'SQLITE_BUSY'
We have a new beta version of our software with some changes, but not around our database layer.
We've just started getting Error 3128 reported in our server logs. It seems that once it happens, it happens for as long as the app is open. The part of the code where it is most apparent is where we log data every second via SQLite. We've generated 47k errors on our server this month alone.
3128 Disk I/O error occurred. Indicates that an operation could not be completed because of a disk I/O error. This can happen if the runtime is attempting to delete a temporary file and another program (such as a virus protection application) is holding a lock on the file. This can also happen if the runtime is attempting to write data to a file and the data can't be written.
I don't know what could be causing this error. Maybe an anti-virus program? Maybe our app is getting confused and writing data on top of each other? We're using async connections.
It's causing lots of issues and we're at a loss. It has happened in our older version, but maybe 100 times in a month rather than 47,000 times. Either way I'd like to make it happen "0" times.
Possible solution: Exception Message: Some kind of disk I/O error occurred
Summary: There is probably not a problem with the database but a problem creating (or deleting) the temporary file once the database is opened. AIR may have permissions to the database, but not to create or delete files in the directory.
One answer that has worked for me is to use the PRAGMA statement to set the journal_mode value to something other than DELETE. You do this by issuing a PRAGMA statement in the same way you would issue a query statement.
PRAGMA journal_mode = OFF
Unfortunately, if the application crashes in the middle of a transaction when the OFF journaling mode is set, then the database file will very likely go corrupt.1.
1 http://www.sqlite.org/pragma.html#pragma_journal_mode
The solution was to make sure database delete, update, insert only happened one at at time by wrapping a little wrapper. On top of that, we had to watch for error 3128 and retry. I think this is because we have a trigger running that could lock the database after we inserted data.
In my (test) application I have multiple threads where each thread makes multiple SELECT calls one after another. Each thread has its own connection to the (same) SQLite database.
Everything works fine for a couple of calls but then I start getting SQLITE_BUSY errors ("database is locked") back from sqlite3_step().
I know I can circumvent this with sqlite3_busy_timeout() but I want to know why this is happening or whether this is expected behavior.
Please note that I'm only doing read operations (unless SELECT is considered a write operation nowadays), so SQLite should only acquire a shared lock, shouldn't it?
I'm on Windows (7 x64) and running SQLite 3.7.13.
I'm Altering multiple sqlite tables with SQL script by calling ExecuteNonQuery. I want to do this operation in transaction and want to roll it back when anything fails.
I looked at BEGIN TRANSACTION and its clear that I have to call ROLLBACK TRANSACTION when anything goes wrong. But I don't know how could TRY...CATCH (Transact-SQL) kind of thing here.
NOTE: Whole of Sql Script file (which contains many other statements apart from these few statements which needs to be fired in one transaction) is read by .ReadToEnd() and then executed in one go as of now. I want to handle this in sql script file itself and don't want to change the code.
Please take a look at SQLite on conflict clause
Start with BEGIN TRANSACTION
You have to add ON CONFLICT ROLLBACK on your actions
And COMMIT TRANSACTION at the end ;-)