I am trying to implement a simple counter using SQLite provided with Python. I am using CGI to write simple dynamic web pages. It's the only simple way I can think of to implement a counter. The problem is I need to first read the counter value and then update it. But ideally, every user should read a unique value, and it's possible for two users to read the same counter value if they read simultaneously. Is there a simple way to make the read/write operation atomic? I unfamiliar with SQL so please give specific statements to do so. Thanks in advance.
I use a table with one column and one row to store the counter.
You may try this flow of SQL statements:
BEGIN EXCLUSIVE TRANSACTION;
// update counter here
// retrieve new value for user
COMMIT TRANSACTION;
While you perform updates in trisection, changes can be seen only with connection on which they was performed. In this case we used EXCLUSIVE transactions, which locks database for other clients till it will commit transaction.
You should better not use the EXCLUSIVE keyword in the transaction to make it more efficient. The first select automatically creates a shared lock and the update statement will then turn it into an exclusive. It is only necessary that the SELECT and the UPDATE are both inside an explicit set transaction.
BEGIN TRANSACTION;
// read counter value
// update counter value
COMMIT TRANSACTION;
Related
I am looking to implement the equivalent of snapshot isolation with a Teradata transaction. Oracle supports this type of isolation, but Teradata does not (at least not in versions 14 or prior that I am aware of). The goal is to create a procedure that deletes a table's contents and then repopulates it all while preventing other users from reading from/writing to the table.
I came across the begin request statement which, according to my understanding, allows the optimizer to know about all the various table locks within the request.
I wrote the procedure below, but don't know how to reliably debug it as easy as I would if I were testing thread locking in a .NET application (easy to set breakpoints and monitor other threads). In Teradata, not sure if what I wrote here will properly lock mydb.destinationtable exclusively for the duration of the procedure. Is this correct?
Edit: I'll add that the procedure does work. It's just being able to properly time a SELECT while it's doing its DELETE/INSERT.
replace procedure mydb.myproc()
begin
begin request
locking mydb.destinationtable for exclusive
delete mydb.destinationtable;
locking mydb.destinationtable for exclusive
insert into mydb.destinationtable
select * from mydb.sourcetable;
end request;
end;
BEGIN REQUEST/END REQUEST creates a so-called Multi Statement Request (MSR) which is the same a submitting both requests in SQL Assistant using F9.
To see the plan run this with F9:
EXPLAIN
locking mydb.destinationtable for exclusive
delete mydb.destinationtable;
insert into mydb.destinationtable
select * from mydb.sourcetable;
or in BTEQ:
EXPLAIN
locking mydb.destinationtable for exclusive
delete mydb.destinationtable
;insert into mydb.destinationtable
select * from mydb.sourcetable;
Btw, the 2nd lock is redundant.
But. When you run Delete & InsSel as a single transaction both will be Transient Journalled. Which is much slower than seperate requests.
A more common way to do this is to use two copies of the target table and base access on Views not Tables:
-- no BEGIN/END REQUEST
insert into mydb.destinationtable_2
select * from mydb.sourcetable;
-- there's just a short dictionary lock
-- all requests against the view submitted before the replace use the old data
-- and all submitted after the new data
replace view myview as
select * from mydb.destinationtable_2;
delete from mydb.destinationtable_1;
Now your SP only needs the logic to switch between 1 & 2 (based on table [not] empty)
I am developing an application which fetches some data from a Teradata DWH. DWH developers told me to use LOCK ROW FOR ACCESS before all SELECT queries to avoid delaying writes to that table(s).
Being very familiar with MS SQL Servers's WITH(NOLOCK) hint, I see LOCK ROW FOR ACCESS as its equivalent. However, INSERT or UPDATE statements do not allow using LOCK ROW FOR ACCESS (it is not clear for me why this fails, since it should apply for table(s) the statement selects from, not to the one I insert into):
-- this works
LOCK ROW FOR ACCESS
SELECT Cols
FROM Table
-- this does not work
LOCK ROW FOR ACCESS
INSERT INTO SomeVolatile
SELECT Cols
FROM PersistentTable
I have seen that LOCKING TABLE ... FOR ACCESS can be used, but it is unclear if it fits my need (NOLOCK equivalent - do not block writes).
Question: What hint should I use to minimize writes delaying when selecting within an INSERT statement?
You can't use LOCK ROW FOR ACCESS on an INSERT-SELECT statement. The INSERT statement will put a WRITE lock on the table to which it's writing and a READ lock on the tables from which it's selecting.
If it's absolutely imperative that you get LOCK ROW FOR ACCESS on the INSERT-SELECT, then consider creating a view like:
CREATE VIEW tmpView_PersistentTable AS
LOCK ROW FOR ACCESS
SELECT Cols FROM PersistentTable;
And then perform your INSERT-SELECT from the view:
INSERT INTO SomeVolatile
SELECT Cols FROM tmpView_PersistentTable;
Not a direct answer, but it's always been my understanding that this is one of the reasons your users/applications/etc should access data through views. Views lock for access, which does not prevent inserts/updates. Selecting from a table uses read locks, which will prevent inserts/updates.
The downside is with access locks, the possibility for dirty reads exists.
Change your query as below and you should be good.
LOCKING TABLE PersistentTable FOR ACCESS
INSERT INTO SomeVolatile
SELECT Cols
FROM PersistentTable ;
when looking in symfony2 at the DB Queries I have 18 queries. (I think it is a lot) I am only updating one records.
Anyway. I also saw that some of thoses queries are named 'COMMIT' and 'START TRANSACTION'. Which is not very clear what it does. Any one could explain it to me? thanks.
here an exemple of what I have with Symfony2
Jovan's link is a better read, but in layman's terms, all queries between START TRANSACION and COMMIT are treated as a whole; meaning, if one of those queries fails then they all 'rollback' and fail. START TRANSACTION simply signifies the start of the grouping, and COMMIT finishes it. It's used when you logically need all of the queries to happen.
Say for instance you want to insert a new employee, and they have to have some permissions set. You need to insert into a user table and then a user_roles table.
Now, say you don't use the above, and try running the following two inserts:
INSERT INTO user (...);
INSERT INTO user_roles (...);
What if the first query works, but the second query doesn't? You'll have a 'dangling` user without any permissions, and they'll be sitting in database limbo.
If you instead use a transaction as shown below, you can make sure that either all the queries work, or none of them work.
START TRANSACTION;
INSERT INTO user (...);
INSERT INTO user_roles (...);
COMMIT;
Because database transactions are very serious topic, instead of me trying to explain it here very briefly, I think it would be more beneficial to you to read this wiki article:
Wikipedia: Database Transactions
SQLite allows to define custom functions that can be called from SQL statements. I use this to get notified of trigger activity in my application:
CREATE TRIGGER AfterInsert AFTER INSERT ON T
BEGIN
-- main trigger code involves some extra db modifications
-- ...
-- invoke application callback
SELECT ChangeNotify('T', 'INSERT', NEW.id);
END;
However, user-defined functions are added only to current database connection. There may be other clients who haven't defined ChangeNotify and I don't want them to get a "no such function" error.
Is is possible to call a function only if it's defined? Any alternative solution is also appreciated.
SQLite is designed as an embedded database, so it is assumed that your application controls what is done with the database.
SQLite has no SQL function to check for user-defined functions.
If you want to detect changes made only from your own program, use sqlite3_update_hook.
Prior to calling your user defined function, you can check if the function exists by selecting from pragma_function_list;
select exists(select 1 from pragma_function_list where name='ChangeNotify');
1
It would be possible by combining a query against pragma_function_list and a WHEN statement on the trigger --
CREATE TRIGGER AfterInsert AFTER INSERT ON T
WHEN EXISTS (SELECT 1 FROM pragma_function_list WHERE name = 'ChangeNotify')
BEGIN
SELECT ChangeNotify('T', 'INSERT', NEW.id);
END;
except that query preparation attempts to resolve functions prior to execution. So, afaik, this isn't possible to do in a trigger.
I need to do the same thing and asked here: https://sqlite.org/forum/forumpost/a997b1a01d
Hopefully they come back with a solution.
Update
SQLite forum suggestion is to use create temp trigger when your extension loads -- https://sqlite.org/forum/forumpost/96160a6536e33f71
This is actually a great solution as temp triggers are:
not visible to other connections
are cleaned up when the connection creating them ends
Suppose I have a user table that creates strong relationships (Enforce Foreign Key Constraint) with many additional tables. Such orders table ..
If we try to delete a user with some orders then SqlException will arise.. How can I catch this exception and treat it properly?
Is this strategy at all?
1) first try the delete action if an exception Occur handel it?
2) Or maybe before the delete action using code adapted to ensure that offspring records throughout the database and alert according to .. This piece of work ...
So how to do it?
--Edit:
The goal is not to delete the records from the db! the goal is to inform the user that this record has referencing records. do i need to let sql to execute the delete command and try to catch SqlException? And if so, how to detect that is REFERENCE constraint SqlException?
Or - should I need to write some code that will detect if there are referencing records before the delete command. The last approach give me more but its a lot of pain to implement this kind of verification to each entity..
Thanks
Do you even really want to actually delete User records? Instead I'd suggest having a "deleted" flag in your database, so when you "delete" a user through the UI, all it does is update that record to set the flag to 1. After all, you wouldn't want to delete users that had orders etc.
Then, you just need to support this flag in the appropriate areas (i.e. don't show "deleted" users in the UI).
Edit:
"...but just for the concept, assume that i do want delete the user how do i do that?"
You'd need to delete the records from the other tables that reference that user first, before deleting the user record (i.e. delete the referencing records first then delete the referenced records). But to me that doesn't make sense as you would be deleting e.g. order data.
Edit 2:
"And if so, how to detect that is REFERENCE constraint SqlException?"
To detect this specific error, you can just check the SqlException.Number - I think for this error, you need to check for 547 (this is the error number on SQL 2005). Alternatively, if using SQL 2005 and above, you could handle this error entirely within SQL using the TRY...CATCH support:
BEGIN TRY
DELETE FROM User WHERE UserId = #MyUserId
END TRY
BEGIN CATCH
IF (ERROR_NUMBER() = 547)
BEGIN
-- Foreign key constraint violation. Handle as you wish
END
END CATCH
However, I'd personally perform a pre-check like you suggested though, to save the exception. It's easily done using an EXISTS check like this:
IF NOT EXISTS(SELECT * FROM [Orders] WHERE UserId=#YourUserId)
BEGIN
-- User is not referenced
END
If there are more tables that reference a User, then you'd need to also include those in the check.