Let me explain this a bit further.
I have a table in sqlite that looks something like this:
table
------
id numeric primary key,
uuid text not null,
other_field text
Now id is the standard auto-increment type. This table can have entries added and removed at any time so if a rowid that had been used once before was used again, that'd be fine. I'm not using that 'table full' feature of sqlite. There are allowed to be multiple entries with the same uuid. The idea is I'm only interested in the last inserted entry in general.
This raises the question. I know I can do a call like
"select other_field from table where uuid=? order by rowid desc"
This would be ok, but what if rowid wraps around? order by rowid desc will not give me the newest entry.
All I can think of is to add a creation_time field like
table
------
id numeric primary key,
uuid text not null,
other_field text
creation_time datetime
and then when it gets created put datetime('now') in that field.
select other_field from table where uuid=? order by creation_time desc
But this means adding an extra field and a bigger index. Is there a built in way to do this?
If you want to select the newest ID or RowID, have a look into the command LIMIT. With the LIMIT you can return only 1 row. For your example:
SELECT * from table ORDER BY id DESC LIMIT 1
With this solution, you don't have to add another field "creation_time".
Hops this helps :)
Related
I have two SQLite files, each of them has one table and the same table design. One Column is set as Primary Key. I want to copy all data from ItemsB into ItemsA. All data should be updated. The ItemsB Table is the newer one.
I've tried:
ATTACH DATABASE ItemsB AS ItemsB;
INSERT INTO ItemsA.PMItem (ItemID,VarID,Name1) SELECT ItemID,VarID,Name1 FROM ItemsB.PMItem;
Obviously this can't work due the Primary Key (which is the column VarID).
Then I tried it with ON CONFLICT:
ON CONFLICT (VarID) DO UPDATE SET Name1=excluded.Name1
But this won't work either.
Example Table:
CREATE TABLE PMItem (
ItemID INTEGER,
VarID INTEGER PRIMARY KEY,
Name1 TEXT
);
You need a WHERE clause with an always true condition, to overcome the ambiguity that is raised when ON CONFLICT is used after a SELECT statement:
INSERT INTO PMItem (ItemID,VarID,Name1)
SELECT ItemID,VarID,Name1
FROM ItemsB.PMItem
WHERE 1
ON CONFLICT(VarID) DO UPDATE
SET Name1 = EXCLUDED.Name1;
I have a table indexed on a text column, and I want all my queries to return results ordered by name without any performance hit.
Table has around 1 million rows if it matters.
Table -
CREATE TABLE table (Name text)
Index -
CREATE INDEX "NameIndex" ON "Files" (
"Name" COLLATE nocase ASC
);
Query 1 -
select * from table where Name like "%a%"
Query plan, as expected a full scan -
SCAN TABLE table
Time -
Result: 179202 rows returned in 53ms
Query 2, now using order by to read from index -
select * from table where Name like "%a%" order by Name collate nocase
Query plan, scan using index -
SCAN TABLE table USING INDEX NameIndex
Time -
Result: 179202 rows returned in 672ms
Used DB Browser for SQLite to get the information above, with default Pragmas.
I'd assume scanning the index would be as performant as scanning the table, is it not the case or am I doing something wrong?
Another interesting thing I noticed, that may be relevant -
Query 3 -
select * from table where Name like "a%"
Result: 23026 rows returned in 9ms
Query 4 -
select * from table where name like "a%" order by name collate nocase
Result: 23026 rows returned in 101ms
And both has them same query plan -
SEARCH TABLE table USING INDEX NameIndex (Name>? AND Name<?)
Is this expected? I'd assume the performance be the same if the plan was the same.
Thanks!
EDIT - The reason the query is slower was because I used select * and not select name, causing SQLite to go between the table and the index.
The solution was to use clustered index, thanks #Tomalak for helping me find it -
create table mytable (a text, b text, primary key (a,b)) without rowid
The table will be ordered by default using a + b combination, meaning that full scan queries will be much faster (now 90ms).
A LIKE pattern that starts with % can never use an index. It will always result in a full table scan (or index scan, if the query can be covered by the index itself).
It's logical when you think about it. Indexes are not magic. They are sorted lists of values, exactly like a keyword index in a book, and that means they are only only quick for looking up a word if you know how the given word starts. If you're searching for the middle part of a word, you would have to look at every index entry in a book as well.
Conclusion from the ensuing discussion in the comments:
The best course of action to get a table that always sorts by a non-unique column without a performance penalty is to create it without ROWID, and turn it into a clustering index over a the column in question plus a second column that makes the combination unique:
CREATE TABLE MyTable (
Name TEXT COLLATE NOCASE,
Id INTEGER,
Other TEXT,
Stuff INTEGER,
PRIMARY KEY(Name, Id) -- this will sort the whole table by Name
) WITHOUT ROWID;
This will result in a performance penalty for INSERT/UPDATE/DELETE operations, but in exchange sorting will be free since the table is already ordered.
I'm having a little trouble getting my head around this statement. The idea is it's meant to initialize a table with a single row of values for each player in the database, but I can't figure out from a browser full of search tabs what I'm doing wrong. All I know is apparently my syntax is rubbish.
INSERT INTO tblKebabs
(TransactionID, PlayerID, Amount, Description, Timestamp)
SELECT
(COUNT(tblPlayers.PlayerID)) AS TransactionID,
tblPlayers.PlayerID AS PlayerID,0 as Amount,
"Initializer" as Description,"now" AS Timestamp)
FROM tblPlayers
WHERE tblPlayers.PlayerID > 0;
If you want the TransactionID column to be AUTOINCREMENT you have to define it in the CREATE statement.
If you already have defined it as INTEGER PRIMARY KEY it is already AUTOINCREMENT and you don't need to change something.
If you have nothing of the above then you have to recreate the table with INTEGER PRIMARY KEY for this column because SQLite does not allow such changes with ALTER. Now you can omit this column from your statement:
INSERT INTO tblKebabs
(PlayerID, Amount, Description, Timestamp)
SELECT
PlayerID,
0 as Amount,
'Initializer',
CURRENT_TIMESTAMP
FROM tblPlayers
WHERE PlayerID > 0;
You don't need aliases in the SELECT statement.
Also I used CURRENT_TIMESTAMP.
I'm not sure what you're trying to do here but if it's just about the count, try this
ROW_NUMBER() OVER(ORDER BY tblPlayers.PlayerID) AS TransactionID
I have got 3 columns in the table, I need to check the email field before insertion of new row, so if the email exist, it should not insert that row. Do I have to go through 2 queries to achieve that, so for instance first check :
Select count(*) FROM PRODUCT WHERE email = #email
AND THEN RUN AN INSERT
Insert INTO PRODUCT (....) VALUES (....)
Or is there any better way to achieve that ,
Any suggestions or advice will be appreciated .
Thanks
You can have an INSERT statement that checks for the condition, and only inserts something, if it doesn't already exist:
IF NOT EXISTS (SELECT * FROM dbo.Product WHERE email = #email)
INSERT INTO dbo.Product(list of columns)
VALUES(list of values)
As marc_s mentioned you may use a conditional insert into. On the other hand, using a unique constraint on your email column may be very helpful too!
Just think of the possibility to insert data without your application. There would be no rule to avoid the same email!
CREATE TABLE Product
(
xxx int NOT NULL,
xxx xxx xxx,
UNIQUE (email)
)
Just google for alter/create table(s) with unique constraints!
I have a table with unique usernames and a bunch of string data I am keeping track of. Each user will have 1000 rows and when I select them I want to return them in the order they were added. Is the following code a necessary and correct way of doing this:
CREATE TABLE foo (
username TEXT PRIMARY KEY,
col1 TEXT,
col2 TEXT,
...
order_id INTEGER NOT NULL
);
CREATE INDEX foo_order_index ON foo(order_id);
SELECT * FROM foo where username = 'bar' ORDER BY order_id;
Add a DateAdded field and default it to the date/time the row was added and sort on that.
If you absolutely must use the order_ID, which I don't suggest. Then at least make it an identity column. The reason I advise against this is because you are relying on side affects to do your sorting and it will make your code harder to read.
If each user will have 1000 rows, then username should not be the primary key. One option is to use the int identity column which all tables have (which optimizes I/O reads since it's typically stored in that order).
Read under "RowIds and the Integer Primary Key" # http://www.sqlite.org/lang_createtable.html
The data for each table in SQLite is stored as a B-Tree structure
containing an entry for each table row, using the rowid value as the
key. This means that retrieving or sorting records by rowid is fast.
Because it's stored in that order in the B-tree structure, it should be fast to order by the int primary key. Make sure it's an alias for rowid though - more in that article.
Also, if you're going to be doing queries where username = 'bob', you should consider an index on the username column - especially there's going to be many users which makes the index effective because of high selectivity. In contrast, adding an index on a column with values like 1 and 0 only leads to low selectivity and renders the index very ineffective. So, if you have 3 users :) it's not worth it.
You can remove the order_id column & index entirely (unless you need them for something other than this sorting).
SQLite tables always have a integer primary key - in this case, your username column has silently been made a unique key, so the table only has the one integer primary key. The key column is called rowid. For your sorting purpose, you'll want to explicitly make it AUTOINCREMENT so that every row always has a higher rowid than older rows.
You probably want to read http://www.sqlite.org/autoinc.html
CREATE TABLE foo (
rowid INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT UNIQUE KEY,
...
Then your select becomes
select * from foo order by rowed;
One advantage of this approach is that you're re-using the index SQLite will already be placing on your table. A date or order_id column is going to mean an extra index, which is just overhead here.