In Sqlite, I defined a view as a union all of two tables. When I run queries using that view, the index is used if the query is simple enough. For certain complex queries, it does not and ends up running full table scans. Are there ways around this so that I can use views performantly?
Table/View Definitions:
CREATE TABLE 'Table1' (Id varchar (18) PRIMARY KEY UNIQUE ON CONFLICT ROLLBACK, Name varchar (255) )
CREATE TABLE 'Table2' (Id varchar (18) PRIMARY KEY UNIQUE ON CONFLICT ROLLBACK, Name varchar (255) )
CREATE TABLE 'Table3' (Id varchar (18) PRIMARY KEY UNIQUE ON CONFLICT ROLLBACK, Name varchar (255) )
CREATE VIEW [UnionView] AS SELECT 'T1' tid, T1.rowid, T1.* FROM [Table1] T1 UNION ALL SELECT 'T2' tid, T2.rowid, T2.* FROM [Table2] T2
Simple Query (indexes are used):
SELECT Id FROM [UnionView] WHERE Id = 'asdf'
Explain Query Plan:
COMPOUND QUERY
LEFT-MOST SUBQUERY
SEARCH TABLE Table1 AS T1 USING INDEX sqlite_autoindex_Table1_1 (Id=?)
UNION ALL
SEARCH TABLE Table2 AS T2 USING INDEX sqlite_autoindex_Table2_1 (Id=?)
LEFT JOIN Query (indexes are not used):
SELECT T3.Id FROM [Table3] T3 LEFT JOIN [UnionView] T ON T3.Id=T.Id WHERE T3.Id = 'asdf'
Explain Query Plan
MATERIALIZE 2
COMPOUND QUERY
LEFT-MOST SUBQUERY
SCAN TABLE Table1 AS T1
UNION ALL
SCAN TABLE Table2 AS T2
SEARCH TABLE Table3 AS T3 USING COVERING INDEX sqlite_autoindex_Table3_1 (Id=?)
SCAN SUBQUERY 2 AS T
Your complex query does full table scans of Table1 and Table2 because you are not doing any filtering on UnionView.
It does use though sqlite_autoindex_Table3_1.
Also, the WHERE clause is applied after the joins.
If you filter UnionView before the join then indexes will be used:
EXPLAIN QUERY PLAN
SELECT T3.Id
FROM [Table3] T3
LEFT JOIN (SELECT Id FROM [UnionView] WHERE Id = 'asdf') T
ON T3.Id=T.Id
WHERE T3.Id = 'asdf'
Result:
MATERIALIZE 3
COMPOUND QUERY
LEFT-MOST SUBQUERY
SEARCH TABLE Table1 AS T1 USING INDEX sqlite_autoindex_Table1_1 (Id=?)
UNION ALL
SEARCH TABLE Table2 AS T2 USING INDEX sqlite_autoindex_Table2_1 (Id=?)
SEARCH TABLE Table3 AS T3 USING COVERING INDEX sqlite_autoindex_Table3_1 (Id=?)
SCAN SUBQUERY 3
Related
I'm trying to get the result into a variable (is it possible to do it as a %rowtype of an existing table? probably not because of conflicting columns) where it would display me all the values where the two refs overlap and the i_ref(which is inputted ) also overlaps with the ref from t1.
select *
into aRow
from table1 t1
where t1.ref = i_ref
and (select * from table2 t2 where t1.ref = t2.ref);
What am I doing wrong with my select?
You can join tables instead of nested subquery:
select t1.*
into aRow
from table1 t1 join table2 t2 on t1.ref = t2.ref
where t1.ref = i_ref
just wondering how i can create a temp table and then select from it further down the script.
Example.
CREATE TEMPORARY TABLE TEMP_TABLE1 AS
Select
L.ID,
SUM(L.cost)/2 as Costs,
from Table1 L
JOIN Table2 C on L.ID = C.ID
Where C.name = 'mike'
Group by L.ID
Select
Count(L.ID)
from Table1 L
JOIN TEMP_TABLE1 TT1 on L.ID = TT1.ID;
Where L.ID not in (TT1)
And Sum(L.Cost) > TT1.Costs
Ideally I want to have a temp table then use it later in the script to reference from.
Any help would be great!
You simply refer to the table as temp.<table> or <table> the latter only if it is a unique table name.
As per :-
If a schema-name is specified, it must be either "main", "temp", or
the name of an attached database. In this case the new table is
created in the named database. If the "TEMP" or "TEMPORARY" keyword
occurs between the "CREATE" and "TABLE" then the new table is created
in the temp database. It is an error to specify both a schema-name and
the TEMP or TEMPORARY keyword, unless the schema-name is "temp". If no
schema name is specified and the TEMP keyword is not present then the
table is created in the main database.
SQL As Understood By SQLite - CREATE TABLE
The following example creates 3 tables :-
table1 with 3 columns as a permanent table.
table1 a temporary copy of the permanent table1.
temp_table another temporary copy of the permanent table1.
:-
DROP TABLE IF EXISTS temp.table1;
DROP TABLE IF EXISTS table1;
DROP TABLE IF EXISTS temp_table;
CREATE TABLE table1 (columnA INTEGER,columnB INTEGER, columnC INTEGER);
When creating the permanent table 1 it is loaded with 4 rows
:-
INSERT INTO table1 (columnA,columnB,columnC) VALUES
(1,5,20),
(2,7,21),
(3,8,80),
(4,3,63);
CREATE TEMP TABLE table1 AS select * from table1;;
CREATE TEMPORARY TABLE temp_table AS SELECT * FROM table1;
both temp tables are then used to in a union all to basically duplicate the rows, but with an indicator of the source table as a new column from_table
Not that two forms of referring to the temp tables are used. temp. and just the table name.
The latter only usable if the temporary table is a unique table name.
:-
SELECT 'temp_table' AS from_table,* FROM temp_table
UNION ALL
SELECT 'temp.table1' as from_table,* FROM temp.table1;
The result being :-
Re addition of example :-
CREATE TEMPORARY TABLE TEMP_TABLE1 AS
Select
L.ID,
SUM(L.cost)/2 as Costs,
from Table1 L
JOIN Table2 C on L.ID = C.ID
Where C.name = 'mike'
Group by L.ID
Select
Count(L.ID)
from Table1 L
JOIN TEMP_TABLE1 TT1 on L.ID = TT1.ID;
Where L.ID not in (TT1)
And Sum(L.Cost) > TT1.Costs
There are a few issues with this example bar the misuse of the aggregate (commented out) the following works.
Note for my convenience I've added an _ to the table names.
:-
DROP TABLE IF EXISTS Table_1;
DROP TABLE IF EXISTS Table_2;
DROP TABLE If EXISTS temp.temp_table1;
CREATE TABLE Table_1 (ID INTEGER PRIMARY KEY, cost REAL);
CREATE TABLE Table_2 (ID INTEGER PRIMARY KEY, name TEXT);
INSERT INTO Table_1 (cost) VALUES (100.45),(56.78),(99.99);
INSERT INTO Table_2 (name) VALUES ('mike'),('mike'),('fred');
CREATE TEMP TABLE temp_table1 AS
SELECT L.ID,
sum(L.cost)/2 as Costs
FROM Table_1 L
JOIN Table_2 C ON L.ID = C.ID
WHERE C.name = 'mike'
GROUP BY L.ID;
SELECT
count(L.ID)
FROM Table_1 L
JOIN temp_table1 TT1 ON L.ID = TT1.[L.ID]
WHERE
L.ID NOT IN (TT1.[L.ID])
-- AND Sum(L.cost) > TT1.costs --<<<< misuse of aggregate
The issues are based upon the column name being L.ID so this has to be enclosed (rules here SQL As Understood By SQLite - SQLite Keywords apply) [ and ] have been used above.
of course you could circumvent the need for enclosure by naming the column using AS e..g SELECT
L.ID AS lid, --<<<< AS lid ADDED
SUM(L.cost)/2 as Costs, ,.......
Adding the following may be suitable for getting around the misuse of aggregate :-
GROUP BY L.ID
HAVING sum(L.cost) > TT1.costs
Adding the following to the end of the script :-
SELECT
count(L.ID), *
FROM Table_1 L
JOIN temp_table1 TT1 ON L.ID = TT1.[L.ID];
results in :-
If this is only to be used by one SELECT statement then you can use the WITH clause:
WITH TmpTable(id,cost) AS
(
...SELECT statement that returns the two columns (id and cost)...
)
SELECT id, cost FROM TmpTable WHERE ...;
I have two tables whose relevant columns look more or less like the following:
table1.id:
zIXuJeY-qH
zJrcULGLXK
zXyIgnlylb
zZ7MYDGDAV
zbCFI4wKpe
ziGeauO-0O
zmzWbaGSb7
table2.id
zIXuJeY-qH
zIXuJeY-qH
zIXuJeY-qH
zIXuJeY-qH
zJrcULGLXK
zJrcULGLXK
in which table1.id is a primary key and table2.id is not a primary key. I'm trying to grab all the IDs from table 1 that are not in table 2, but am having trouble. If, for instance, I select a count of all the values that are in table two, I get:
>SELECT COUNT(id) FROM table1 WHERE id IN (SELECT id FROM table2);
>2
But if I try to find the IDs not in table2, I get 0:
>SELECT COUNT(id) FROM table1 WHERE id NOT IN (SELECT id FROM table2);
>
There are 503 unique IDs in table1 and only 163 in table2. What am I doing wrong?
I suppose there can be nulls in table2.id?
NOT IN ( ..., null, ...)
does never result in true unfortunately (because - so the argumentation - null is the unknown value, and we cannot guarantee that the unknown value is not the value you are looking for obviously). A bad trap.
Try
SELECT COUNT(id) FROM table1 WHERE id NOT IN (SELECT id FROM table2 where id is not null);
or an EXISTS clause instead.
I am having two tables which i wish to join.
I am not using foreign keys.
table structure table1
id,val1
table structure table2
id,val2
I am using now following command:
update table1 set val1=(SELECT val2 FROM table2 WHERE table1.id LIKE table2.id)
How can i force sqlite not to insert a value in table1 if an id does not exist in table2?
To use exact comparisons, use = instead of LIKE.
To update only those rows where a match is found, use a WHERE clause:
UPDATE Table1
SET val1 = (SELECT val2
FROM Table2
WHERE Table1.id = Table2.id)
WHERE id IN (SELECT id FROM Table2);
If you omit the WHERE clause, rows without a match get updated with a NULL value.
I have a DropDownList that is populated with the primary key of Table A. Table B uses Table A primary key as a foreign key. The foreign key can only be used once.
So currently my list always populates with: 1, 2, 3, 4, 5 (exists as primary in Table A). Table B is already using keys: 1, 2, 3. When they keys are already existing in Table B, I need them not to populate in the dropdownlist.
I have tried to state in an SQL query, to only populate keys that don't exist in Table B. The query I'm trying, but doesn't work is:
SELECT Table_A.Grades_ID FROM Table_A INNER JOIN Table_B ON Table_A_ID = Table_B.Grades_ID WHERE Table_A.Grades_ID != Table_B.Grades_ID
How do I get my dropdownlist to not populate keys (data) that already exists in Table B?
Just adjust your SQL to exclude the PKey values that already exist in TableB
SELECT Table_A.Grades_ID
FROM
Grades
LEFT JOIN Table_B ON Table_A_ID = Table_B.Grades_ID
WHERE Table_B.Grades_ID Is Null
Alternatively, you may wish to use "not in"
SELECT
Tables_A.Grades_ID
WHERE
Tables_A.Grades_ID not in (Select Grades_ID from Table_B)
Try this:
select table_a.grades_id
from table_a left outer join
on table_a.grades_id = table_b.grades_id
where table_b.<some other field> is null
SELECT Grades_ID FROM Grades WHERE Grades_ID NOT IN (SELECT Grades_ID FROM Table_B)
Select Table_A.Grades_ID
from Table_A
where Table_A.Grades_ID NOT IN (Select Grades_ID from Table_B)