SQLite list ALL foreign keys in a database - sqlite

Is there a way of listing ALL foreign keys in a SQLite database?
They don't seem to be stored in sqlite_master and PRAGMA foreign_key_list('table') only lists one at a time.
Alternatively, is there a way of listing what foreign keys reference a table?

It seems that all (or many) of the PRAGMA commands can be programatically selected with a little trick;
Usually the are called like:
PRAGMA table_info('my_table');
PRAGMA foreign_key_list('my_table');
But this can also be done:
SELECT * FROM pragma_table_info('my_table');
SELECT * FROM pragma_foreign_key_list('my_table');
And the schema can also be (more or less) obtained:
.schema pragma_table_info
/* pragma_table_info(cid,name,type,"notnull",dflt_value,pk) */;
.schema pragma_foreign_key_list
/* pragma_foreign_key_list(id,seq,"table","from","to",on_update,on_delete,"match") */
So, to get all the fks a JOIN between sqlite_master and pragma_foreign_key_list can do the trick:
SELECT
m.name
, p.*
FROM
sqlite_master m
JOIN pragma_foreign_key_list(m.name) p ON m.name != p."table"
WHERE m.type = 'table'
ORDER BY m.name
;
Just take care, that some fields of pragma_foreign_key_list like table, from, ... must be quoted;

With the SQLite shell, use the .schema instruction, and use GREP to filter lines containing REFERENCES.
From shell.c in the SQLite repository, today's version in the trunk, two queries:
SELECT sql
FROM (
SELECT sql sql, type type, tbl_name tbl_name, name name
FROM sqlite_master
UNION ALL
SELECT sql, type, tbl_name, name
FROM sqlite_temp_master
)
WHERE tbl_name LIKE shellstatic()
AND type != 'meta'
AND sql NOTNULL
ORDER BY substr(type, 2, 1), name
and
SELECT sql
FROM (
SELECT sql sql, type type, tbl_name tbl_name, name name
FROM sqlite_master
UNION ALL
SELECT sql, type, tbl_name, name
FROM sqlite_temp_master
)
WHERE type != 'meta'
AND sql NOTNULL
AND name NOT LIKE 'sqlite_%'
ORDER BY substr(type, 2, 1), name
The second one is probably what you are looking for.

Related

How to get a list of tables that have one-on-one relationship to a given table in SQLite3?

Is there a way to get a list of tables that have one-on-one relationship to a given table in SQLite3?
For example, here table ab has a one-on-one relationship with both table abc and abd. Is there a query or queries to return abc and abd for the given table name ab?
-- By default foreign key is diabled in SQLite3
PRAGMA foreign_keys = ON;
CREATE TABLE a (
aid INTEGER PRIMARY KEY
);
CREATE TABLE b (
bid INTEGER PRIMARY KEY
);
CREATE TABLE ab (
aid INTEGER,
bid INTEGER,
PRIMARY KEY (aid, bid)
FOREIGN KEY (aid) REFERENCES a(aid)
FOREIGN KEY (bid) REFERENCES b(bid)
);
-- tables 'ab' and 'abc' have a one-on-one relationship
CREATE TABLE abc (
aid INTEGER,
bid INTEGER,
name TEXT NOT NULL,
PRIMARY KEY (aid, bid) FOREIGN KEY (aid, bid) REFERENCES ab(aid, bid)
);
-- tables 'ab' and 'abd' have a one-on-one relationship
CREATE TABLE abd (
aid INTEGER,
bid INTEGER,
value INTEGER CHECK( value > 0 ),
PRIMARY KEY (aid, bid) FOREIGN KEY (aid, bid) REFERENCES ab(aid, bid)
);
CREATE TABLE w (
id INTEGER PRIMARY KEY
);
The following tedious precedure may get me the list of tables I want:
Get primary keys for table ab:
SELECT l.name FROM pragma_table_info('ab') as l WHERE l.pk > 0;
get foreign keys for other tables (this case is for table abd):
SELECT * from pragma_foreign_key_list('abd');
Do parsing to get what the list of tables of one-on-one relationships.
However, there must exist a more elegant way, I hope.
For SQL Server, there are sys.foreign_keys and referenced_object_id avaible (see post). Maybe there is something similar to that in SQLite?
Edit: adding two more tables for test
-- tables 'ab' and 'abe' have a one-on-one relationship
CREATE TABLE abe (
aid INTEGER,
bid INTEGER,
value INTEGER CHECK( value < 0 ),
PRIMARY KEY (aid, bid) FOREIGN KEY (aid, bid) REFERENCES ab
);
-- tables 'ab' and 'abf' have a one-on-one relationship
CREATE TABLE abf (
aidQ INTEGER,
bidQ INTEGER,
value INTEGER,
PRIMARY KEY (aidQ, bidQ) FOREIGN KEY (aidQ, bidQ) REFERENCES ab(aid, bid)
);
Edit: verify FK for table abe
sqlite> PRAGMA foreign_keys;
1
sqlite> .schema abe
CREATE TABLE abe (
aid INTEGER,
bid INTEGER,
value INTEGER CHECK( value < 0 ),
PRIMARY KEY (aid, bid) FOREIGN KEY (aid, bid) REFERENCES ab
);
sqlite> DELETE FROM abe;
sqlite> INSERT INTO abe (aid, bid, value) VALUES (2, 1, -21);
sqlite> INSERT INTO abe (aid, bid, value) VALUES (-2, 1, -21);
Error: FOREIGN KEY constraint failed
sqlite> SELECT * FROM ab;
1|1
1|2
2|1
Alternative
Although not a single query solution the following only requires submission/execution of a series of queries and is therefore platform independent.
It revolves around using two tables:-
a working copy of sqlite_master
a working table to store the the output of SELECT pragma_foreign_key_list(?)
Both tables are created via a CREATE-SELECT, although neither has any rows copied, so the tables are empty.
A trigger is applied to the working copy of sqlite_master to insert into the table that stores the result of SELECT pragma_foreign_key_list(table_name_from_insert);
The relevant rows are copied from sqlite_master via a SELECT INSERT and thus the triggering populates the store table.
The following is the testing code :-
DROP TABLE IF EXISTS fklist;
DROP TABLE IF EXISTS master_copy;
DROP TRIGGER IF EXISTS load_fklist;
/* Working version of foreign_key_list to store ALL results of SELECT pragma_foreign_key_list invocation */
CREATE TABLE IF NOT EXISTS fklist AS SELECT '' AS child,*
FROM pragma_foreign_key_list((SELECT name FROM sqlite_master WHERE type = 'not a type' LIMIT 1));
/* Working version of sqlite master */
CREATE TABLE IF NOT EXISTS master_copy AS SELECT * FROM sqlite_master WHERE type = 'not a type';
/* Add an after insert trigger for master copy to add to fklist */
CREATE TRIGGER IF NOT EXISTS load_fklist
AFTER INSERT ON master_copy
BEGIN
INSERT INTO fklist SELECT new.name,* FROM pragma_foreign_key_list(new.name);
END
;
/* Populate master_copy from sqlite_master (relevant rows)
and thus build the fklist
*/
INSERT INTO master_copy SELECT *
FROM sqlite_master
WHERE type = 'table'
AND instr(sql,' REFERENCES ') > 0
;
SELECT * FROM fklist;
DROP TABLE IF EXISTS fklist;
DROP TABLE IF EXISTS master_copy;
DROP TRIGGER IF EXISTS load_fklist;
Using a similar test base as per the previous answer the above results in :-
Is there a way to get a list of tables that have one-on-one relationship to a given table in SQLite3?
Not with certainty as coding a Foreign Key constraint does not define a relationship (rather it supports a relationship), that is relationships can exists without a FK constraint.
A Foreign Key constraint defines:-
a) a rule that enforces referential integrity
b) optionally maintains/alters referential integrity when the referred to column is changed (ON DELETE and ON UPDATE )
As such looking at the Foreign Key List only tells you where/if a FK constraint has been coded.
Saying that the following will get the tables with the constraint and the referenced tables.
More elegant is a matter of opinion, so it's up to you :-
WITH cte_part(name,reqd,rest) AS (
SELECT name,'',substr(sql,instr(sql,' REFERENCES ') + 12)||' REFERENCES '
FROM sqlite_master
WHERE sql LIKE '% REFERENCES %(%'
UNION ALL
SELECT
name,
substr(rest,0,instr(rest,' REFERENCES ')),
substr(rest,instr(rest,' REFERENCES ') + 12)
FROM cte_part
WHERE length(rest) > 12
)
SELECT DISTINCT
CASE
WHEN length(reqd) < 1 THEN name
ELSE
CASE substr(reqd,1,1)
WHEN '''' THEN substr(replace(reqd,substr(reqd,1,1),''),1,instr(reqd,'(')-3)
WHEN '[' THEN substr(replace(replace(reqd,'[',''),']',''),1,instr(reqd,'(')-3)
WHEN '`' THEN substr(replace(reqd,substr(reqd,1,1),''),1,instr(reqd,'(')-3)
ELSE substr(reqd,1,instr(reqd,'(')-1)
END
END AS tablename
FROM cte_part
;
As an example of it's use/results :-
screenshot from Navicat
Here's an adaptation of the above that includes, where appropriate, the child table that references the parent :-
WITH cte_part(name,reqd,rest) AS (
SELECT name,'',substr(sql,instr(sql,' REFERENCES ') + 12)||' REFERENCES '
FROM sqlite_master
WHERE sql LIKE '% REFERENCES %(%'
UNION ALL
SELECT
name,
substr(rest,0,instr(rest,' REFERENCES ')),
substr(rest,instr(rest,' REFERENCES ') + 12)
FROM cte_part
WHERE length(rest) > 12
)
SELECT DISTINCT
CASE
WHEN length(reqd) < 1 THEN name
ELSE
CASE substr(reqd,1,1)
WHEN '''' THEN substr(replace(reqd,substr(reqd,1,1),''),1,instr(reqd,'(')-3)
WHEN '[' THEN substr(replace(replace(reqd,'[',''),']',''),1,instr(reqd,'(')-3)
WHEN '`' THEN substr(replace(reqd,substr(reqd,1,1),''),1,instr(reqd,'(')-3)
ELSE substr(reqd,1,instr(reqd,'(')-1)
END
END AS tablename,
CASE WHEN length(reqd) < 1 THEN '' ELSE name END AS referrer
FROM cte_part
;
Example of the Result :-
the artists table is referenced by albums as the SQL used to create the albums table is CREATE TABLE 'albums'([AlbumId] INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL ,[Title] TEXT NOT NULL ,[ArtistId] INTEGER NOT NULL , FOREIGN KEY ([ArtistId]) REFERENCES 'artists'([ArtistId]))
i.e. FOREIGN KEY ([ArtistId]) REFERENCES 'artists'([ArtistId]))
the employees table is self-referencing as per CREATE TABLE 'employees'(.... REFERENCES 'employees'([EmployeeId]))
Additional re comment:-
(I am still trying to understand your code...)
The code is based upon selecting rows from sqlite_master where the row is for a table (type = 'table'), as opposed to an index, trigger or view and where the sql column contains the word REFERENCES with a space before and after and there is a following left parenthesis.
The last condition used to weed out the likes of CREATE TABLE oops (`REFERENCES` TEXT, `x REFERENCES Y`);
For each selected row 3 columns are output:-
name which is the name of the table as extracted from the name column of sqlite_master,
reqd is initially an empty string (i.e. initial)
rest the rest of sql that follows the referred to table name with suffixed with REFERENCES.
The UNION ALL adds rows that are built upon what is newly added to the CTE, i.e. the three columns are extracted as per :-
name is the name
reqd is the sql from the rest column up until the first REFERENCES term (i.e. the table and referenced column(s))
rest is the sql from after the REFERENCES term
As with any recursion the end needs to be detected, this is when the entire sql statement has been reduced to being less than 12 (i.e the length of " REFERENCES ", the term used for splitting the sql statement).
This is what is termed as a RECURSIVE CTE
Finally the resultant CTE is then queried. If the reqd field is empty then the tablename column is the name column otherwise (i.e. the reqd column contains data(part of the sql)) the table name is extracted (part up to left parenthesis if not enclosed (`,' or [ with ])) or extracted from between the enclosure.
The following is what the final query results in if all the CTE columns are included (some data has been truncated):-
As can clearly be seen the extracted sql progressively reduces
The answer is intended as in-principle and has not been extensively tested to consider all scenarios, it may well need tailoring.

bulk update in SQLite

I have 2 tables with identical structure I want to update one table using data from the other, matching on primary key. SQLite has a with (CTE) statement but the following doesn't work (sqlite3 v. 3.29.0):
sqlite> select * from main;
1|A
2|B
4|D
5|E
6|F
sqlite> select * from temp;
1|aa
2|bb
3|cc
4|dd
5|ee
sqlite> with mapping as (select main.ID, temp.Desc from main join temp on temp.ID=main.ID) update main set Desc=mapping.Desc where main.ID=mapping.ID;
Error: no such column: mapping.Desc
I've tried using "select main.ID as ID, temp.Desc as Desc", but get the same error message.
To update your main table from your cte, use a subquery, since sqlite doesn't support update from
with mapping as
(select main.ID, temp.Desc
from main
join temp on temp.ID=main.ID)
update main set Desc=
(select Desc from mapping where ID = main.ID limit 1);
see dbfiddle

How to fetch names of virtual tables?

Example schema:
CREATE VIRTUAL TABLE posts USING FTS5(title, body);
Select table names:
SELECT name FROM sqlite_master WHERE type='table';
Result:
posts
posts_data
posts_idx
posts_content
posts_docsize
posts_config
How to fetch result only for virtual tables, without *_data, *_idx, *_content, *_docsize and *_config?
The FTS modules use shadow tables to store the actual data and its indexes.
But those are 'real' tables, so you can simply use a filter to get only sqlite_master entries for virtual tables:
SELECT name
FROM sqlite_master
WHERE type = 'table'
AND sql LIKE 'CREATE VIRTUAL TABLE%';

ssdt: change the types of primary key/secondary keys

Original table1 and Table2. Both tables has data.
CREATE TABLE [dbo].[Table1]
(
[Id] int NOT NULL PRIMARY KEY
)
CREATE TABLE [dbo].[Table2]
(
[Id] INT NOT NULL PRIMARY KEY,
[Table1Id] Int NULL,
Constraint [FK_Table1_Table2] foreign key ([Table1Id]) references [Table1] (Id)
)
I'd like to change the Table1.Id to UNIQUEIDENTIFIER.
Obviously just jump in and change the type from int to UNIQUEIDENTIFIER for Table1.Id and 'Table2.Table1Id'. Then Publish. Here is the code:
CREATE TABLE [dbo].[tmp_ms_xx_Table1] (
[Id] UNIQUEIDENTIFIER NOT NULL,
PRIMARY KEY CLUSTERED ([Id] ASC)
);
IF EXISTS (SELECT TOP 1 1
FROM [dbo].[Table1])
BEGIN
INSERT INTO [dbo].[tmp_ms_xx_Table1] ([Id])
SELECT [Id]
FROM [dbo].[Table1]
ORDER BY [Id] ASC;
END
This code will fail because original Table1.Id is Int while temp table Id is UNIQUEIDENTIFIER.
Then, I try to with Pre-Scripts. Ideally all the changes will be done manually.
--drop fk constraint
alter table [Table2] drop constraint [FK_Table1_Table2];
--rename table1.id
exec sp_rename 'Table1.Id', 'Id2', 'COLUMN';
alter table [Table1] add Id uniqueidentifier not null
default newid();
--rename table2.table1id
exec sp_rename 'Table2.Table1Id', 'Table1Id2', 'COLUMN';
alter table [Table2] add Table1Id uniqueidentifier null;
update t2 set t2.Table1ID = t1.Id
from Table2 t2 left join Table1 t1 on t2.Table1Id2 = t1.Id2;
alter table [Table2] add constraint [FK_Table1_Table2] foreign key (Table1Id) references Table1 (Id);
However it FAIL again as SSDT is trying to compare its data structure again the target database.
Any idea please?
You're right. The problem is that you can't include schema changes in the pre-deployment script because SSDT's deployment script is generated prior to your schema changes. It is therefore only useful for data-only changes.
The solution is to do this outside of the SSDT process altogether. Yes, it's a pre-pre-deployment script! Essentially you have to apply your change by yourself before you even get to the SSDT bit.
(There's probably a way to do this via a custom deployment contributor. After all, everything is possible in code...)
Can I convince you to take a look at a migration-based solution as it appears that you have sufficient need for an element of fine-grained script "customisation". DBUp is a popular open source solution. ReadyRoll is a more-integrated commercial solution that shares a lot with SSDT.
the problem is the old data. it will be ok without the data in table in step 2.
1.pre-script: copy/process old data to temp tables, delete them from original tables
create table #table1 (
id int null,
id2 uniqueidentifier null
);
insert into #table1 (id,id2)
select id,newid() from Table1;
create table #table2 (
id int null,
table1id int null,
table1id2 uniqueidentifier null
);
insert into #table2 (id, table1id)
select id,table1id from Table2;
update t2 set t2.table1id2=t1.id2
from #table2 t2 left join #table1 t1 on t2.table1id = t1.id;
delete from table2;
delete from table1;
dacpac will auto generate the changes for schema. it will be ok because no data is existing any more.
post-script: insert data back from temp tables in pre-script:
insert into table1 (id)
select id2 from #table1;
insert into table2 (id,table1id)
select id, table1id2 from #table2;

How can I get the list of a columns in a table for a SQLite database?

I am looking to retrieve a list of columns in a table. The database is the latest release of SQLite (3.6, I believe). I am looking for code that does this with a SQL query. Extra bonus points for metadata related to the columns (e.g. length, data type, etc...)
What you're looking for is called the data dictionary. In sqlite a list of all tables can be found by querying sqlite_master table (or view?)
sqlite> create table people (first_name varchar, last_name varchar, email_address varchar);
sqlite> select * from sqlite_master;
table|people|people|2|CREATE TABLE people (first_name varchar, last_name varchar, email_address varchar)
To get column information you can use the pragma table_info(table_name) statement:
sqlite> pragma table_info(people);
0|first_name|varchar|0||0
1|last_name|varchar|0||0
2|email_address|varchar|0||0
For more information on the pragma statements, see the documentation.
Here's the simple way:
.schema <table>
The question is old but the following hasn't been mentioned yet.
Another convenient way in many cases is to turn headers on by:
sqlite> .headers on
Then,
sqlite> SELECT ... FROM table
will display a headline showing all selected fields (all if you SELECT *) at the top of the output.
Here's a SELECT statement that lists all tables and columns in the current database:
SELECT m.name as tableName,
p.name as columnName
FROM sqlite_master m
left outer join pragma_table_info((m.name)) p
on m.name <> p.name
order by tableName, columnName
;
just go into your sqlite shell:
$ sqlite3 path/to/db.sqlite3
and then just hit
sqlite> .schema
and you will get everything.
This is a query that lists all tables with their columns, and all the metadata I could get about each column as OP requested (as bonus points).
SELECT
m.name AS table_name,
p.cid AS col_id,
p.name AS col_name,
p.type AS col_type,
p.pk AS col_is_pk,
p.dflt_value AS col_default_val,
p.[notnull] AS col_is_not_null
FROM sqlite_master m
LEFT OUTER JOIN pragma_table_info((m.name)) p
ON m.name <> p.name
WHERE m.type = 'table'
ORDER BY table_name, col_id
Thanks to #David Garoutte for showing me how to get pragma_table_info to work in a query.
Run this query to see all the table metadata:
SELECT * FROM sqlite_master WHERE type = 'table'
Building on the above, you can do it all at once:
sqlite3 yourdb.db ".schema"
That will give you the SQL to create the table, which is effectively a list of the columns.
In case if you want to get all column names into one single comma separated string, you can use below.
SELECT GROUP_CONCAT(NAME,',') FROM PRAGMA_TABLE_INFO('table_name')
Here the pragma table_info is used as pragma_table_info for the select statement and GROUP_CONCAT is to combine all the field names into one string. for the second parameter of GROUP_CONCAT you can pass the separator.
I know, it’s been a long time but it’s never too late…
I had a similar question with TCL as interpreter and after several search, found nothing good for me. So I propose something based on PRAGMA, knowing that your DB is “main”
db eval { PRAGMA main.table_info(<your table name>) } TBL { puts $TBL(name) }
And array use to obtain a list
set col_list {}
db eval { PRAGMA main.table_info(<your table name>) } TBL { lappend col_list $TBL(name) }
puts $col_list

Resources