I have a query like this (simplified and anonymised):
SELECT
Department.id,
Department.name,
Department.manager_id,
Employee.name AS manager_name
FROM
Department
LEFT OUTER JOIN Employee
ON Department.manager_id = Employee.id;
The field Department.manager_id may be NULL. If it is non-NULL then it is guaranteed to be a valid id for precisely one row in the Employee table, so the OUTER JOIN is there just for the rows in the Department table where it is NULL.
Here is the problem: old instances of the database do not have this Department.manager_id column at all. In those cases, I would like the query to act as if the field did exist but was always NULL, so e.g. the manager_name field is returned as NULL. If the query only used the Department table then I could just use SELECT * and check for the column in my application, but the JOIN seems to make this impossible. I would prefer not to modify the database, partly so that I can load the database in read only mode. Can this be done just by clever adjustment of the query?
For completeness, here is an answer that does not require munging both possible schemas into one query (but still doesn't need you to actually do the schema migration):
Check for the schema version, and use that to determine which SELECT query to issue (i.e. with or without the manager_id column and JOIN) as a separate step. Here are a few possibilities to determine the schema version:
The ideal situation is that you already keep track of the schema by assigning version numbers to the schema and recording them in the database. Commonly this is done with either:
The user_version pragma.
A table called "Schema" or similar with one row containing the schema version number.
You can directly determine whether the column is present in the table. Two possibilities:
Use the table_info pragma to determine the list of columns in the table.
Use a simple SELECT * FROM Table LIMIT 1 and look at what columns are returned (this is probably better as it is independent of the database engine).
This seems to work:
SELECT
Dept.id,
Dept.name,
Dept.manager_id,
Employee.name AS manager_name
FROM
(SELECT *, NULL AS manager_id FROM Department) AS Dept
LEFT OUTER JOIN Employee
ON Dept.manager_id = Employee.id;
If the manager_id column is present in Department then it is used for the join, whereas if it is not then Dept.manager_id and Employee.name are both NULL.
If I swap the column order in the subquery:
(SELECT NULL AS manager_id, * FROM Department) AS Dept
then the Dept.manager_id and Employee.name are both NULL even if the Department.manager_id column exists, so it seems that Dept.manager_id refers to the first column in the Dept subquery that has that name. It would be good to find a reference in the SQLite documentation saying that this behaviour is guaranteed (or explicitly saying that it is not), but I can't find anything (e.g. in the SELECT or expression pages).
I haven't tried this with other database systems so I don't know if it will work with anything other than SQLite.
Related
For example, let say DB has foreign key A.b_id -> B.id with SET NULL on delete.
If record with some B.id get deleted, all b_id references will be set to NULL.
But if A already contains record where A.b_id has value that is not in B.id (it was inserted without foreign keys support), is there a way to force SQLite DB check foreign keys and set to NULL such data?
In fact, in first place I'm solving an DB upgrading task.
On start app checks if internal DB (resource) has higher version than user DB.
If so it backups user DB, copies internal empty DB to user storage. Than turns off foreign keys support and fills new DB with data from backup, inserting automatically in loop table by table for all columns with same name. Turns on foreign keys support back.
Everything works fine, but if in some table in old DB there is no foreign key constrain previously, while new DB has one, the data will be inserted as is and link can point nowhere (possibly wrong links is unavoidable and not related to question).
Yes, I understand a way to insert without turning off foreign keys support, but it would need knowledge of tables dependencies order that I would like to avoid.
Thanks for any help in advance!
Although I don't know of a way that automatically will set to NULL all orphaned values of a column in a table that (should) reference another column in another table, there is a way to get a report of all these cases and then act accordingly.
This is the PRAGMA statement foreign_key_check:
PRAGMA schema.foreign_key_check;
or for a single table check:
PRAGMA schema.foreign_key_check(table-name);
From the documenation:
The foreign_key_check pragma checks the database, or the table called
"table-name", for foreign key constraints that are violated. The
foreign_key_check pragma returns one row output for each foreign key
violation. There are four columns in each result row. The first column
is the name of the table that contains the REFERENCES clause. The
second column is the rowid of the row that contains the invalid
REFERENCES clause, or NULL if the child table is a WITHOUT ROWID
table. The third column is the name of the table that is referred to.
The fourth column is the index of the specific foreign key constraint
that failed. The fourth column in the output of the foreign_key_check
pragma is the same integer as the first column in the output of the
foreign_key_list pragma. When a "table-name" is specified, the only
foreign key constraints checked are those created by REFERENCES
clauses in the CREATE TABLE statement for table-name.
Check a simplified demo of the way to use this PRAGMA statement, or its function counterpart pragma_foreign_key_check().
You can get a list of the rowids of all the problematic rows of each table.
In your case, you can execute an UPDATE statement that will set to NULL all the orphaned b_ids:
UPDATE A
SET b_id = NULL
WHERE rowid IN (SELECT rowid FROM pragma_foreign_key_check() WHERE "table" = 'A')
This also works in later versions of SQLite:
UPDATE A
SET b_id = NULL
WHERE rowid IN (SELECT rowid FROM pragma_foreign_key_check('A'))
but it does not seem to work up to SQLite 3.27.0
Similar to this question and this solution for PostgreSQL (in particular "INSERT missing FK rows at the same time"):
Suppose I am making an address book with a "Groups" table and a "Contact" table. When I create a new Contact, I may want to place them into a Group at the same time. So I could do:
INSERT INTO Contact VALUES (
"Bob",
(SELECT group_id FROM Groups WHERE name = "Friends")
)
But what if the "Friends" Group doesn't exist yet? Can we insert this new Group efficiently?
The obvious thing is to do a SELECT to test if the Group exists already; if not do an INSERT. Then do an INSERT into Contacts with the sub-SELECT above.
Or I can constrain Group.name to be UNIQUE, do an INSERT OR IGNORE, then INSERT into Contacts with the sub-SELECT.
I can also keep my own cache of which Groups exist, but that seems like I'm duplicating functionality of the database in the first place.
My guess is that there is no way to do this in one query, since INSERT does not return anything and cannot be used in a subquery. Is that intuition correct? What is the best practice here?
My guess is that there is no way to do this in one query, since INSERT
does not return anything and cannot be used in a subquery. Is that
intuition correct?
You could use a Trigger and a little modification of the tables and then you could do it with a single query.
For example consider the folowing
Purely for convenience of producing the demo:-
DROP TRIGGER IF EXISTS add_group_if_not_exists;
DROP TABLE IF EXISTS contact;
DROP TABLE IF EXISTS groups;
One-time setup SQL :-
CREATE TABLE IF NOT EXISTS groups (id INTEGER PRIMARY KEY, group_name TEXT UNIQUE);
INSERT INTO groups VALUES(-1,'NOTASSIGNED');
CREATE TABLE IF NOT EXISTS contact (id INTEGER PRIMARY KEY, contact TEXT, group_to_use TEXT, group_reference TEXT DEFAULT -1 REFERENCES groups(id));
CREATE TRIGGER IF NOT EXISTS add_group_if_not_exists
AFTER INSERT ON contact
BEGIN
INSERT OR IGNORE INTO groups (group_name) VALUES(new.group_to_use);
UPDATE contact SET group_reference = (SELECT id FROM groups WHERE group_name = new.group_to_use), group_to_use = NULL WHERE id = new.id;
END;
SQL that would be used on an ongoing basis :-
INSERT INTO contact (contact,group_to_use) VALUES
('Fred','Friends'),
('Mary','Family'),
('Ivan','Enemies'),
('Sue','Work colleagues'),
('Arthur','Fellow Rulers'),
('Amy','Work colleagues'),
('Henry','Fellow Rulers'),
('Canute','Fellow Ruler')
;
The number of values and the actual values would vary.
SQL Just for demonstration of the result
SELECT * FROM groups;
SELECT contact,group_name FROM contact JOIN groups ON group_reference = groups.id;
Results
This results in :-
1) The groups (noting that the group "NOTASSIGNED", is intrinsic to the working of the above and hence added initially) :-
have to be careful regard mistakes like (Fellow Ruler instead of Fellow Rulers)
-1 used because it would not be a normal value automatically generated.
2) The contacts with the respective group :-
Efficient insertion
That could likely be debated from here to eternity so I leave it for the fence sitters/destroyers to decide :). However, some considerations:-
It works and appears to do what is wanted.
It's a little wasteful due to the additional wasted column.
It tries to minimise the waste by changing the column to an empty string (NULL may be even more efficient, but for some can be confusing)
There will obviously be an overhead BUT in comparison to the alternatives probably negligible (perhaps important if you were extracting every Facebook user) but if it's user input driven likely irrelevant.
What is the best practice here?
Fences again. :)
Note Hopefully obvious, but the DROP statements are purely for convenience and that all other SQL up until the INSERT is run once
to setup the tables and triggers in preparation for the single INSERT
that adds a group if necessary.
I'm working on a sqlite database and try to make a special request between two tables.
In the first table (table1 for example), i have two columns named "reference" and "ID". I want to search an ID in it, get it value in "reference" and display all informations from the table which have this value as name.
I try to find something on the internet but I didn't find an answer.
This is the request I made:
select * from (select Reference from table1 where Name='Value1')
It only give me the result of
select Reference from table1 where Name='Value1'
EDIT:
I want
select Reference from table1 where Name='Value1' => name of table
select * from name of table => show all elements
I'm new in sqlite but I hope you can help me.
Thank you by advance
Matt
If I understand your question correctly, I don't think there's a way to do it in sql completely (or at least not in a portable way). I'd recommend one of 3 solutions:
Do exactly what you want, but do some processing in Python. That means query your master table, then construct new query based on each of the rows returned.
If you have many tables, possibly changing dynamically - it may be a good idea to rethink your database design. Maybe you can move some of the changing table names into a new column and put your data in one table?
If you have only a few tables available as the Reference and they never change, you could join all the possible tables, like:
SELECT ... FROM table1
LEFT JOIN table2
ON table1.id = table2.id AND table1.Reference = "table2"
LEFT JOIN table3 ...
But you may need to explain it all a bit better...
SELECT tbl_user.userid,
tbl_user.firstname,
tbl_user.lastname,
tbl_user.email,
tbl_user.created,
tbl_user.createdby,
tbl_organisation.organisationname
FROM tbl_user
INNER JOIN tbl_organisation
ON tbl_user.organisationid = tbl_organisation.organisationid
WHERE organisationid = #OrganisationID;
I am using this statement to do a databind. I am getting a error here.
Column 'OrganisationID' in where clause is ambiguous
What should I do is it wrong to name the OrganisationID in tbl_user same as tbl_organisation.
OrganisationID is a foreign key from tbl_Organisation
Since you have two columns with the same name on two different tables (and that's not a problem, it's even recommended on many cases), you must inform MySQL which one you want to filter by.
Add the table name (or alias, if you were using table aliases) before the column name. In your case, either
WHERE tbl_user.OrganisationID
or
WHERE tbl_Organisation.OrganisationID
should work.
You just need to indicate which table you are targeting with that statement, like "tbl_user.OrganisationID". Otherwise the engine doesn't know which OrganisationID you meant.
It is not wrong the have the same column names in two tables. In many (even most) cases, it is actually perferred.
I have recently stumbled upon a problem with selecting relationship details from a 1 table and inserting into another table, i hope someone can help.
I have a table structure as follows:
ID (PK) Name ParentID<br>
1 Myname 0<br>
2 nametwo 1<br>
3 namethree 2
e.g
This is the table i need to select from and get all the relationship data. As there could be unlimited number of sub links (is there a function i can create for this to create the loop ?)
Then once i have all the data i need to insert into another table and the ID's will now have to change as the id's must go in order (e.g. i cannot have id "2" be a sub of 3 for example), i am hoping i can use the same function for selecting to do the inserting.
If you are using SQL Server 2005 or above, you may use recursive queries to get your information. Here is an example:
With tree (id, Name, ParentID, [level])
As (
Select id, Name, ParentID, 1
From [myTable]
Where ParentID = 0
Union All
Select child.id
,child.Name
,child.ParentID
,parent.[level] + 1 As [level]
From [myTable] As [child]
Inner Join [tree] As [parent]
On [child].ParentID = [parent].id)
Select * From [tree];
This query will return the row requested by the first portion (Where ParentID = 0) and all sub-rows recursively. Does this help you?
I'm not sure I understand what you want to have happen with your insert. Can you provide more information in terms of the expected result when you are done?
Good luck!
For the retrieval part, you can take a look at Common Table Expression. This feature can provide recursive operation using SQL.
For the insertion part, you can use the CTE above to regenerate the ID, and insert accordingly.
I hope this URL helps Self-Joins in SQL
This is the problem of finding the transitive closure of a graph in sql. SQL does not support this directly, which leaves you with three common strategies:
use a vendor specific SQL extension
store the Materialized Path from the root to the given node in each row
store the Nested Sets, that is the interval covered by the subtree rooted at a given node when nodes are labeled depth first
The first option is straightforward, and if you don't need database portability is probably the best. The second and third options have the advantage of being plain SQL, but require maintaining some de-normalized state. Updating a table that uses materialized paths is simple, but for fast queries your database must support indexes for prefix queries on string values. Nested sets avoid needing any string indexing features, but can require updating a lot of rows as you insert or remove nodes.
If you're fine with always using MSSQL, I'd use the vendor specific option Adrian mentioned.