SQLite - Inserting values from multiple rows in the same table into a single row in another table - sqlite

I have a table of phone numbers (tblPhoneNumbers):
ID, UserID, PhoneNumber
and I need to move them into a Users table (tblUsers) that contains:
ID, PhoneNumber1, PhoneNumber2
tblPhoneNumbers is assumed to have 2 rows for every user. Is it possible to move the PhoneNumber value of the first row into PhoneNumber1, and the PhoneNumber value of the second row into PhoneNumber2?
Essentially this is reverse-normalization but this is the task I need help with.
Thanks!
I need to use SQLite so I cannot use any syntax not available to SQLite.

If you're using sqlite 3.25 or better, you can use window functions to do it all in one statement (I assume here that the UserID column from tblPhoneNumbers is a foreign key that references ID from tblUsers, and that the given userid already has a record in that table; adjust as needed):
WITH allnumbers AS
(SELECT UserID
, PhoneNumber
, row_number() OVER (PARTITION BY UserID) AS num
FROM tblPhoneNumbers)
UPDATE tblUsers AS t
SET PhoneNumber1 = (SELECT a.PhoneNumber
FROM allnumbers AS a
WHERE a.UserID = t.ID AND num = 1)
, PhoneNumber2 = (SELECT a.PhoneNumber
FROM allnumbers AS a
WHERE a.UserID = t.ID AND num = 2);
(And if your system only has an older version that don't support window functions, you can always download a copy of the latest version of the sqlite3 shell and use it instead of the OS provided one).
(edit: You'll want an index on tblPhoneNumbers.UserID for better performance)

You could use the following :-
-- Create a temporary swap table
CREATE TEMP TABLE IF NOT EXISTS swapPhoneNumbers (ID INTEGER PRIMARY KEY, UserID INTEGER, PhoneNumber TEXT, replacementPhoneNumber TEXT);
-- Clear the temporary swap table in case it's used more than once
DELETE FROM swapPhoneNumbers;
-- Populate the temporary swap table according to the original data
INSERT INTO swapPhoneNumbers (ID,UserID,PhoneNumber) SELECT * FROM tblPhoneNumbers;
-- Update the swap table to include the replacement phone numbers
UPDATE swapPhoneNumbers SET replacementPhoneNumber = (
SELECT PhoneNumber FROM tblPhoneNumbers
WHERE swapPhoneNumbers.userID = tblPhoneNumbers.userID
AND swapPhoneNumbers.ID <> tblPhoneNumbers.ID
);
-- Update the original table with the new phone numbers
UPDATE tblPhoneNumbers SET PhoneNumber = (
SELECT replacementPhoneNumber FROM swapPhoneNumbers
WHERE tblPhoneNumbers.ID = swapPhoneNumbers.ID
);
The following is the SQL used to test the above.
-- Create Testing Table with some data
DROP TABLE IF EXISTS tblphoneNumbers;
CREATE TABLE IF NOT EXISTS tblPhoneNumbers (ID INTEGER PRIMARY KEY, userID INTEGER, PhoneNumber TEXT);
INSERT INTO tblPhoneNumbers (userID, PhoneNumber) VALUES
(1,'0111111111'),(1,'0222222222'),(2,'0333333333'),(2,'0444444444'),(3,'0555555555'),(3,'0666666666')
;
-- Show what is in the original table
SELECT * FROM tblPhoneNumbers;
-- Create a temporary swap table
CREATE TEMP TABLE IF NOT EXISTS swapPhoneNumbers (ID INTEGER PRIMARY KEY, UserID INTEGER, PhoneNumber TEXT, replacementPhoneNumber TEXT);
-- Clear the temporary swap table in case it's used more than once
DELETE FROM swapPhoneNumbers;
-- Populate the temporary swap table according to the original data
INSERT INTO swapPhoneNumbers (ID,UserID,PhoneNumber) SELECT * FROM tblPhoneNumbers;
-- Show what is in the swap table
SELECT * FROM swapPhoneNumbers;
-- Update the swap table to include the replacement phone numbers
UPDATE swapPhoneNumbers SET replacementPhoneNumber = (
SELECT PhoneNumber FROM tblPhoneNumbers
WHERE swapPhoneNumbers.userID = tblPhoneNumbers.userID
AND swapPhoneNumbers.ID <> tblPhoneNumbers.ID
);
-- Show what is now in the swap table
SELECT * FROM swapPhoneNumbers;
-- Update the original table with the new phone numbers
UPDATE tblPhoneNumbers SET PhoneNumber = (
SELECT replacementPhoneNumber FROM swapPhoneNumbers
WHERE tblPhoneNumbers.ID = swapPhoneNumbers.ID
);
-- Show what is in the original table
SELECT * FROM tblPhoneNumbers;
And this is some screen shots from doing it

Related

Insert the id from a previous query inside a sqlite trigger

In a sqlite3 database I would like to create a trigger on a view so that I can insert data over the view. Inside the trigger I would like to insert something in the tblmedia table. The id of the inserted row should be now also inserted into the tblbook as id.
In sqlite there are no variables. Otherwise I would store the returning value in the variable and would use it in the second query.
Can this even be achieved in sqlite?
Following my sql schema:
CREATE TABLE tblmedia(
id INTEGER PRIMARY KEY NOT NULL,
title VARCHAR NOT NULL,
raiting INTEGER,
file_name VARCHAR NOT NULL,
media_type TEXT NOT NULL
);
CREATE TABLE tblbook(
id INTEGER PRIMARY KEY NOT NULL,
author VARCHAR,
FOREIGN KEY (id) REFERENCES tblmedia(id) ON DELETE CASCADE
);
CREATE VIEW book AS
SELECT
m.id as id,
m.title as title,
b.author as author,
m.raiting as raiting,
m.file_name as file_name
FROM tblbook b
LEFT JOIN tblmedia m ON m.id = b.id;
CREATE TRIGGER insert_book
INSTEAD OF INSERT ON book
BEGIN
INSERT INTO tblmedia(title, raiting, file_name)
VALUES(new.title, new.raiting, new.file_name);
INSERT INTO tblbook(id, author)
VALUES (xx, new.author); -- xx should be the id from the previous insert
END

Create composed unique key field with trigger after insert in SQLITE

I have a table with multiple columns and one (unique key) should be a value composed from the values of other two columns.
CREATE TABLE batches (
id TEXT PRIMARY KEY UNIQUE,
name TEXT NOT NULL,
project_id INTEGER);
On each insert, I want to generate the id based on the value of 'name' and 'project_id' (this one can be null):
INSERT INTO batches (name,project_id) VALUES
('21.01',NULL),
('21.01',1),
('21.02',2);
So, I have created a table TRIGGER but doesn't execute.
CREATE TRIGGER create_batches_id
AFTER INSERT ON batches FOR EACH ROW
BEGIN
UPDATE batches
SET id = SELECT quote(name ||"_"|| (CASE project_id
WHEN NULL THEN '' ELSE project_id END )
FROM batches WHERE rowid = (SELECT MAX(rowid) FROM batches))
WHERE rowid = (SELECT MAX(rowid) FROM batches);
END;
Error:
SQL Error [1]: [SQLITE_ERROR] SQL error or missing database (near "SELECT": syntax error)
I expect:
id = 21.01_
id = 21.01_1
id = 21.01_2
What am I doing wrong? If I run only the SELECT/CASE statment it returns ok: '21.01_2'
I have also tried without the quote() function, no success.
UPDATE I:
I have managed to execute the whole create trigger statement (parenthesis were missing):
CREATE TRIGGER create_batch_id
AFTER INSERT ON batches FOR EACH ROW
BEGIN
UPDATE batches
SET id = (SELECT name ||"_"|| (CASE project_id WHEN NULL THEN 0 ELSE project_id END ) FROM batches WHERE rowid = (SELECT MAX(rowid) FROM batches) )
WHERE rowid = (SELECT MAX(rowid) FROM batches);
END;
It seems my editor (DBeaver) has a glitch with the following new line character. If it is inside the selection it runs into this exception (or I am missing something):
SQL Error [1]: [SQLITE_ERROR] SQL error or missing database (incomplete input)
If I manually select only the above lines (from CREATE to ;), the trigger is created, however, not the expected result. If value in project_id is NULL, no id value is created.
Don't add the column id in the table.
Instead define the combination of name and project_id as the PRIMARY KEY of the table, so that it is also UNIQUE:
CREATE TABLE batches (
name TEXT NOT NULL,
project_id INTEGER,
PRIMARY KEY(name, project_id)
)
Then, whenever you need that id you can run a query:
SELECT name || '_' || COALESCE(project_id, '') AS id,
name,
project_id
FROM batches
Or create a view:
CREATE VIEW v_batches AS
SELECT name || '_' || COALESCE(project_id, '') AS id,
name,
project_id
FROM batches
and query the view:
SELECT * FROM v_batches
See the demo.
Or if your version of SQLite is 3.31.0+ you can have the column id as a generated column:
CREATE TABLE batches (
name TEXT NOT NULL,
project_id INTEGER,
id TEXT GENERATED ALWAYS AS (name || '_' || COALESCE(project_id, '')),
PRIMARY KEY(name, project_id)
);

Sqlite INSERTION optimization

Good day SO,
I'm working on a program in PowerShell to manipulate an SQLite DB I created. I've never written a serious applications to utilize a DB so right now I'm super interested in optimizing my querys, so I'm really interested in feed back. My primary issue is I have a lot of data that I want to include in a separate table that may or not exist already. All my research really seemed to lead to perform an INSERT and let the UNIQUE constraints sort it out, than do a select on the new record which seemed like two table scans and inefficient. So my solution was Create a temp table, insert into the temp table FROM the table with data I want and perform an INSERT if the data was not in the temporary table. I'm a few drinks in tonight and haven't tested the code so please don't critique small typos, I just want to know if my methodology is out to lunch, and if so please provide better direction.
My table is as shown:
CREATE TABLE Processes (
pk INTEGER PRIMARY KEY AUTOINCREMENT
UNIQUE,
hostname INTEGER NOT NULL,
artifacttype INTEGER REFERENCES ArtifactType (pk),
processname INTEGER REFERENCES ProcessesName (pk),
filelocation INTEGER NOT NULL
REFERENCES files (pk),
pid INTEGER,
ppid INTEGER,
starttime INTEGER,
stoptime INTEGER,
token STRING,
logonid INTEGER,
exitstatus INTEGER,
threadcount INTEGER,
commandline INTEGER REFERENCES ProcessesCommandline (pk),
user INTEGER REFERENCES users (pk),
PeakVirtualSize INTEGER,
VirtualSize INTEGER,
PeakWorkingSetSize INTEGER,
suspicious BOOLEAN,
malicious BOOLEAN
);
Transaction:
#"
CREATE TEMPORARY TABLE IF NOT EXISTS Results(pk INTEGER, data TEXT);
INSERT INTO Results(pk, data) VALUES ((SELECT pk, name FROM ProcessesName WHERE name = #processname));
INSERT INTO ProcessesName(name) VALUES (SELECT #processname WHERE NOT EXISTS (SELECT pk FROM Results WHERE data = #processname));
INSERT INTO Results (pk, data) VALUES ((SELECT last_insert_row_id, #processname WHERE NOT EXISTS (SELECT pk FROM Results WHERE data =#processname)));
INSERT INTO Results(pk, data) VALUES ((SELECT pk, file FROM Files WHERE file = #filelocation));
INSERT INTO Files(file) VALUES (SELECT #filelocation WHERE NOT EXISTS (SELECT pk FROM Results WHERE data = #filelocation));
INSERT INTO Results (pk, data) VALUES ((SELECT last_insert_row_id, #filelocation WHERE NOT EXISTS (SELECT pk FROM Results WHERE data =#filelocation)));
INSERT INTO Results(pk, data) VALUES ((SELECT pk, commandline FROM ProcessesCommandline WHERE commandline = #commandline));
INSERT INTO ProcesseCommandline(commandline) VALUES (SELECT #commandline WHERE NOT EXISTS (SELECT pk FROM Results WHERE data = #commandline));
INSERT INTO Results (pk, data) VALUES ((SELECT last_insert_row_id, #filelocation WHERE NOT EXISTS (SELECT pk FROM Results WHERE data =#commandline)));
INSERT INTO Results(pk, data) VALUES ((SELECT pk, SID FROM Users WHERE SID = #SID));
INSERT INTO Users(SID) VALUES (SELECT #SID WHERE NOT EXISTS (SELECT pk FROM Results WHERE data = #filelocation));
INSERT INTO Results (pk, data) VALUES ((SELECT last_insert_row_id, #SID WHERE NOT EXISTS (SELECT pk FROM Results WHERE data =#SID)));
INSERT INTO processes(hostname, artifacttype, processname, filelocation, pid, ppid, starttime, threadcount, commandline, user, PeakVirtualSize, VirtualSize, PeakWorkingSetSize)
VALUES (#hostname, #artifacttype, (SELECT pk FROM Results WHERE data = #processname), (SELECT pk FROM Results WHERE data #filelocation), #pid, #ppid, #starttime, #threadcount, (SELECT pk FROM Results where data = #commandline), SELECT pk FROM Results WHERE data = #SID, #PeakVirtualSize, #VirtualSize, #PeakWorkingSetSize);
DROP TABLE Results;
"#
*there are a few foreign keys where the data is being tracked application side so no complex queries are required.
So my core question is, is there a more efficient way to do this?
Thanks guys!

Read last Inserted row in Sql according to its Time stamp

sql has a Table called emp.
emp(emp_id int IDENTITY primary key, EmployeeName varchar(50),.......)
I want to Insert a record to above table. Here is my code in asp.net.
DBconnection dbcon = new DBconnection();
string query = "insert into emp values('" + TextBox_EmpName.Text + "','" + ....);
int no1 = dbcon.insertQuery(query);
I have another table called emp-relation
emp-relation(emp_id int primary key, count int, ....)
-- foreign key (emp_id)references emp(emp_id)
My problem is when I inserting the emp row ,I dont know what is the emp_id since it created by auto. And when I am going to insert to emp-relation , I want to get emp-id since it is the foreign key.
How can I do this? Is there any way to read last Insert row in Sql according to Time stamp or some thing? I believe that records are not sorted according to inserted timestamp in nature. please help me.
There's bascally two ways. The first way is to return the new ID from the first insert query:
insert into emp values(...)
select scope_identity() as NewID
The second way is to lookup the first row when you insert into the relation table:
insert emp-relation
(emp_idm, ...)
select emp_id
, ...
from emp
where emp_name = #EmpName
You have to pass in enough columns to make the reference unique.

Refactor SQLite Table by splitting it in two and link with foreign keys

I'm working on a SQLite Database. The database is already filled, but I want to refactor it. Here is a sample of what I need to do:
I currently have one table:
CREATE TABLE Cars (ID INTEGER PRIMARY KEY,
Name VARCHAR(32),
TopSpeed FLOAT,
EngineCap FLOAT);
I want to split this into two tables:
CREATE TABLE Vehicles (ID INTEGER PRIMARY KEY,
Name VARCHAR(32),
TopSpeed FLOAT);
CREATE TABLE Cars (ID INTEGER PRIMARY KEY,
VehicleID INTEGER CONSTRAINT FK_Cars REFERENCES [Vehicles](ID),
EngineCap FLOAT);
I have figured out to create a temporary table with the Cars table contents, and I can fill up the Vehicles table with the contents of the Cars table:
CREATE TEMPORARY TABLE Cars_temp AS SELECT * FROM Cars;
INSERT INTO Vehicles (Name, TopSpeed)
SELECT Name, TopSpeed FROM Cars_temp;
But I am still looking for a way to go over that same selection, while putting the EngineCap field into the new Cars table and somehow extracting the corresponding ID value from the Vehicles table to put into the VehicleID foreign key field on the Cars table.
I'm open for workaround or alternative approaches.
Thanks.
Since #mateusza did not provide an example, I've made one:
Suppose you have this table:
CREATE TABLE [Customer] (
[name] TEXT,
[street] TEXT,
[city] TEXT);
Now you want to move street and city into a separate table Address, so you'll end up with two tables:
CREATE TABLE [Customer2] (
[name] TEXT,
[addr] INTEGER);
CREATE TABLE [Address] (
[rowid] INTEGER NOT NULL,
[street] TEXT,
[city] TEXT,
PRIMARY KEY ([rowid])
);
(For this example, I'm doing the conversion in the same database. You'd probably use two DBs, converting one into the other, with an SQL ATTACH command.)
Now we create a view (which imitates our original table using the new tables) and the trigger:
CREATE VIEW Customer1 (name, street, city) AS
SELECT C.name, A.street, A.city FROM Customer2 AS C
JOIN Address as A ON (C.addr == A.rowid);
CREATE TEMP TRIGGER TempTrig INSTEAD OF INSERT ON Customer1 FOR EACH ROW BEGIN
INSERT INTO Address (street, city) SELECT NEW.street, NEW.city;
INSERT INTO Customer2 (addr, name) SELECT last_insert_rowid(), NEW.name;
END;
Now you can copy the table rows:
INSERT INTO Customer1 (name, street, city) SELECT name, street, city FROM Customer;
The above is a simplified case where you'd only move some data into a single new table.
A more complex (and more general) case is where you want to...
Separate your original table's columns into several foreign tables, and
Have unique entries in the foreign tables (that's usually the reason why you'd refactor your table).
This adds some additional challenges:
You'll end up inserting into multiple tables before you can insert their rowids into the table with the referencing rowids. This requires storing the results of each INSERT's last_insert_rowid() into a temporary table.
If the value already exists in the foreign table, its rowid must be stored instead of the one from the (non-executed) insertion operation.
Here's a complete solution for this. It manages a database of music records, constisting of a song's name, album title and artist name.
-- Original table
CREATE TABLE [Song] (
[title] TEXT,
[album] TEXT,
[artist] TEXT
);
-- Refactored tables
CREATE TABLE [Song2] (
[title] TEXT,
[album_rowid] INTEGER,
[artist_rowid] INTEGER
);
CREATE TABLE [Album] (
[rowid] INTEGER PRIMARY KEY AUTOINCREMENT,
[title] TEXT UNIQUE
);
CREATE TABLE [Artist] (
[rowid] INTEGER PRIMARY KEY AUTOINCREMENT,
[name] TEXT UNIQUE
);
-- Fill with sample data
INSERT INTO Song VALUES ("Hunting Girl", "Songs From The Wood", "Jethro Tull");
INSERT INTO Song VALUES ("Acres Wild", "Heavy Horses", "Jethro Tull");
INSERT INTO Song VALUES ("Broadford Bazar", "Heavy Horses", "Jethro Tull");
INSERT INTO Song VALUES ("Statue of Liberty", "White Music", "XTC");
INSERT INTO Song VALUES ("Standing In For Joe", "Wasp Star", "XTC");
INSERT INTO Song VALUES ("Velvet Green", "Songs From The Wood", "Jethro Tull");
-- Conversion starts here
CREATE TEMP TABLE [TempRowIDs] (
[album_id] INTEGER,
[artist_id] INTEGER
);
CREATE VIEW Song1 (title, album, artist) AS
SELECT Song2.title, Album.title, Artist.name
FROM Song2
JOIN Album ON (Song2.album_rowid == Album.rowid)
JOIN Artist ON (Song2.artist_rowid == Artist.rowid);
CREATE TEMP TRIGGER TempTrig INSTEAD OF INSERT ON Song1 FOR EACH ROW BEGIN
INSERT OR IGNORE INTO Album (title) SELECT NEW.album;
UPDATE TempRowIDs SET album_id = (SELECT COALESCE (
(SELECT rowid FROM Album WHERE changes()==0 AND title==NEW.album), last_insert_rowid()
) ) WHERE rowid==1;
INSERT OR IGNORE INTO Artist (name) SELECT NEW.artist;
UPDATE TempRowIDs SET artist_id = (SELECT COALESCE (
(SELECT rowid FROM Artist WHERE changes()==0 AND name==NEW.artist), last_insert_rowid()
) ) WHERE rowid==1;
INSERT INTO Song2 (title, album_rowid, artist_rowid) SELECT
NEW.title, (SELECT album_id FROM TempRowIDs), (SELECT artist_id FROM TempRowIDs);
END;
INSERT INTO TempRowIDs DEFAULT VALUES;
INSERT INTO Song1 (title, album, artist) SELECT title, album, artist FROM Song;
DROP TRIGGER TempTrig;
DROP TABLE TempRowIDs;
-- Conversion ends here
-- Print results
SELECT * FROM Song;
SELECT * FROM Song1;
-- Check if original and copy are identical (https://stackoverflow.com/a/13865679/43615)
SELECT CASE WHEN (SELECT COUNT(*) FROM (SELECT * FROM Song UNION SELECT * FROM Song1)) == (SELECT COUNT() FROM Song) THEN 'Success' ELSE 'Failure' END;
Note that this example has one potential issue: If the constraints on the foreign table are more complex, the SELECT rowid FROM search for the existing entry needs to be updated accordingly. Ideally, SQLite should provide a way to determine the conflicting rowid somehow, but it doesn't, unfortunately (see this related question).
Simple solution without triggers:
create VEHICLES_TEMP table including the CAR_ID
create your new CARS table without the VEHICLES columns you don't want
update CARS with VEHICLE_ID taken from VEHICLES_TEMP (identified by the CAR_ID)
create final VEHICLES table without the CAR_ID
Create a table New_Cars and a INSTEAD OF INSERT trigger, which will insert data to both tables Vehicles and Cars. When inserting to Cars, you can use last_insert_rowid() function to refer to inserted row in Vehicles table.
This can be temporary solution, or you can leave it in your database for further modifications.

Resources