SQLite and multiple insert clean - sqlite

I would like to populate a freshly created Table in a SQLite DB.
In this table, some keys are references to other tables and I'd like not to hard-code these references
-> I'm currently using a "mapping" table in order to fetch ids using names (~ constants emulation)
The problem is: this solution works but is very verbose
Minimal working example: (storing dictionary words, using foreign keys to a category table)
-- Tables creation
CREATE TABLE categories(
id INTEGER PRIMARY KEY,
name TEXT
);
CREATE TABLE words(
id INTEGER PRIMARY KEY,
id_category INTEGER NOT NULL,
name TEXT,
FOREIGN KEY(id_category) REFERENCES categories(id)
);
CREATE TABLE CONSTANTS(
name TEXT PRIMARY KEY,
value INTEGER NOT NULL
);
INSERT INTO categories(name) VALUES("noun");
INSERT INTO CONSTANTS(name, value) VALUES("category_noun", last_insert_rowid());
INSERT INTO categories(name) VALUES("abreviation");
INSERT INTO CONSTANTS(name, value) VALUES("category_abreviation", last_insert_rowid());
INSERT INTO categories(name) VALUES("character");
INSERT INTO CONSTANTS(name, value) VALUES("category_character", last_insert_rowid());
And now, the core of the problem: too much verbose.
In this example is only one foreign key, a few insert to illustrate the problem
INSERT INTO words(id_category, name) VALUES
((SELECT value FROM CONSTANTS WHERE name = "category_noun"),
"hello"),
((SELECT value FROM CONSTANTS WHERE name = "category_abreviation"),
"SO"),
((SELECT value FROM CONSTANTS WHERE name = "category_abreviation"),
"user"),
((SELECT value FROM CONSTANTS WHERE name = "category_character"),
"!")
;
I would like to have something looking like this pseudo-sqlite code:
-- same table creations as before
INSERT INTO words(id_category, name) VALUES
-- Fetch constants once
CAT_NOUM = SELECT value FROM CONSTANTS WHERE name = "category_noum"),
CAT_ABREV = SELECT value FROM CONSTANTS WHERE name = "category_abreviation"),
CAT_CHAR = SELECT value FROM CONSTANTS WHERE name = "category_abreviation")
)
-- Fill the table, using constants
(CAT_NOUM, "Hello"),
(CAT_ABREV, "SO"),
(CAT_NOUM, "user"),
(CAT_CHAR, "SO"),
...
;
I'm wondering if
There is already a SQLite solution to this problem
I should use something like sed to replace a hard-coded string like __SED__CAT_NOUM with its greped value in the SQLite script
Doing this stuff programmatically would be the right way

It is better to use INSERT...SELECT with UNION ALL instead of INSERT...VALUES:
INSERT INTO words(id_category, name)
SELECT value, 'hello' FROM CONSTANTS WHERE name = 'category_noun' UNION ALL
SELECT value, 'SO' FROM CONSTANTS WHERE name = 'category_abreviation' UNION ALL
SELECT value, 'user' FROM CONSTANTS WHERE name = 'category_abreviation' UNION ALL
SELECT value, '!' FROM CONSTANTS WHERE name = 'category_character';
See the demo.
Or use Row Values to join to CONSTANTS:
INSERT INTO words(id_category, name)
SELECT c.value, t.column2
FROM CONSTANTS C INNER JOIN (
VALUES ('category_noun', 'hello'),
('category_abreviation', 'SO'),
('category_abreviation', 'user'),
('category_character', '!')
) t ON t.column1 = c.name;
See the demo.
Results:
SELECT * FROM words;
| id | id_category | name |
| --- | ----------- | ----- |
| 1 | 1 | hello |
| 2 | 2 | SO |
| 3 | 2 | user |
| 4 | 3 | ! |

Related

MARIADB sequences - incrementing by 2

I have the following MARIADB code. It's supposed to demonstrate:
Constructing tables using sequences for incrementing the ID.
Using a temporary table+join to INSERT data into a table, while incrementing the ID.
Procedure:
Sequence S1 and table T1 are created. T1_ID is incremented with S1
Sequence S2 and table T2 are created. T2_ID is incremented with S2
Table T1 is filled with data. All is fine.
Temporary table TEMP_T2 is created and filled with data. No ID in this table. Column T1_NAME is a cross reference to SHORT_NAME in table T1.
The T1_ID is introduced into table TEMP_T2 with a join. The result of this SELECT is inserted into T2. Here, the sequence S2 should auto-increment T2_ID.
For some reason, at the end, T2 looks like this:
T2_ID|T1_ID|NAME|
-----+-----+----+
2| 1|y |
4| 2|x |
6| 2|z |
Why was T2_ID double-incremented?
Thanks!
USE DB1;
SET FOREIGN_KEY_CHECKS = 0;
DROP SEQUENCE IF EXISTS `S2`;
DROP SEQUENCE IF EXISTS `S1`;
DROP TABLE IF EXISTS `T2`;
DROP TABLE IF EXISTS `T1`;
-- Create sequence S1 and able T1
CREATE SEQUENCE `S1` start with 1 minvalue 1 maxvalue 9223372036854775806 increment by 1 cache 1000 nocycle ENGINE=InnoDB;
SELECT SETVAL(`S1`, 1, 0);
CREATE TABLE `T1` (
`T1_ID` tinyint(4) NOT NULL DEFAULT nextval(`S1`),
`SHORT_NAME` varchar(10) NOT NULL,
PRIMARY KEY (`T1_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
-- Create sequence T2 and table T2
CREATE SEQUENCE `S2` start with 1 minvalue 1 maxvalue 9223372036854775806 increment by 1 cache 1000 nocycle ENGINE=InnoDB;
SELECT SETVAL(`S2`, 1, 0);
CREATE TABLE `T2` (
`T2_ID` int(11) NOT NULL DEFAULT nextval(`S2`),
`T1_ID` int(11) DEFAULT NULL,
`NAME` varchar(100) DEFAULT NULL COLLATE 'utf8mb3_bin',
PRIMARY KEY (`T2_ID`),
UNIQUE KEY `T2_NAME_UN` (`NAME`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
-- Load data into T1
DELETE FROM T1;
INSERT INTO T1(SHORT_NAME) VALUES
('a'),
('b'),
('c');
SELECT * FROM T1;
-- Create temporary file for joining with T1
DROP TABLE IF EXISTS `TEMP_T2`;
CREATE TEMPORARY TABLE `TEMP_T2` (
`T1_NAME` varchar(10) DEFAULT NULL,
`NAME` varchar(100) DEFAULT NULL,
UNIQUE KEY `T2_NAME_UN` (`NAME`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
DELETE FROM TEMP_T2 ;
-- Insert data into the temporary table
INSERT INTO TEMP_T2(T1_NAME,NAME) VALUES
('b','x'),
('a','y'),
('b','z');
SELECT * FROM TEMP_T2;
# Do a join with TEMP_T2 x T1 and insert into T2
INSERT INTO T2(T1_ID,NAME)
SELECT
t1.T1_ID ,
t2.NAME
FROM TEMP_T2 AS t2
INNER JOIN T1 AS t1
ON t2.T1_NAME =t1.SHORT_NAME ;
SELECT * FROM T2;
Thanks for the responses.
I'm using SEQUENCE rather than AUTO_INCREMENT because I was told that it is the more modern way. It also enables retrieving the last ID of any specific table.
It's strange that this should be a bug. It seems like really basic functionality. But so it is...
I've found this as a reported existing bug MDEV-29540 in INSERT ... SELECT as it pertains to sequences in default values of columns.
Because this bug is reported and fix, this problem is/will not occur in the 10.3.37, 10.4.27, 10.5.18, 10.6.11, 10.7.7, 10.8.6, 10.9.4, 10.10.2, 10.11.1 and later versions.

SQLite UPDATE now working or I have done wrong

Current table questions:
rowid: 1 | 2 | 3 | 4 | 5
id: 3 | 4 | 7 | 9 | 10
Trying to achieve the following:
rowid: 1 | 2 | 3 | 4 | 5
id: 1 | 2 | 3 | 4 | 5
I have tried many different variations of SQL without success, this is the latest I am testing:
UPDATE questions SET id = rowid;
Can someone please suggest how I solve this as I have googled and cannot find the solution?
I don't believe that your question completely encompasses all aspects of the issue.
In theory to have a table (the before table) where SELECT rowid, id results in
rowid: 1 | 2 | 3 | 4 | 5
id: 3 | 4 | 7 | 9 | 10
The id column must not be an alias of the rowid column (otherwise the values would be identical)
However, if the id column is an alias of the rowid column, the both columns would be the same so the before table above would not be as above.
As an example using :-
--<<<<<<<<<< WORKS >>>>>>>>>>
-- as ID is not an alias of the rowid column update changes id column
DROP TABLE IF EXISTS questionsv3;
CREATE TABLE IF NOT EXISTS questionsv3 (ID INTEGER);
INSERT INTO questionsv3 VALUES (3),(4),(7),(9),(10);
SELECT rowid, id FROM questionsv3;
UPDATE questionsv3 SET id = rowid;
SELECT rowid, id FROM questionsv3;
results in the expected result as per :-
First Select (before update) :-
Second Select (after update)
Other potential causes
rowid is not in fact the rowid as per SQLITE, but a conceptual idea that it should be 1,2,3 ...... (in which case using VACUUM, if there is no alias to the rowid, may result in the desired re-numbering of the rowid column, which if followed by the update may then result in the id being re-sequenced).
That the update is done within a transaction that hasn't been committed and is rolled back.
You may wish to consider the following permutations of different table creations (see comments) :-
-- as ID is an alias of rowid, then rowid is set according to ID so update does nothing
DROP TABLE IF EXISTS questionsv1;
CREATE TABLE IF NOT EXISTS questionsv1 (ID INTEGER PRIMARY KEY);
INSERT INTO questionsv1 VALUES (3),(4),(7),(9),(10);
SELECT rowid, id FROM questionsv1;
UPDATE questionsv1 SET id = rowid;
SELECT rowid, id FROM questionsv1;
-- as ID is an alias of the rowid column, then rowid is set according to the ID so update does nothing
DROP TABLE IF EXISTS questionsv2;
CREATE TABLE IF NOT EXISTS questionsv2 (ID INTEGER PRIMARY KEY AUTOINCREMENT);
INSERT INTO questionsv2 VALUES (3),(4),(7),(9),(10);
SELECT rowid, id FROM questionsv2;
UPDATE questionsv2 SET id = rowid;
SELECT rowid, id FROM questionsv2;
--<<<<<<<<<< WORKS >>>>>>>>>>
-- as ID is not an alias of the rowid column update changes id column
DROP TABLE IF EXISTS questionsv3;
CREATE TABLE IF NOT EXISTS questionsv3 (ID INTEGER);
INSERT INTO questionsv3 VALUES (3),(4),(7),(9),(10);
SELECT rowid, id FROM questionsv3;
UPDATE questionsv3 SET id = rowid;
SELECT rowid, id FROM questionsv3;
--<<<<<<<<<< WORKS >>>>>>>>>>
-- as ID is not an alias of rowid the ID column is updated accordingly
DROP TABLE IF EXISTS questionsv4;
CREATE TABLE IF NOT EXISTS questionsv4 (ID TEXT PRIMARY KEY); -- not an alias of rowid
INSERT INTO questionsv4 VALUES (3),(4),(7),(9),(10);
SELECT rowid, id FROM questionsv4;
UPDATE questionsv4 SET id = rowid;
SELECT rowid, id FROM questionsv4;
--<<<<<<<<<< FAILS >>>>>>>>>>
DROP TABLE IF EXISTS questionsv13;
CREATE TABLE IF NOT EXISTS questionsv13 (ID INTEGER PRIMARY KEY) WITHOUT ROWID;
INSERT INTO questionsv13 VALUES (3),(4),(7),(9),(10);
SELECT id FROM questionsv13;
UPDATE questionsv13 SET id = rowid; -- would fail no such column
SELECT id FROM questionsv13;

Select several event params in a single row for Firebase events stored in Google BigQuery

I'm trying to perform a very simple query for Firebase events stored in Google BigQuery but I´m not able to find a way to do it.
In the Android app, I´m logging an event like this:
Bundle params = new Bundle();
params.putInt("productID", productId);
params.putInt(FirebaseAnalytics.Param.VALUE, value);
firebaseAnalytics.logEvent("productEvent", params);
So, in BigQuery I have something like this:
___________________ _______________________ ____________________________
| event_dim.name | event_dim.params.key | event_dim.params.int_value |
|___________________|_______________________|____________________________|
| productEvent | productID | 25 |
| |_______________________|____________________________|
| | value | 1253 |
|___________________|_______________________|____________________________|
When I get the data from this table I get two rows:
___________________ _______________________ ____________________________
|event_dim.name | event_dim.params.key | event_dim.params.int_value |
|___________________|_______________________|____________________________|
| productEvent | productID | 25 |
| productEvent | value | 12353 |
But what I really need is a SELECT clause from this table to get the data as below:
___________________ _____________ _________
| name | productID | value |
|___________________|_____________|_________|
| productEvent | 25 | 12353 |
Any idea or suggestion?
You can pivot the values into columns like this
SELECT
event_dim.name as name,
MAX(IF(event_dim.params.key = "productID", event_dim.params.int_value, NULL)) WITHIN RECORD productID,
MAX(IF(event_dim.params.key = "value", event_dim.params.int_value, NULL)) WITHIN RECORD value,
FROM [events]
In case you want to generate this command using SQL, see this solution: Pivot Repeated fields in BigQuery
Using standard SQL (uncheck "Use Legacy SQL" under "Show Options" in the UI), you can express the query as:
SELECT
event_dim.name as name,
(SELECT value.int_value FROM UNNEST(event_dim.params)
WHERE key = "productID") AS productID,
(SELECT value.int_value FROM UNNEST(event_dim.params)
WHERE key = "value") AS value
FROM `dataset.mytable` AS t,
t.event_dim AS event_dim;
Edit: updated example to include int_value as part of value based on the comment below. Here is a self-contained example that demonstrates the approach as well:
WITH T AS (
SELECT ARRAY_AGG(event_dim) AS event_dim
FROM (
SELECT STRUCT(
"foo" AS name,
ARRAY<STRUCT<key STRING, value STRUCT<int_value INT64, string_value STRING>>>[
("productID", (10, NULL)), ("value", (5, NULL))
] AS params) AS event_dim
UNION ALL
SELECT STRUCT(
"bar" AS name,
ARRAY<STRUCT<key STRING, value STRUCT<int_value INT64, string_value STRING>>>[
("productID", (13, NULL)), ("value", (42, NULL))
] AS params) AS event_dim
)
)
SELECT
event_dim.name as name,
(SELECT value.int_value FROM UNNEST(event_dim.params)
WHERE key = "productID") AS productID,
(SELECT value.int_value FROM UNNEST(event_dim.params)
WHERE key = "value") AS value
FROM T AS t,
t.event_dim AS event_dim;

confused on how to properly use %rowtype for many tables

I'm attempting to create a derived table of country data from several other tables. Those tables look something like this:
Countries
ID | Name
Country_demographics
ID | date | Population | urban_pop | birth_rate
country_financials
ID | date | GDP | GDP_per_capita
Now, I'm trying to make a new table with
New_Table
ID | Name | date | population | urban_pop | birth_rate | gdp | gdp_per_capita
I have a stored procedure that currently looks something like this:
CREATE OR REPLEACE PROCEDURE SP_COUNTRY (
chunkSize IN INT
) AS
--create tables to hold IDs and stats
TYPE idTable IS TABLE OF COUNTRIES.ID%TYPE;
TYPE dateTable IS TABLE OF COUNTRY_DEMOGRAPHICS.EVALUATION_DATE%TYPE;
TYPE totPopTable IS TABLE OF COUNTRY_DEMOGRAPHICS.POPULATION_TOTAL_COUNT%TYPE;
TYPE urbanPopTable IS TABLE OF COUNTRY_DEMOGRAPHICS.POPULATION_URBAN_COUNT%TYPE;
--constructors
ids idTable;
dates dateTable;
totpop totPopTable;
urbanpop urbanPopTable;
--cursors
CURSOR countryCur IS
SELECT c.ID,cd.EVALUATION_DATE,cd.POPULATION_TOTAL_COUNT,cd.POPULATION_URBAN_COUNT
FROM COUNTRIES c,COUNTRY_DEMOGRAPHICS cd
WHERE c.id=cd.COUNTRY_ID
ORDER BY ID,EVALUATION_DATE;
BEGIN
dbms_output.enable(999999);
--open cursor
OPEN countryCur;
LOOP
--fetch and bulk collect
FETCH countryCur BULK COLLECT INTO ids,dates,totpop,urbanpop
LIMIT chunkSize;
--loop over collections
FOR j in ids.FIRST..ids.LAST
LOOP
--populate record
country.COUNTRY_ID := ids(j);
country.EVALUATION_DATE := dates(j);
country.POPULATION_TOTAL_COUNT := totpop(j);
country.POPULATION_URBAN_COUNT := urbanpop(j);
--update/insert table with record (much confusion here on how to update/insert and check if already exists in derived table..)
UPDATE NEW_TABLE SET ROW = country WHERE COUNTRY_ID = ids(j);
dbms_output.put_line('id: ' || country.COUNTRY_ID || ' date: ' || country.EVALUATION_DATE);
dbms_output.put_line(' pop: ' || country.POPULATION_TOTAL_COUNT || ' urban: ' || country.POPULATION_URBAN_COUNT);
END LOOP;
END LOOP;
--close cursor
CLOSE countryCur;
END;
As you can see, I'm using a different table type for each piece of data. I then plan on making a loop and then just inserting/updating in my new_table. I think there must be a better way to do this with %rowtype, or maybe creating a record and inserting the record? I'm not sure
Unless I'm missing something by simplifying this, and assuming cd.date and cf.date are equal, this should work:
INSERT INTO NEW_TABLE (ID, Name, date, population, urban_pop, birth_rate, gdp, gdp_per_capita)
values
(select c.id, c.name, cd.date,
cd.population, cd.urban_pop, cd.birthrate,
cf.gdp, cf.gdp_per_capita)
from Countries c, country_demographics cd, country_financials cf
where c.id = cd.id
and cd.id = cf.id);
Edit: Use the MERGE statement to update or insert depending on if the primary key exists:
MERGE INTO NEW_TABLE nt
USING ( select c.id, c.name, cd.date,
cd.population, cd.urban_pop, cd.birthrate,
cf.gdp, cf.gdp_per_capita
from Countries c, country_demographics cd, country_financials cf
where c.id = cd.id
and cd.id = cf.id ) a
ON (nt.id = a.id )
WHEN MATCHED THEN
UPDATE SET nt.Name = a.Name,
nt.date = a.date,
nt.population = a.population,
nt.urban_pop = a.urban_pop,
nt.birth_rate = a.birth_rate,
nt.gdp = a.gdp,
nt.gdp_per_capita = a.gdp_per_capita
WHEN NOT MATCHED THEN
INSERT (ID, Name, date, population, urban_pop, birth_rate, gdp, gdp_per_capita)
VALUES (a.id, a.Name, a.date, a.population, a.urban_pop, a.birth_rate, a.gdp, a.gdp_per_capita);

How to use join query to fetch data from more than two tables without foreign key and primary key

Once I need your help,
I have three table in my database.
1> wedding
Column Name Data Type
-------------------------------------
wedID Int (primary key)
wedName varchar (50)
2> selectSite
Column Name Data Type
-----------------------------------
wedID Int (without foreign key and primary key with null allow)
siteID Int (without foreign key and primary key with null allow)
siteStatus varchar(50)
3> webSite
Column Name Data Type
--------------------------------
siteID Int (Primary Key)
siteName varchar(50)
siteFile varchar(MAX)
I want the following output:
ws.siteID ws.siteName ws.siteFile s.wedID s.sitestatus
----------------------------------------------------------------------------
1 Show show.jpeg 6 Yes
2 My View my view.png 5
3 Dream dream.jpeg 3
ws is alias of webSite table, s is alias of siteSelect table.
All data from webSite table and only data from siteSelect where s.siteStatus do not display Yes if wedID not match
I try following sql query
select
DISTINCT(ws.siteID), s.wedID, ws.siteName,
ws.siteFile, s.siteStatus
from wedding wd, webSite ws
left outer join siteSelect s on ws.siteID = s.siteID
where wd.wedID = #wedID
but output is like this:
ws.siteID ws.siteName ws.siteFile s.wedID s.sitestatus
-----------------------------------------------------------------------
1 Show show.jpeg 6 Yes
2 My View my view.png 5 Yes
3 Dream dream.jpeg 3 Yes
DEMO SQL FIDDLE
Check the demo add some values as you having and run this query
select
ws.siteId, ws.siteName, ws.siteFile, ss.wedID,
(case
when ss.wedID = 1 then ss.siteStatus else '' end) as siteStatus
from
website ws
join
selectsite ss on ss.siteID=ws.siteID
let me knw if any prob
try this
select
DISTINCT(ws.siteID), s.wedID, ws.siteName, ws.siteFile,
(case
when s.wedID = #wedID
then s.siteStatus
else '' end) as siteStatus, wd.wedName
from webSite ws
left join selectSite s on ws.siteID = s.siteID
Left Join wedding wd on wd.wedID = s.wedID

Resources