At work I've been given a task that I'm told is "simple and straightforward", but I'm having difficulty with:
I have a view that contains 4 columns, a PK, FK, comments, and column #4. My manager is telling me to make a table that contains 75 or so keywords, and then a query that will go through each row in the view, compare the comments to the keyword table, and then append each found keyword to column #4. I've searched google and SO, and have not found a query that would do this. Any help would be appreciated.
Try below. I am new to Teradata.
--***************************************************************
DROP TABLE x;
--***************************************************************
CREATE MULTISET VOLATILE TABLE x, NO FALLBACK,
CHECKSUM = DEFAULT,
LOG
(
RCD_ID INTEGER,
FK INTEGER,
CMT VARCHAR(200),
COL_4 VARCHAR(200),
RN INTEGER
)
PRIMARY INDEX (RCD_ID)
ON COMMIT PRESERVE ROWS;
--***************************************************************
INSERT INTO x VALUES (1,10,'DID YOU SEE THE COW?','',0);
INSERT INTO x VALUES (2,20,'DID YOU SEE THE CAT?','',0);
INSERT INTO x VALUES (3,30,'DID YOU SEE THE FOX?','',0);
INSERT INTO x VALUES (4,40,'DID YOU SEE THE GOAT, FOX, AND CAT?','',0);
INSERT INTO x VALUES (5,50,'DID YOU SEE THE DUCK AND THE COW?','',0);
--***************************************************************
SELECT * FROM x ORDER BY 1;
--***************************************************************
DROP TABLE y;
--***************************************************************
CREATE MULTISET VOLATILE TABLE y, NO FALLBACK,
CHECKSUM = DEFAULT,
LOG
(
RCD_ID INTEGER,
KEY_WORD VARCHAR(20)
)
PRIMARY INDEX (RCD_ID)
ON COMMIT PRESERVE ROWS;
--***************************************************************
INSERT INTO y VALUES (1,'COW');
INSERT INTO y VALUES (2,'CAT');
INSERT INTO y VALUES (3,'FOX');
INSERT INTO y VALUES (4,'GOAT');
INSERT INTO y VALUES (5,'DUCK');
--***************************************************************
SELECT * FROM y ORDER BY 1;
--***************************************************************
DROP TABLE z;
--***************************************************************
CREATE MULTISET VOLATILE TABLE z AS(
SELECT x.RCD_ID,x.CMT,x.COL_4,y.key_word, ROW_NUMBER() OVER(PARTITION BY x.RCD_ID ORDER BY x.RCD_ID) AS RN
FROM x JOIN y ON x.cmt LIKE '%' || y.KEY_WORD || '%'
)
WITH DATA PRIMARY INDEX (RCD_ID)
ON COMMIT PRESERVE ROWS;
--***************************************************************
SELECT * FROM z ORDER BY 1,5;
--***************************************************************
WITH RECURSIVE RPT AS(
SELECT
RCD_ID,FK,CMT,COL_4,RN
FROM x
UNION ALL
SELECT
b.RCD_ID,b.FK,b.CMT,b.COL_4 || ';' || a.KEY_WORD,a.RN
FROM z AS a
JOIN RPT AS b
ON b.RCD_ID = a.RCD_ID
AND b.RN = a.RN-1
)
SELECT *
FROM RPT
QUALIFY ROW_NUMBER() OVER (PARTITION BY RCD_ID ORDER BY RCD_ID, RN DESC) = 1
ORDER BY 1,5;
--***************************************************************
Related
Want to run an insert n-times, the n is defined by inserted value on another table.
table 1:
create table Table1
(
id INTEGER default 1 not null
primary key
unique,
name TEXT not null
unique,
keys_number INTEGER default 1 not null
);
table 2:
create table table_2
(
id INTEGER default 1 not null
primary key
unique,
table_1_id INTEGER not null
references table_1,
);
Now when i insert into table_1 like:
INSERT INT table_1 (name, keys_number) VALUES ('dummy_name', 5)
want to run a trigger
CREATE TRIGGER insert_keys_after_insert_table_1
AFTER INSERT ON table_1
WHEN (SELECT COUNT(*) FROM table_2 WHERE table_1_id=new.id) < new.keys_number
BEGIN
INSERT INTO keys (table_1_id) VALUES (new.id);
END;
But this runs only once.
Finally, I want to have on table_2 values after the trigger
INSERT INT table_1 (name, keys_number) VALUES ('dummy_name_1', 5)
make the result like:
id;name;keys_number
1;dummy_name_1;5
and the result on table_2 after the trigger, should be
id;table_1_id
1;1
2;1
3;1
4;1
5;1
or if I insert another one on table_1
INSERT INT table_1 (name, keys_number) VALUES ('dummy_name_2', 2)
will make the result on table_1 like:
id;name;keys_number
1;dummy_name_1;5
2;dummy_name_2;2
and the trigger will give the result on table_2:
id;table_1_id
1;1
2;1
3;1
4;1
5;1
6;2
7;2
For this requirement you need a loop, which in SQL is implemented via a recursive CTE.
Unfortunately SQLite does not allow CTEs inside a trigger.
A workaround is to create a new table with only 1 column which will store the numbers 1 to the max expected value of keys_number in table_1:
CREATE TABLE table_numbers AS
WITH cte(keys_number) AS (
SELECT 1
UNION ALL
SELECT keys_number + 1 FROM cte WHERE keys_number < 100 -- change 100 to the max keys_number you expect
)
SELECT keys_number FROM cte
Now you can create your trigger:
CREATE TRIGGER insert_keys_after_insert_table_1 AFTER INSERT ON table_1
BEGIN
INSERT INTO table_2(table_1_id)
SELECT NEW.id
FROM table_numbers
WHERE keys_number <= NEW.keys_number - (SELECT COUNT(*) FROM table_2 WHERE table_1_id = NEW.id);
END;
See the demo.
I am stuck with the limitations of both SQLite and the design of some of its tables. Here's what I'd like to achieve:
Create a variable to track the number of iterations of a loop. Then use this variable to help create unique table rows during the loop. Also insert it as the literal count value in a record.
Concatenate said variable to existing strings to create unique IDs that can be referenced multiple times.
Loop code using the previous variable to track iterations, and the previous concatenations as IDs for inserting new rows for every loop. 255 loops is an arbitrary number, I doubt I'd need 255 loops in 99.9% of cases, but want to avoid failure in the cases where they are needed. I am realistically looking at 50 loops minimum, with 100 as a rare maximum. 200 would likely be closer to the true maximum outlier number. 255 is just to be safe.
Here is what I have attempted so far:
DECLARE #cnt INT = 1;
WHILE #cnt < 256
BEGIN
#seyield = 'BUILDING_STOCK_EXCHANGE_YIELD_' + #cnt;
#secitizens = 'BUILDING_STOCK_EXCHANGE_CITIZENS_' + #cnt;
#secount = 'COUNT_CITIZENS_' + #cnt;
INSERT INTO
BuildingModifiers (BuildingType, ModifierId)
VALUES
('BUILDING_STOCK_EXCHANGE', #seyield);
INSERT INTO
Modifiers (ModifierId, ModifierType, RunOnce, Permanent, SubjectRequirementSetId)
VALUES
(#seyield, 'MODIFIER_BUILDING_YIELD_CHANGE', 0, 0, #secitizens);
INSERT INTO
ModifierArguments (ModifierID, Name, Value)
VALUES
(#seyield, 'BuildingType', 'BUILDING_STOCK_EXCHANGE'),
(#seyield, 'Amount', '2'),
(#seyield, 'YieldType', 'YIELD_GOLD');
INSERT INTO
RequirementSets(RequirementSetId, RequirementSetType)
VALUES
(#secitizens, 'REQUIREMENT_TEST_ALL');
INSERT INTO
RequirementSetRequirements(RequirementSetId, RequirementId)
VALUES
(#secitizens, #secount);
INSERT INTO
Requirements(RequirementId, RequirementType)
VALUES
(#secount, 'REQUIREMENT_COLLECTION_ATLEAST');
INSERT INTO
RequirementArguments(RequirementId, Name, Value)
VALUES
(#secount, 'CollectionType', 'COLLECTION_CITY_PLOT_YIELDS'),
(#secount, 'Count', #cnt);
SET #cnt = #cnt + 1;
END;
Of course this does not work due to the limitations of SQLite.
Are there any valid workarounds to this?
I know of one, but is almost unfeasible: Leave out the loop, variable, and concatenations, and manually copy and paste this code bloc, manually changing the relevant fields each time. However, this would require 255 copy and pastes multiplied by around 8 or 9 times for each different BUILDING_TYPE I need to attach rows to. I'd rather not do this if there is a faster and more efficient way!
You can do it with a recursive CTE and the use of a temporary table:
drop table if exists temp.temptable;
create temporary table temptable(cnt int, seyield text, secitizens text, secount text);
with
recursive constants as (
select
'BUILDING_STOCK_EXCHANGE_YIELD_' seyield,
'BUILDING_STOCK_EXCHANGE_CITIZENS_' secitizens,
'COUNT_CITIZENS_' secount
),
numbers as (
select 1 cnt
from constants
union all
select cnt + 1 from numbers
where cnt < 255
),
cte as (
select
n.cnt cnt,
c.seyield || n.cnt seyield,
c.secitizens || n.cnt secitizens,
c.secount || n.cnt secount
from numbers n cross join constants c
)
insert into temptable
select * from cte;
INSERT INTO BuildingModifiers (BuildingType, ModifierId)
SELECT 'BUILDING_STOCK_EXCHANGE', seyield FROM temptable;
INSERT INTO Modifiers (ModifierId, ModifierType, RunOnce, Permanent, SubjectRequirementSetId)
SELECT seyield, 'MODIFIER_BUILDING_YIELD_CHANGE', 0, 0, secitizens FROM temptable;
INSERT INTO ModifierArguments (ModifierID, Name, Value)
SELECT seyield, 'BuildingType', 'BUILDING_STOCK_EXCHANGE' FROM temptable
UNION ALL
SELECT seyield, 'Amount', '2' FROM temptable
UNION ALL
SELECT seyield, 'YieldType', 'YIELD_GOLD' FROM temptable;
INSERT INTO RequirementSets(RequirementSetId, RequirementSetType)
SELECT secitizens, 'REQUIREMENT_TEST_ALL' FROM temptable;
INSERT INTO RequirementSetRequirements(RequirementSetId, RequirementId)
SELECT secitizens, secount FROM temptable;
INSERT INTO Requirements(RequirementId, RequirementType)
SELECT secount, 'REQUIREMENT_COLLECTION_ATLEAST' FROM temptable;
INSERT INTO RequirementArguments(RequirementId, Name, Value)
SELECT secount, 'CollectionType', 'COLLECTION_CITY_PLOT_YIELDS' FROM temptable
UNION ALL
SELECT secount, 'Count', cnt FROM temptable;
See the demo.
sample data I have 2 columns old_store_id, changed_new_store_id and there are cases when changed_new_store_id value will also get updated to new value. how can i traverse through DB(teradata) to get the last value (changed_new_store_id ) of the respective old_store_id
let say in 1 st row
old_store_id = A ;
changed_new_store_id = B
and 5 th row contains
old_store_id = B ;
changed_new_store_id = C
and some other nth row C is changed to X etc
how to get final value of A which is X ?
I can try using multiple self joins
using Stored procedure but it will not be an efficient way (for many reasons)
Is there any way to find ?
Please anyone suggest me
This assumes no "loops", and uses "bottom-up" recursion. Something very similar could be done "top-down", limiting the seed query to rows where the "old" value doesn't appear anywhere as a "new" value.
CREATE VOLATILE TABLE #Example (
Old_Store_ID VARCHAR(8),
New_Store_ID VARCHAR(8)
)
PRIMARY INDEX(Old_Store_ID)
ON COMMIT PRESERVE ROWS;
INSERT INTO #Example VALUES ('A', 'B');
INSERT INTO #Example VALUES ('D', 'c');
INSERT INTO #Example VALUES ('B', 'F');
INSERT INTO #Example VALUES ('c', 'FF');
INSERT INTO #Example VALUES ('FF', 'GG');
INSERT INTO #Example VALUES ('F', 'X');
WITH RECURSIVE #Traverse(Old_Store_ID,New_Store_ID,Final_ID)
AS
(
--Seed Query - start with only the rows having no further changes
SELECT Old_Store_ID
,New_Store_ID
,New_Store_ID as Final_ID
FROM #Example as This
WHERE NOT EXISTS (
SELECT 1 FROM #Example AS Other WHERE This.New_Store_ID = Other.Old_Store_ID
)
UNION ALL
--Recursive Join
SELECT NewRow.Old_Store_ID
,NewRow.New_Store_ID
,OldRow.Final_ID
FROM #Example AS NewRow
INNER JOIN #Traverse AS OldRow
ON NewRow.New_Store_ID = OldRow.Old_Store_ID
)
SELECT *
FROM #Traverse
;
A recursive answer:
CREATE VOLATILE TABLE #SearchList (
SearchID CHAR(2),
ParentSearchID CHAR(2)
)
PRIMARY INDEX(SearchID)
ON COMMIT PRESERVE ROWS;
INSERT INTO #SearchList VALUES ('A', 'B');
INSERT INTO #SearchList VALUES ('D', 'c');
INSERT INTO #SearchList VALUES ('B', 'F');
INSERT INTO #SearchList VALUES ('c', 'FF');
INSERT INTO #SearchList VALUES ('FF', 'GG');
INSERT INTO #SearchList VALUES ('F', 'X');
CREATE VOLATILE TABLE #IntermediateResults(
SearchID CHAR(2),
ParentSearchID CHAR(2),
SearchLevel INTEGER
)
ON COMMIT PRESERVE ROWS;
INSERT INTO #IntermediateResults
WITH RECURSIVE RecursiveParent(SearchID,ParentSearchID,SearchLevel)
AS
(
--Seed Query
SELECT SearchID
,ParentSearchID
,1
FROM #SearchList
UNION ALL
--Recursive Join
SELECT a.SearchID
,b.ParentSearchID
,SearchLevel+1
FROM #SearchList a
INNER JOIN RecursiveParent b
ON a.ParentSearchID = b.SearchID
)
SELECT SearchID
,ParentSearchID
,MAX(SearchLevel)
FROM RecursiveParent
GROUP BY SearchID
,ParentSearchID
;
SELECT RESULTS.*
FROM #IntermediateResults RESULTS
INNER JOIN (SELECT RESULTS_MAX.SearchID
,MAX(RESULTS_MAX.SearchLevel) MaxSearchLevel
FROM #IntermediateResults RESULTS_MAX
GROUP BY RESULTS_MAX.SearchID
) GROUPED_RESULTS
ON RESULTS.SearchID = GROUPED_RESULTS.SearchID
AND RESULTS.SearchLevel = GROUPED_RESULTS.MaxSearchLevel
ORDER BY RESULTS.SearchID ASC
,RESULTS.SearchLevel ASC
;
Output:
SearchID ParentSearchID SearchLevel
-------- -------------- -----------
A X 3
B X 2
c GG 2
D GG 3
F X 1
FF GG 1
I've got two tables already populated with data with the given schemas:
CREATE TABLE objects
(
id BIGINT NOT NULL,
latitude BIGINT NOT NULL,
longitude BIGINT NOT NULL,
PRIMARY KEY (id)
)
CREATE TABLE tags
(
id BIGINT NOT NULL,
tag_key VARCHAR(100) NOT NULL,
tag_value VARCHAR(500),
PRIMARY KEY (id , tag_key)
)
object.id and tags.id refer to the same object
I'd like to populate a third table with the unique combinations of tag_key and tag_value. For example:
INSERT OR REPLACE INTO objects (id) VALUES (0);
INSERT OR REPLACE INTO tags (id, tag_key, tag_value) VALUES (0, 'a', 'x');
INSERT OR REPLACE INTO objects (id) VALUES (1);
INSERT OR REPLACE INTO tags (id, tag_key, tag_value) VALUES (1, 'a', 'y');
INSERT OR REPLACE INTO objects (id) VALUES (2);
INSERT OR REPLACE INTO tags (id, tag_key, tag_value) VALUES (2, 'a', 'x');
INSERT OR REPLACE INTO tags (id, tag_key, tag_value) VALUES (2, 'a', 'y');
INSERT OR REPLACE INTO objects (id) VALUES (3);
INSERT OR REPLACE INTO tags (id, tag_key, tag_value) VALUES (3, 'a', 'x');
INSERT OR REPLACE INTO objects (id) VALUES (4);
INSERT OR REPLACE INTO tags (id, tag_key, tag_value) VALUES (4, 'a', 'y');
Should result in 3 entries of
0: ([a,x])
1: ([a,y])
3: ([a,x][a,y])
Currently I have:
CREATE TABLE tags_combinations
(
id INTEGER PRIMARY KEY,
tag_key VARCHAR(100) NOT NULL,
tag_value VARCHAR(500)
);
The id shouldn't be related to the original id of the object, just something to group unique combinations.
This is the query I have so far:
SELECT
t1.tag_key, t1.tag_value
FROM
tags t1
WHERE
t1.id
IN
(
/* select ids who's every tags entry is not under one id in tags_combinations */
SELECT
t2.id
FROM
tags t2
WHERE
t2.tag_key, t2.tag_value
NOT IN
(
)
);
The part with the comment is what I am not sure about, how would I select every id from tags that does not have all of the corresponding tag_key and tag_value entries already under one id in tags_combinations?
To clarify exactly the result I am after: From the sample data given, it should return 4 rows with:
row id tag_key tag_value
0 0 a x
1 1 a y
2 2 a x
3 2 a y
SQL is a set-based language. If you reformulate your question in the language of set theory, you can directly translate it into SQL:
You want all rows of the tags table, except those from duplicate objects.
Objects are duplicates if they have exactly the same key/value combinations. However, we still want to return one of those objects, so we define duplicates only as those objects where no other duplicate object with a smaller ID exists.
Two objects A and B have exactly the same key/value combinations if
all key/value combinations in A also exist in B, and
all key/value combinations in B also exist in A.
All key/value combinations in A also exist in B if there is no key/value combination in A that does not exist in B (note: double negation).
SELECT id, tag_key, tag_value
FROM tags
WHERE NOT EXISTS (SELECT 1
FROM tags AS dup
WHERE dup.id < tags.id
AND NOT EXISTS (SELECT 1
FROM tags AS A
WHERE A.id = tags.id
AND NOT EXISTS (SELECT 1
FROM tags AS B
WHERE B.id = dup.id
AND B.tag_key = A.tag_key
AND B.tag_value = A.tag_value)
)
AND NOT EXISTS (SELECT 1
FROM tags AS B
WHERE B.id = dup.id
AND NOT EXISTS (SELECT 1
FROM tags AS A
WHERE A.id = tags.id
AND A.tag_key = B.tag_key
AND A.tag_value = B.tag_value)
)
)
ORDER BY id, tag_key;
This is not easy in SQLite. We want to identify groups of tag key/value pairs. So we could group by id and get a string of the associated pairs with group_concat. This would be the way to do it in another DBMS. SQLite, however, cannot order in group_concat, so we might end up with 2: 'a/x,a/y' and 5: 'a/y,a/x'. Two different strings for the same pairs.
Your best bet may be to write a program and find the distinct pairs iteratively.
In SQLite you may want to try this:
insert into tags_combinations (id, tag_key, tag_value)
select id, tag_key, tag_value
from tags
where id in
(
select min(id)
from
(
select id, group_concat(tag_key || '/' || tag_value) as tag_pairs
from
(
select id, tag_key, tag_value
from tags
order by id, tag_key, tag_value
) ordered_data
group by id
) aggregated_data
group by tag_pairs
);
Ordering the data before applying group_concat is likely to get the tag pairs ordered, but in no way guaranteed! If this is something you want to do only once, it may be worth a try, though.
To merge multiple rows into one value, you need a function like group_concat().
The ORDER BY is needed to ensure a consistent order of the rows within a group:
SELECT DISTINCT group_concat(tag_key) AS tag_keys,
group_concat(tag_value) AS tag_values
FROM (SELECT id,
tag_key,
tag_value
FROM tags
ORDER BY id,
tag_key,
tag_value)
GROUP BY id;
If you want to have keys and values interleaved, as shown in the question, you need to do more string concatenation:
SELECT DISTINCT group_concat(tag_key || ',' || tag_value, ';') AS keys_and_values
FROM (...
I can not figure out how to query a SQLite.
needed:
1) Replace the record (the primary key), if the condition (comparison of new and old fields entries)
2) Insert an entry if no such entry exists in the database on the primary key.
Importantly, it has to work very fast!
I can not come up with an effective inquiry.
Edit.
MyInsertRequest - the desired expression.
Script:
CREATE TABLE testtable (a INT PRIMARY KEY, b INT, c INT)
INSERT INTO testtable VALUES (1, 2, 3)
select * from testtable
1|2|3
-- Adds an entry, because the primary key is not
++ MyInsertRequest VALUES (2, 2, 3) {if c>4 then replace}
select * from testtable
1|2|3
2|2|3
-- Adds
++ MyInsertRequest VALUES (3, 8, 3) {if c>4 then replace}
select * from testtable
1|2|3
2|2|3
3|8|3
-- Does nothing, because such a record (from primary key field 'a')
-- is in the database and none c>4
++ MyInsertRequest VALUES (1, 2, 3) {if c>4 then replace}
select * from testtable
1|2|3
2|2|3
3|8|3
-- Does nothing
++ MyInsertRequest VALUES (3, 34, 3) {if c>4 then replace}
select * from testtable
1|2|3
2|2|3
3|8|3
-- replace, because such a record (from primary key field 'a')
-- is in the database and c>2
++ MyInsertRequest VALUES (3, 34, 1) {if c>2 then replace}
select * from testtable
1|2|3
2|2|3
3|34|1
Isn't INSERT OR REPLACE what you need ? e.g. :
INSERT OR REPLACE INTO table (cola, colb) values (valuea, valueb)
When a UNIQUE constraint violation occurs, the REPLACE algorithm
deletes pre-existing rows that are causing the constraint violation
prior to inserting or updating the current row and the command
continues executing normally.
You have to put the condition in a unique constraint on the table. It will automatically create an index to make the check efficient.
e.g.
-- here the condition is on columnA, columnB
CREATE TABLE sometable (columnPK INT PRIMARY KEY,
columnA INT,
columnB INT,
columnC INT,
CONSTRAINT constname UNIQUE (columnA, columnB)
)
INSERT INTO sometable VALUES (1, 1, 1, 0);
INSERT INTO sometable VALUES (2, 1, 2, 0);
select * from sometable
1|1|1|0
2|1|2|0
-- insert a line with a new PK, but with existing values for (columnA, columnB)
-- the line with PK 2 will be replaced
INSERT OR REPLACE INTO sometable VALUES (12, 1, 2, 6)
select * from sometable
1|1|1|0
12|1|2|6
Assuming your requirements are:
Insert a new row when a doesn't exists;
Replacing row when a exist and existing c greater then new c;
Do nothing when a exist and existing c lesser or equal then new c;
INSERT OR REPLACE fits first two requirements.
For last requirement, the only way I know to make an INSERT ineffective is supplying a empty rowset.
A SQLite command like following whould make the job:
INSERT OR REPLACE INTO sometable SELECT newdata.* FROM
(SELECT 3 AS a, 2 AS b, 1 AS c) AS newdata
LEFT JOIN sometable ON newdata.a=sometable.a
WHERE newdata.c<sometable.c OR sometable.a IS NULL;
New data (3,2,1 in this example) is LEFT JOINen with current table data.
Then WHERE will "de-select" the row when new c is not less then existing c, keeping it when row is new, ie, sometable.* IS NULL.
I tried the others answers because I was also suffering from a solution to this problem.
This should work, however I am unsure about the performance implications. I believe that you may need the first column to be unique as a primary key else it will simply insert a new record each time.
INSERT OR REPLACE INTO sometable
SELECT columnA, columnB, columnC FROM (
SELECT columnA, columnB, columnC, 1 AS tmp FROM sometable
WHERE sometable.columnA = 1 AND
sometable.columnB > 9
UNION
SELECT 1 AS columnA, 1 As columnB, 404 as columnC, 0 AS tmp)
ORDER BY tmp DESC
LIMIT 1
In this case one dummy query is executed and union-ed onto a second query which would have a performance impact depending on how it is written and how the table is indexed. The next performance problem has potential where the results are ordered and limited. However, I expect that the second query should only return one record and therefore it should not be too much of a performance hit.
You can also omit the ORDER BY tmp LIMIT 1 and it works with my version of sqlite, but it may impact performance since it can end up updating the record twice (writing the original value then the new value if applicable).
The other problem is that you end up with a write to the table even if the condition states that it should not be updated.