I can not figure out how to query a SQLite.
needed:
1) Replace the record (the primary key), if the condition (comparison of new and old fields entries)
2) Insert an entry if no such entry exists in the database on the primary key.
Importantly, it has to work very fast!
I can not come up with an effective inquiry.
Edit.
MyInsertRequest - the desired expression.
Script:
CREATE TABLE testtable (a INT PRIMARY KEY, b INT, c INT)
INSERT INTO testtable VALUES (1, 2, 3)
select * from testtable
1|2|3
-- Adds an entry, because the primary key is not
++ MyInsertRequest VALUES (2, 2, 3) {if c>4 then replace}
select * from testtable
1|2|3
2|2|3
-- Adds
++ MyInsertRequest VALUES (3, 8, 3) {if c>4 then replace}
select * from testtable
1|2|3
2|2|3
3|8|3
-- Does nothing, because such a record (from primary key field 'a')
-- is in the database and none c>4
++ MyInsertRequest VALUES (1, 2, 3) {if c>4 then replace}
select * from testtable
1|2|3
2|2|3
3|8|3
-- Does nothing
++ MyInsertRequest VALUES (3, 34, 3) {if c>4 then replace}
select * from testtable
1|2|3
2|2|3
3|8|3
-- replace, because such a record (from primary key field 'a')
-- is in the database and c>2
++ MyInsertRequest VALUES (3, 34, 1) {if c>2 then replace}
select * from testtable
1|2|3
2|2|3
3|34|1
Isn't INSERT OR REPLACE what you need ? e.g. :
INSERT OR REPLACE INTO table (cola, colb) values (valuea, valueb)
When a UNIQUE constraint violation occurs, the REPLACE algorithm
deletes pre-existing rows that are causing the constraint violation
prior to inserting or updating the current row and the command
continues executing normally.
You have to put the condition in a unique constraint on the table. It will automatically create an index to make the check efficient.
e.g.
-- here the condition is on columnA, columnB
CREATE TABLE sometable (columnPK INT PRIMARY KEY,
columnA INT,
columnB INT,
columnC INT,
CONSTRAINT constname UNIQUE (columnA, columnB)
)
INSERT INTO sometable VALUES (1, 1, 1, 0);
INSERT INTO sometable VALUES (2, 1, 2, 0);
select * from sometable
1|1|1|0
2|1|2|0
-- insert a line with a new PK, but with existing values for (columnA, columnB)
-- the line with PK 2 will be replaced
INSERT OR REPLACE INTO sometable VALUES (12, 1, 2, 6)
select * from sometable
1|1|1|0
12|1|2|6
Assuming your requirements are:
Insert a new row when a doesn't exists;
Replacing row when a exist and existing c greater then new c;
Do nothing when a exist and existing c lesser or equal then new c;
INSERT OR REPLACE fits first two requirements.
For last requirement, the only way I know to make an INSERT ineffective is supplying a empty rowset.
A SQLite command like following whould make the job:
INSERT OR REPLACE INTO sometable SELECT newdata.* FROM
(SELECT 3 AS a, 2 AS b, 1 AS c) AS newdata
LEFT JOIN sometable ON newdata.a=sometable.a
WHERE newdata.c<sometable.c OR sometable.a IS NULL;
New data (3,2,1 in this example) is LEFT JOINen with current table data.
Then WHERE will "de-select" the row when new c is not less then existing c, keeping it when row is new, ie, sometable.* IS NULL.
I tried the others answers because I was also suffering from a solution to this problem.
This should work, however I am unsure about the performance implications. I believe that you may need the first column to be unique as a primary key else it will simply insert a new record each time.
INSERT OR REPLACE INTO sometable
SELECT columnA, columnB, columnC FROM (
SELECT columnA, columnB, columnC, 1 AS tmp FROM sometable
WHERE sometable.columnA = 1 AND
sometable.columnB > 9
UNION
SELECT 1 AS columnA, 1 As columnB, 404 as columnC, 0 AS tmp)
ORDER BY tmp DESC
LIMIT 1
In this case one dummy query is executed and union-ed onto a second query which would have a performance impact depending on how it is written and how the table is indexed. The next performance problem has potential where the results are ordered and limited. However, I expect that the second query should only return one record and therefore it should not be too much of a performance hit.
You can also omit the ORDER BY tmp LIMIT 1 and it works with my version of sqlite, but it may impact performance since it can end up updating the record twice (writing the original value then the new value if applicable).
The other problem is that you end up with a write to the table even if the condition states that it should not be updated.
Related
I am stuck with the limitations of both SQLite and the design of some of its tables. Here's what I'd like to achieve:
Create a variable to track the number of iterations of a loop. Then use this variable to help create unique table rows during the loop. Also insert it as the literal count value in a record.
Concatenate said variable to existing strings to create unique IDs that can be referenced multiple times.
Loop code using the previous variable to track iterations, and the previous concatenations as IDs for inserting new rows for every loop. 255 loops is an arbitrary number, I doubt I'd need 255 loops in 99.9% of cases, but want to avoid failure in the cases where they are needed. I am realistically looking at 50 loops minimum, with 100 as a rare maximum. 200 would likely be closer to the true maximum outlier number. 255 is just to be safe.
Here is what I have attempted so far:
DECLARE #cnt INT = 1;
WHILE #cnt < 256
BEGIN
#seyield = 'BUILDING_STOCK_EXCHANGE_YIELD_' + #cnt;
#secitizens = 'BUILDING_STOCK_EXCHANGE_CITIZENS_' + #cnt;
#secount = 'COUNT_CITIZENS_' + #cnt;
INSERT INTO
BuildingModifiers (BuildingType, ModifierId)
VALUES
('BUILDING_STOCK_EXCHANGE', #seyield);
INSERT INTO
Modifiers (ModifierId, ModifierType, RunOnce, Permanent, SubjectRequirementSetId)
VALUES
(#seyield, 'MODIFIER_BUILDING_YIELD_CHANGE', 0, 0, #secitizens);
INSERT INTO
ModifierArguments (ModifierID, Name, Value)
VALUES
(#seyield, 'BuildingType', 'BUILDING_STOCK_EXCHANGE'),
(#seyield, 'Amount', '2'),
(#seyield, 'YieldType', 'YIELD_GOLD');
INSERT INTO
RequirementSets(RequirementSetId, RequirementSetType)
VALUES
(#secitizens, 'REQUIREMENT_TEST_ALL');
INSERT INTO
RequirementSetRequirements(RequirementSetId, RequirementId)
VALUES
(#secitizens, #secount);
INSERT INTO
Requirements(RequirementId, RequirementType)
VALUES
(#secount, 'REQUIREMENT_COLLECTION_ATLEAST');
INSERT INTO
RequirementArguments(RequirementId, Name, Value)
VALUES
(#secount, 'CollectionType', 'COLLECTION_CITY_PLOT_YIELDS'),
(#secount, 'Count', #cnt);
SET #cnt = #cnt + 1;
END;
Of course this does not work due to the limitations of SQLite.
Are there any valid workarounds to this?
I know of one, but is almost unfeasible: Leave out the loop, variable, and concatenations, and manually copy and paste this code bloc, manually changing the relevant fields each time. However, this would require 255 copy and pastes multiplied by around 8 or 9 times for each different BUILDING_TYPE I need to attach rows to. I'd rather not do this if there is a faster and more efficient way!
You can do it with a recursive CTE and the use of a temporary table:
drop table if exists temp.temptable;
create temporary table temptable(cnt int, seyield text, secitizens text, secount text);
with
recursive constants as (
select
'BUILDING_STOCK_EXCHANGE_YIELD_' seyield,
'BUILDING_STOCK_EXCHANGE_CITIZENS_' secitizens,
'COUNT_CITIZENS_' secount
),
numbers as (
select 1 cnt
from constants
union all
select cnt + 1 from numbers
where cnt < 255
),
cte as (
select
n.cnt cnt,
c.seyield || n.cnt seyield,
c.secitizens || n.cnt secitizens,
c.secount || n.cnt secount
from numbers n cross join constants c
)
insert into temptable
select * from cte;
INSERT INTO BuildingModifiers (BuildingType, ModifierId)
SELECT 'BUILDING_STOCK_EXCHANGE', seyield FROM temptable;
INSERT INTO Modifiers (ModifierId, ModifierType, RunOnce, Permanent, SubjectRequirementSetId)
SELECT seyield, 'MODIFIER_BUILDING_YIELD_CHANGE', 0, 0, secitizens FROM temptable;
INSERT INTO ModifierArguments (ModifierID, Name, Value)
SELECT seyield, 'BuildingType', 'BUILDING_STOCK_EXCHANGE' FROM temptable
UNION ALL
SELECT seyield, 'Amount', '2' FROM temptable
UNION ALL
SELECT seyield, 'YieldType', 'YIELD_GOLD' FROM temptable;
INSERT INTO RequirementSets(RequirementSetId, RequirementSetType)
SELECT secitizens, 'REQUIREMENT_TEST_ALL' FROM temptable;
INSERT INTO RequirementSetRequirements(RequirementSetId, RequirementId)
SELECT secitizens, secount FROM temptable;
INSERT INTO Requirements(RequirementId, RequirementType)
SELECT secount, 'REQUIREMENT_COLLECTION_ATLEAST' FROM temptable;
INSERT INTO RequirementArguments(RequirementId, Name, Value)
SELECT secount, 'CollectionType', 'COLLECTION_CITY_PLOT_YIELDS' FROM temptable
UNION ALL
SELECT secount, 'Count', cnt FROM temptable;
See the demo.
I've got two tables already populated with data with the given schemas:
CREATE TABLE objects
(
id BIGINT NOT NULL,
latitude BIGINT NOT NULL,
longitude BIGINT NOT NULL,
PRIMARY KEY (id)
)
CREATE TABLE tags
(
id BIGINT NOT NULL,
tag_key VARCHAR(100) NOT NULL,
tag_value VARCHAR(500),
PRIMARY KEY (id , tag_key)
)
object.id and tags.id refer to the same object
I'd like to populate a third table with the unique combinations of tag_key and tag_value. For example:
INSERT OR REPLACE INTO objects (id) VALUES (0);
INSERT OR REPLACE INTO tags (id, tag_key, tag_value) VALUES (0, 'a', 'x');
INSERT OR REPLACE INTO objects (id) VALUES (1);
INSERT OR REPLACE INTO tags (id, tag_key, tag_value) VALUES (1, 'a', 'y');
INSERT OR REPLACE INTO objects (id) VALUES (2);
INSERT OR REPLACE INTO tags (id, tag_key, tag_value) VALUES (2, 'a', 'x');
INSERT OR REPLACE INTO tags (id, tag_key, tag_value) VALUES (2, 'a', 'y');
INSERT OR REPLACE INTO objects (id) VALUES (3);
INSERT OR REPLACE INTO tags (id, tag_key, tag_value) VALUES (3, 'a', 'x');
INSERT OR REPLACE INTO objects (id) VALUES (4);
INSERT OR REPLACE INTO tags (id, tag_key, tag_value) VALUES (4, 'a', 'y');
Should result in 3 entries of
0: ([a,x])
1: ([a,y])
3: ([a,x][a,y])
Currently I have:
CREATE TABLE tags_combinations
(
id INTEGER PRIMARY KEY,
tag_key VARCHAR(100) NOT NULL,
tag_value VARCHAR(500)
);
The id shouldn't be related to the original id of the object, just something to group unique combinations.
This is the query I have so far:
SELECT
t1.tag_key, t1.tag_value
FROM
tags t1
WHERE
t1.id
IN
(
/* select ids who's every tags entry is not under one id in tags_combinations */
SELECT
t2.id
FROM
tags t2
WHERE
t2.tag_key, t2.tag_value
NOT IN
(
)
);
The part with the comment is what I am not sure about, how would I select every id from tags that does not have all of the corresponding tag_key and tag_value entries already under one id in tags_combinations?
To clarify exactly the result I am after: From the sample data given, it should return 4 rows with:
row id tag_key tag_value
0 0 a x
1 1 a y
2 2 a x
3 2 a y
SQL is a set-based language. If you reformulate your question in the language of set theory, you can directly translate it into SQL:
You want all rows of the tags table, except those from duplicate objects.
Objects are duplicates if they have exactly the same key/value combinations. However, we still want to return one of those objects, so we define duplicates only as those objects where no other duplicate object with a smaller ID exists.
Two objects A and B have exactly the same key/value combinations if
all key/value combinations in A also exist in B, and
all key/value combinations in B also exist in A.
All key/value combinations in A also exist in B if there is no key/value combination in A that does not exist in B (note: double negation).
SELECT id, tag_key, tag_value
FROM tags
WHERE NOT EXISTS (SELECT 1
FROM tags AS dup
WHERE dup.id < tags.id
AND NOT EXISTS (SELECT 1
FROM tags AS A
WHERE A.id = tags.id
AND NOT EXISTS (SELECT 1
FROM tags AS B
WHERE B.id = dup.id
AND B.tag_key = A.tag_key
AND B.tag_value = A.tag_value)
)
AND NOT EXISTS (SELECT 1
FROM tags AS B
WHERE B.id = dup.id
AND NOT EXISTS (SELECT 1
FROM tags AS A
WHERE A.id = tags.id
AND A.tag_key = B.tag_key
AND A.tag_value = B.tag_value)
)
)
ORDER BY id, tag_key;
This is not easy in SQLite. We want to identify groups of tag key/value pairs. So we could group by id and get a string of the associated pairs with group_concat. This would be the way to do it in another DBMS. SQLite, however, cannot order in group_concat, so we might end up with 2: 'a/x,a/y' and 5: 'a/y,a/x'. Two different strings for the same pairs.
Your best bet may be to write a program and find the distinct pairs iteratively.
In SQLite you may want to try this:
insert into tags_combinations (id, tag_key, tag_value)
select id, tag_key, tag_value
from tags
where id in
(
select min(id)
from
(
select id, group_concat(tag_key || '/' || tag_value) as tag_pairs
from
(
select id, tag_key, tag_value
from tags
order by id, tag_key, tag_value
) ordered_data
group by id
) aggregated_data
group by tag_pairs
);
Ordering the data before applying group_concat is likely to get the tag pairs ordered, but in no way guaranteed! If this is something you want to do only once, it may be worth a try, though.
To merge multiple rows into one value, you need a function like group_concat().
The ORDER BY is needed to ensure a consistent order of the rows within a group:
SELECT DISTINCT group_concat(tag_key) AS tag_keys,
group_concat(tag_value) AS tag_values
FROM (SELECT id,
tag_key,
tag_value
FROM tags
ORDER BY id,
tag_key,
tag_value)
GROUP BY id;
If you want to have keys and values interleaved, as shown in the question, you need to do more string concatenation:
SELECT DISTINCT group_concat(tag_key || ',' || tag_value, ';') AS keys_and_values
FROM (...
C:\Users\pengsir>sqlite3 e:\\test.db
sqlite> create table test (f1 TEXT,f2 TEXT, f3 TEXT);
sqlite> insert into test values("x1","y1","w1");
sqlite> insert into test values("x1","y1","w2");
sqlite> insert into test values("x1","y3","w2");
sqlite> insert into test values("x2","y3","w2");
sqlite> insert into test values("x3","y4","w4");
sqlite> insert into test values("x2","y3","w4");
sqlite> insert into test values("x1","y3","w2");
sqlite>
1.select the record rows which contain the same f1 and f2 ,and the rowid .
sqlite> select rowid,f1,f2 from test group by f1,f2 having(count(f2)>1 and count(f2)>1);
2|x1|y1
7|x1|y3
6|x2|y3
I want the result to be :
1|x1|y1
2|x1|y1
3|x1|y3
4|x2|y3
6|x2|y3
7|x1|y3
2.select the record rows which contain the same f1 f2 and f3,and the rowid .
sqlite> select rowid,f1,f2,f3 from test group by f1,f2,f3 having(count(f2)>1 and count(f3)>1);
7|x1|y3|w2
I want the result to be
3|x1|y3|w2
7|x1|y3|w2
let us discuss this problem further , i want to delete one |x1|y3|w2 and keep one |x1|y3|w2 in the table?here is my method.
DELETE FROM test
WHERE rowid in(
SELECT rowid FROM test
WHERE (SELECT count(*)
FROM test AS t2
WHERE t2.f1 = test.f1
AND t2.f2 = test.f2
AND t2.f3 = test.f3
) >= 2 limit 1);
Is there more simple and smart way to do that? (the method is wrong)
I find the proper way to do .
delete from test
where rowid not in
(
select max(rowid)
from test
group by
f1,f2,f3
);
and the method to more than one duplicate for a f1/f2 combination is :
delete from test
where rowid not in
(select rowid from test group by f1,f2);
It will be executed only one time.
You want records that have duplicates (in those fields), i.e., where the number of records with the same values in those fields is at least two:
SELECT rowid, f1, f2
FROM test
WHERE (SELECT count(*)
FROM test AS t2
WHERE t2.f1 = test.f1
AND t2.f2 = test.f2
) >= 2
This requires executing the subquery for each record.
Alternatively, compute the records with duplicates in a subquery once; this might be more efficient:
SELECT test.rowid, test.f1, test.f2
FROM test
JOIN (SELECT f1, f2
FROM test
GROUP BY f1, f2
HAVING count(*) >= 2
) USING (f1, f2)
If you want to remove one of the duplicates, this is easier to do because GROUP BY already returns exactly one output row for each group:
DELETE FROM test
WHERE rowid IN (SELECT max(rowid)
FROM test
GROUP BY f1, f2
HAVING COUNT(*) >= 2)
(If there is more than one duplicate for a f1/f2 combination, you have to execute this multiple times.)
Try using SELECT ALL instead of SELECT but according to the linked docs SELECT should behave like SELECT ALL by default and not like SELECT DISTINCT so I don't know where the problem may be.
ALTER PROCEDURE [dbo].[K_FS_InsertMrpDetails]
#date datetime,
#feedtype varchar(50),
#rateperkg float,
#rateper50kg float,
#updatedby varchar(50)
AS
BEGIN
INSERT INTO K_FS_FeedMrpDetails([date], feedtype, rateperkg, rateper50kg, updatedby, updatedon)
VALUES(#date, #feedtype, #rateperkg, #rateper50kg, #updatedby, getdate())
SELECT '1' AS status
END
With this query we insert 9 rows at a time but what I want is in one same date do not insert again different details. How can I please help me.
Add a unique constraint on the column [date]. That will prevent you from adding more than one row with the same [date] value.
Update:
To allow 9 rows for each date you can add a computed column D that removes the time part and you need to add a column that will hold the values 1 to 9 R. Use a check constraint on R to only allow 1-9. Finally you create a unique constraint on (R, D).
Sample table definition:
create table T
(
ID int identity primary key,
DT datetime not null,
R tinyint check (R in (1,2,3,4,5,6,7,8,9)) not null,
D as dateadd(day, datediff(day, 0, DT), 0),
constraint ux_RD unique (R,D)
)
Try with this:
insert into T(DT, R) values(getdate(), 1)
insert into T(DT, R) values(getdate(), 2)
insert into T(DT, R) values(getdate(), 1)
First and second insert works fine, the third raises a unique constraint exception.
I'm adding an 'index' column to a table in SQLite3 to allow the users to easily reorder the data, by renaming the old database and creating a new one in its place with the extra columns.
The problem I have is that I need to give each row a unique number in the 'index' column when I INSERT...SELECT the old values.
A search I did turned up a useful term in Oracle called ROWNUM, but SQLite3 doesn't have that. Is there something equivalent in SQLite?
You can use one of the special row names ROWID, OID or _ROWID_ to get the rowid of a column. See http://www.sqlite.org/lang_createtable.html#rowid for further details (and that the rows can be hidden by normal columns called ROWID and so on).
Many people here seems to mix up ROWNUM with ROWID. They are not the same concept and Oracle has both.
ROWID is a unique ID of a database ROW. It's almost invariant (changed during import/export but it is the same across different SQL queries).
ROWNUM is a calculated field corresponding to the row number in the query result. It's always 1 for the first row, 2 for the second, and so on. It is absolutely not linked to any table row and the same table row could have very different rownums depending of how it is queried.
Sqlite has a ROWID but no ROWNUM. The only equivalent I found is ROW_NUMBER() function (see http://www.sqlitetutorial.net/sqlite-window-functions/sqlite-row_number/).
You can achieve what you want with a query like this:
insert into new
select *, row_number() over ()
from old;
No SQLite doesn't have a direct equivalent to Oracle's ROWNUM.
If I understand your requirement correctly, you should be able to add a numbered column based on ordering of the old table this way:
create table old (col1, col2);
insert into old values
('d', 3),
('s', 3),
('d', 1),
('w', 45),
('b', 5465),
('w', 3),
('b', 23);
create table new (colPK INTEGER PRIMARY KEY AUTOINCREMENT, col1, col2);
insert into new select NULL, col1, col2 from old order by col1, col2;
The new table contains:
.headers on
.mode column
select * from new;
colPK col1 col2
---------- ---------- ----------
1 b 23
2 b 5465
3 d 1
4 d 3
5 s 3
6 w 3
7 w 45
The AUTOINCREMENT does what its name suggests: each additional row has the previous' value incremented by 1.
I believe you want to use the constrain LIMIT in SQLite.
SELECT * FROM TABLE can return thousands of records.
However, you can constrain this by adding the LIMIT keyword.
SELECT * FROM TABLE LIMIT 5;
Will return the first 5 records from the table returned in you query - if available
use this code For create Row_num 0....count_row
SELECT (SELECT COUNT(*)
FROM main AS t2
WHERE t2.col1 < t1.col1) + (SELECT COUNT(*)
FROM main AS t3
WHERE t3.col1 = t1.col1 AND t3.col1 < t1.col1) AS rowNum, * FROM Table_name t1 WHERE rowNum=0 ORDER BY t1.col1 ASC