I have the following SQLite table
CREATE TABLE IF NOT EXISTS bonds (a1 INTEGER,a2 INTEGER, hits INTEGER, lasthit INTEGER,PRIMARY KEY(a1,a2)) WITHOUT ROWID;
For every unique a1-a2 pair there will ever be only one entry - PRIMARY KEY(a1,a2) imposes that constraint.
Now consider the following UPSERT
INSERT INTO bonds (a1,a2,hits,lasthit)
VALUES(1,1,1,1) ON CONFLICT(a1,a2) DO UPDATE SET hits = hits + 1;
followed by
SELECT a1,a2,hits FROM bonds;
which returns
a1 a2 hits
================
1 1 1
Re-issuing the same INSERT/SELECT pair as above then yields
a1 a2 hits
================
1 1 2
which appears to suggest that the way I have written the INSERT statement is in fact correct. However, I have not been able to find any examples in the SQLite documentation on how to use the ON CONFLICT clause with 2+ columns. Is my way, ON CONFLICT(a1,a2) correct?
Here an example of a separate UPSERT statements to check for 2 different unique constraint violations
INSERT INTO biases (data,link,bias_type,ignores,scraps) VALUES(1,1,1,1,100);
SELECT * FROM biases;
INSERT INTO biases (data,link,bias_type,ignores,scraps) VALUES(1,1,1,1,100)
ON CONFLICT(data,link,bias_type) DO UPDATE SET ignores = ignores + 1;
SELECT * FROM biases;
INSERT INTO biases (data,link,bias_type,ignores,scraps) VALUES(1,1,1,1,100)
ON CONFLICT(scraps) DO UPDATE SET ignores = ignores + 1;
SELECT * FROM biases;
I don't know if this helps found this here --> Correct usage of the SQLite ON CONFLICT clause
Related
Given a need to store 4 digit random unique integers, how would I do a efficient insert of a large quality new numbers.
If the values are created outside sqlite there's a chance the values already existing in the database
What would be the best method to do such a task?
You could make the column where the numbers will be stored UNIQUE and use INSERT OR IGNORE on a SINGLE INSERT with multiple values (for efficiency). e.g. :-
INSERT OR IGNORE INTO rndm_id VALUES
('0001'),('0027'),('9999'),('0412'),('2108'),
('0001'), -- duplicate will be skipped
('3085') -- and so on
;
Note values have been enclosed in quotes to maintain 4 numerics. The Table was defined using :-
CREATE TABLE IF NOT EXISTS rndm_id (myid TEXT UNIQUE);
If you are considering a large number of values then may need to consider :-
Maximum Length Of An SQL Statement
The maximum number of bytes in the text of an SQL statement is limited
to SQLITE_MAX_SQL_LENGTH which defaults to 1000000. You can redefine
this limit to be as large as the smaller of SQLITE_MAX_LENGTH and
1073741824.
If an SQL statement is limited to be a million bytes in length, then
obviously you will not be able to insert multi-million byte strings by
embedding them as literals inside of INSERT statements. But you should
not do that anyway. Use host parameters for your data. Prepare short
SQL statements like this:
INSERT INTO tab1 VALUES(?,?,?); Then use the sqlite3_bind_XXXX()
functions to bind your large string values to the SQL statement. The
use of binding obviates the need to escape quote characters in the
string, reducing the risk of SQL injection attacks. It is also runs
faster since the large string does not need to be parsed or copied as
much.
The maximum length of an SQL statement can be lowered at run-time
using the sqlite3_limit(db,SQLITE_LIMIT_SQL_LENGTH,size) interface.
Limits In SQLite
Considering the comment
Is there a way I can do a select to give a a set of new values which I
can use then do a insert later?
So assuming that you wanted 1000 4 digit unique random values then the following may suffice :-
DROP TABLE IF EXISTS save_for_later; -- Drop the table
CREATE TEMP TABLE IF NOT EXISTS save_for_later (four_digit_random_value UNIQUE); -- Create a temporary table
-- Create a table with 1500 random rows
WITH RECURSIVE cte1 AS (
SELECT CAST((abs(random() % 10)||abs(random() % 10)||abs(random() % 10)||abs(random() % 10)) AS TEXT)
UNION ALL SELECT CAST((abs(random() % 10)||abs(random() % 10)||abs(random() % 10)||abs(random() % 10)) AS TEXT)
FROM cte1 LIMIT 1500
)
INSERT OR IGNORE INTO save_for_later SELECT * FROM cte1;
-- Later on extract the 1000 required rows.
SELECT * FROM save_for_later LIMIT 1000;
Suspected requirement
If the question were how can I insert a set number (300) of random 4 numeric unique values into a table (master) with existing data where the new values should also be unique in conjunction with the existing values
Then the following could do that (see note re limitations) :-
DROP TABLE IF EXISTS master; --
CREATE TABLE IF NOT EXISTS master (random_value TEXT UNIQUE);
-- Master (existing) Table populated with some values
INSERT OR IGNORE INTO master VALUES
('0001'),('0027'),('9999'),('0412'),('2108'),
('0001'), -- duplicate will be skipped
('3085') -- and so on
;
SELECT * FROM master; -- Result 1 show what's in the master table
-- Create a table to save the values for later
DROP TABLE IF EXISTS save_for_later; -- Drop the table
CREATE TEMP TABLE IF NOT EXISTS save_for_later (four_digit_random_value UNIQUE); -- Create a temporary table
-- Populate the vales to be saved for later excluding any values that already exist
-- 1500 rows perhaps excessive but very likely to result in 300 unique values
WITH RECURSIVE cte1(rndm) AS (
SELECT
CAST((abs(random() % 10)||abs(random() % 10)||abs(random() % 10)||abs(random() % 10)) AS TEXT)
UNION ALL
SELECT
CAST((abs(random() % 10)||abs(random() % 10)||abs(random() % 10)||abs(random() % 10)) AS TEXT)
FROM cte1
LIMIT 1500 --<<<<<< LIMIT important otherwise would be infinite
)
INSERT OR IGNORE INTO save_for_later
SELECT * FROM cte1
WHERE rndm NOT IN(SELECT * FROM master)
;
-- Later on extract the required rows (300 here) and insert them.
INSERT INTO master
SELECT * FROM save_for_later
LIMIT 300;
SELECT * FROM master; -- Should be 6 original/existing rows + 300 so 306 rows (although perhaps a chance that 300 weren't generated)
Note with 4 numerics there is a limitation of 10,000 possible values (0000-9999), so the more values that exist in the original table the greater the chance that there will be issues finding values that would be unique.
The above would result in :-
The first result the master table before generation of the new values :-
The result after adding the new values (original 6 rows + new 300 rows) :-
I need the equivalent of this Count with Case for a Firebird 3 database. I get an error when I try it:
SQL error code = -104.
Invalid usage of boolean expression.
I was just recently introduced to the Case command and I can't seem to rework it myself. I managed to get it to work with SQLite just fine.
The intent is to do an AND operation, the Where can't do an AND because the keywords are in rows.
SELECT Count((CASE WHEN keywords.keyword LIKE '%purchased%'
THEN 1 END) AND
(CASE WHEN keywords.keyword LIKE '%item%'
THEN 1 END)) AS TRows
FROM products
LEFT OUTER JOIN keywords_products ON
products.product_rec_id = keywords_products.product_rec_id
LEFT OUTER JOIN keywords ON
keywords_products.keyword_rec_id = keywords.keyword_rec_id
WHERE (keywords.keyword LIKE '%purchased%' OR
keywords.keyword LIKE '%item%')
I have three SQLite tables, a products table, a keywords_products table, and a keywords table.
CREATE TABLE products (
product_rec_id INTEGER PRIMARY KEY NOT NULL,
name VARCHAR (100) NOT NULL
);
CREATE TABLE keywords_products (
keyword_rec_id INTEGER NOT NULL,
product_rec_id INTEGER NOT NULL
);
CREATE TABLE keywords (
keyword_rec_id INTEGER PRIMARY KEY NOT NULL,
keyword VARCHAR (50) NOT NULL UNIQUE
);
The keywords_products table holds the the record id of a product and a record id of a keyword. Each product can be assigned multiple keywords in the keywords table.
The keyword table looks like this:
keyword_rec_id keyword
-------------- -----------
60 melee
43 scifi
87 water
The keywords_products table looks like this (one keyword can be assigned to many products):
keyword_rec_id product_rec_id
-------------- --------------
43 1
60 1
43 2
87 3
The products table looks like this:
product_rec_id name
-------------- --------------
1 Scifi Melee Weapons
2 Scifi Ray Weapon
3 Lake House
I'm assuming you want to count how many rows there are where both conditions are true.
The error occurs because you can't use AND between integer values. The values must be true booleans.
So, change your code to
Count((CASE WHEN keywords.keyword LIKE '%purchased%'
THEN TRUE END) AND
(CASE WHEN keywords.keyword LIKE '%item%'
THEN TRUE END))
However that is far too complex. You can simplify your expression to
count(nullif(
keywords.keyword LIKE '%purchased%' and keywords.keyword LIKE '%item%',
false))
The use of NULLIF is needed because COUNT will count all non-NULL values (as required by the SQL standard), and false is non-NULL as well. So to achieve the (assumed) desired effect, we transform false to NULL using NULLIF.
You have to use ONE single CASE expression with multiple WHEN branches.
Making Boolean functions of distinct CASE expressions just makes no sense - the CASE is not Boolean function itself.
You can see rules and an example at CASE.
case
when Age >= 18 then 'Yes'
when Age < 18 then 'No'
end;
Remake you two CASE clauses to a single CASE clause following this pattern.
However, you only use CASE when you can not move filters and conditions into standard part of SQL select. Normal approach would be to minimize data that SQL engine has to fetch, using pre-filtering. The CASE uses post-filtering, it makes SQL engine to fetch all the data, regardless if it needs it or not, and then discard the unneeded fetched data. That is redundant work slowing down the process.
In your case you already extracted the condition into WHERE clause, that is good.
SELECT
...
WHERE (keywords.keyword LIKE '%purchased%')
OR (keywords.keyword LIKE '%item%')
Since you pre-filter your data stream to always contain "item" or "purchase" then the CASE clause of yours would always return 1 on all rows selected under this WHERE pre-filtering. Hence - just remove the redundant CASE clause and put "1" instead.
SELECT Count(1)
FROM products
LEFT JOIN keywords_products ON products.product_rec_id = keywords_products.product_rec_id
LEFT JOIN keywords ON keywords_products.keyword_rec_id = keywords.keyword_rec_id
WHERE (keywords.keyword LIKE '%purchased%')
OR (keywords.keyword LIKE '%item%')
Now, given that WHERE clause is processed logically after JOINing, this your query de facto transformed LEFT JOINs into FULL JOINs ( your WHERE clause just discards rows with NULL "keyword" column values ) but aghain in unreliable and inefficient method. Since you do not want to have "keyword is NULL" kind of rows anyway - just convert your left joins to normal joins.
I understand that SQLite does not have If-Else condition check, and people have been using case statements to get around it. However I want to do a if condition check before executing a certain portion of the script, like the following:
IF (condition = true)
INSERT INTO tableA(A, B)
VALUES (a, b)
....
END
From what I have been trying, case statement doesn't seem to work. Is there any way I can accomplish the above in SQLite?
Thanks for all your help!
You could perhaps use an INSERT SELECT
INSERT INTO table SELECT ...;
The second form of the INSERT statement contains a SELECT statement
instead of a VALUES clause.
A new entry is inserted into the table for
each row of data returned by executing the SELECT statement.
If a
column-list is specified, the number of columns in the result of the
SELECT must be the same as the number of items in the column-list.
Otherwise, if no column-list is specified, the number of columns in
the result of the SELECT must be the same as the number of columns in
the table.
Any SELECT statement, including compound SELECTs and SELECT
statements with ORDER BY and/or LIMIT clauses, may be used in an
INSERT statement of this form.
extract from SQL As Understood By SQLite - INSERT
e.g.
INSERT into xxx
SELECT null as id,
CASE
WHEN filesize < 1024 THEN 'just a little bit'
WHEN filesize >= 1024 THEN 'quite a bit'
END AS othercolumn
FROM filesizes
WHERE filesize < 1024 * 1024
The above will insert rows into table xxx which consists of 2 columns id (rowid alias) and othercolumn according to the results (2 columns id (always set as null) and othercolumn) of the SELECT, which is selecting from the filesizes table where the value of the filesize column is less than 1024 * 1024 (1048576), thus conditionally inserting.
Furthermore, if the filesize is less than 1024 the othercolumn is set to just a little bit, if the filesize is greater than 1023 then the othercolumn is set to quite a bit. So making the conditional insert more complex.
Assuming the filesizes table were :-
The running the above would result in :-
I want to execute a SELECT query on a database table that has 6 key fields, let's assume they are keyA, keyB, ..., keyF.
As input parameters to my ABAP function module I do receive an internal table with exactly that structure of the key fields, each entry in that internal table therefore corresponds to one tuple in the database table.
Thus I simply need to select all tuples from the database table that correspond to the entries in my internal table.
Furthermore, I want to aggregate an amount column in that database table in exactly the same query.
In pseudo SQL the query would look as follows:
SELECT SUM(amount) FROM table WHERE (keyA, keyB, keyC, keyD, keyE, keyF) IN {internal table}.
However, this representation is not possible in ABAP OpenSQL.
Only one column (such as keyA) is allowed to state, not a composite key. Furthermore I can only use 'selection tables' (those with SIGN, OPTIOn, LOW, HIGH) after they keyword IN.
Using FOR ALL ENTRIES seems feasible, however in this case I cannot use SUM since aggregation is not allowed in the same query.
Any suggestions?
For selecting records for each entry of an internal table, normally the for all entries idiom in ABAP Open SQL is your friend. In your case, you have the additional requirement to aggregate a sum. Unfortunately, the result set of a SELECT statement that works with for all entries is not allowed to use aggregate functions. In my eyes, the best way in this case is to compute the sum from the result set in the ABAP layer. The following example works in my system (note in passing: using the new ABAP language features that came with 7.40, you could considerably shorten the whole code).
report zz_ztmp_test.
start-of-selection.
perform test.
* Database table ZTMP_TEST :
* ID - key field - type CHAR10
* VALUE - no key field - type INT4
* Content: 'A' 10, 'B' 20, 'C' 30, 'D' 40, 'E' 50
types: ty_entries type standard table of ztmp_test.
* ---
form test.
data: lv_sum type i,
lt_result type ty_entries,
lt_keys type ty_entries.
perform fill_keys changing lt_keys.
if lt_keys is not initial.
select * into table lt_result
from ztmp_test
for all entries in lt_keys
where id = lt_keys-id.
endif.
perform get_sum using lt_result
changing lv_sum.
write: / lv_sum.
endform.
form fill_keys changing ct_keys type ty_entries.
append :
'A' to ct_keys,
'C' to ct_keys,
'E' to ct_keys.
endform.
form get_sum using it_entries type ty_entries
changing value(ev_sum) type i.
field-symbols: <ls_test> type ztmp_test.
clear ev_sum.
loop at it_entries assigning <ls_test>.
add <ls_test>-value to ev_sum.
endloop.
endform.
I would use FOR ALL ENTRIES to fetch all the related rows, then LOOP round the resulting table and add up the relevant field into a total. If you have ABAP 740 or later, you can use REDUCE operator to avoid having to loop round the table manually:
DATA(total) = REDUCE i( INIT sum = 0
FOR wa IN itab NEXT sum = sum + wa-field ).
One possible approach is simultaneous summarizing inside SELECT loop using statement SELECT...ENDSELECT statement.
Sample with calculating all order lines/quantities for the plant:
TYPES: BEGIN OF ls_collect,
werks TYPE t001w-werks,
menge TYPE ekpo-menge,
END OF ls_collect.
DATA: lt_collect TYPE TABLE OF ls_collect.
SELECT werks UP TO 100 ROWS
FROM t001w
INTO TABLE #DATA(lt_werks).
SELECT werks, menge
FROM ekpo
INTO #DATA(order)
FOR ALL ENTRIES IN #lt_werks
WHERE werks = #lt_werks-werks.
COLLECT order INTO lt_collect.
ENDSELECT.
The sample has no business sense and placed here just for educational purpose.
Another more robust and modern approach is CTE (Common Table Expressions) available since ABAP 751 version. This technique is specially intended among others for total/subtotal tasks:
WITH
+plants AS (
SELECT werks UP TO 100 ROWS
FROM t011w ),
+orders_by_plant AS (
SELECT SUM( menge )
FROM ekpo AS e
INNER JOIN +plants AS m
ON e~werks = m~werks
GROUP BY werks )
SELECT werks, menge
FROM +orders_by_plant
INTO TABLE #DATA(lt_sums)
ORDER BY werks.
cl_demo_output=>display( lt_sums ).
The first table expression +material is your internal table, the second +orders_by_mat quantities totals selected by the above materials and the last query is the final output query.
I want to update a row if it exists in the table else insert it in SQLite in single query. From SQLite documentation I found out that we can use REPLACE command for achieving this.
I want to know how to use REPLACE if there are two or more conditions:
Example:
If I have table TABLE1 with following records:
Name Type InitialValue FinalValue
A 1 20 40
B 2 23 50
A 3 40 60
C 3 54 70
Here Combination of Name and Type will be unique.
I want to set initialvalue = 50 and finalvalue = 90 where name = A and Type = 3 if it exists, else insert it.
I am using this command but it's giving this error:
REPLACE INTO table1 (Name,Type,InitialValue,FinalValue) VALUES
('A',3,50,90 ) WHERE Name='A' AND Type = 3 ;
Error is:
near "WHERE": syntax error Unable to execute statement
How can I achieve my objective? Please help.
replace is just like insert, it just checks if there is duplicate key and if it is
it deletes the row, and inserts the new one, otherwise it just inserts
you can do this if there is for example unique index of (Name,Type) and if you type the following command
REPLACE INTO table1 (Name,Type,InitialValue,FinalValue) VALUES ('A',3,50,90 )
and there already exists a row with Name = 'A' and Type = 3 it will be replaced
http://www.sqlite.org/lang_createindex.html
CREATE UNIQUE INDEX idx_name_type ON table1(Name,Type)
EDIT:
a quick note - REPLACE always DELETES and then INSERTs, so it is never a very good idea to use it in heavy load because it needs exclusive lock when it deletes, and then when it inserts
some of the database engines have
INSERT ... ON DUPLICATE KEY UPDATE ...
sqlite 3 does not, but you can do try {} catch if insert fails
http://blog.client9.com/2007/11/sqlite3-and-on-duplicate-key-update.html
SQLite UPSERT - ON DUPLICATE KEY UPDATE