I'm having a difficult time getting the UPDATE command in Sqlite to update all records with a new value it finds in another table using a subquery select statement. It incorrectly updates every column in Table1 with the first value it finds in column Table2.
When I run the select portion of the query alone it runs fine and returns all the proper values. It basically looks up the nearest value in multiples of 300000 from a lookup table called Adjustment.
code:
Update TEMP1
set New_position =
(
select (Temp1.Col1 + Adjustment.Offset) as NewValue
from Adjustment, TEMP1
where Adjustment.LookupValue = cast(TEMP1.Col1 / 300000 as Int) * 300000
)
You are trying to use a correlated subquery:
A SELECT statement used as either a scalar subquery or as the right-hand operand of an IN, NOT IN or EXISTS expression may contain references to columns in the outer query. Such a subquery is known as a correlated subquery. A correlated subquery is reevaluated each time its result is required. An uncorrelated subquery is evaluated only once and the result reused as necessary.
But your subquery does not actually contain a reference to columns in the outer query; the FROM Temp1 in the inner query is a new instance of the Temp1 table, and Temp1.Col1 refers to that. The value of Temp1.Col1 is from some random row in Temp1.
Just drop the Temp1 from the FROM clause:
UPDATE Temp1
SET New_position = (SELECT Temp1.Col1 + Adjustment.Offset
FROM Adjustment
WHERE LookupValue = CAST(Temp1.Col1 / 300000 AS INT) * 300000);
Related
I need to merge some randomly uppercased data that has been collected in an SQLite table key_val, such that key is always lowercase and no vals are lost. There is a unique compound index on key,val.
The initial data looks like this:
key|val
abc|1
abc|5
aBc|1
aBc|5
aBc|3
aBc|2
AbC|1
abC|3
The result after the merge would be
key|val
abc|1
abc|2
abc|3
abc|5
In my programmer brain, I would
for each `key` with upper case letters;
if a lower cased `key` is found with the same value
then delete `key`
else update `key` to lower case
Re implementing the loop has a sub query for each row found with upper case letters, to check if the val already exists as a lower case key
If it does, I can delete the cased key.
From there I can UPDATE key = lower(key) as the "duplicates" have been removed.
The first cut of the programming method of finding the dupes is:
SELECT * FROM key_val as parent
WHERE parent.key != lower(parent.key)
AND 0 < (
SELECT count(s.val) FROM key_val as s
WHERE s.key = lower(parent.key) AND s.val = parent.val
)
ORDER BY parent.key DESC;
I'm assuming there's a better way to do this in SQLite? The ON CONFLICT functionality seems to me like it should be able to handle the dupe deletion on UPDATE but I'm not seeing it.
First delete all the duplicates:
DELETE FROM key_val AS k1
WHERE EXISTS (
SELECT 1
FROM key_val AS k2
WHERE LOWER(k2.key) = LOWER(k1.key) AND k2.val = k1.val AND k2.rowid < k1.rowid
);
by keeping only 1 combination of key and val with the min rowid.
It is not important if you kept the key with all lower chars or not, because the 2nd step is to update the table:
UPDATE key_val
SET key = LOWER(key);
See the demo.
Honestly it might just be easier to create a new table and then insert into it. As it seems you really just want a distinct select here, use:
INSERT INTO kev_val_new ("key", val)
SELECT DISTINCT LOWER("key"), val
FROM key_val;
Once you have populated the new table, you may drop the old one, and then rename the new one to the previous name:
DROP TABLE key_val;
ALTER TABLE key_val_new RENAME TO key_val;
I agree with #Tim that it would be easire to re-create table using simple select distict lower().. statement, but that's not always easy if table has dependant objects (indexes, triggers, views). In this case this can be done as sequence of two steps:
insert lowered keys which are not still there:
insert into t
select distinct lower(tr.key) as key, tr.val
from t as tr
left join t as ts on ts.key = lower(tr.key) and ts.val = tr.val
where ts.key is null;
now when we have all lowered keys - remove other keys:
delete from t where key <> lower(key);
See fiddle: http://sqlfiddle.com/#!5/84db50/11
However this method assumes that key is always populated (otherwise it would be a strange key)
If vals can be null then "ts.val = tr.val" should be replaced with more complex stuff like ifnull(ts.val, -1) = ifnull(tr.val, -1) where -1 is some unused value (can be different). If we can't assume any unused value like -1 then it should be more complex check for null / not null cases.
I came across a situation where i have to delete a column from a table based on a condition from other table
Let me break it down to you!
There is a master table called MORTALITY (containing info regarding deceased individuals)
And another table called INC_MORTALITY (incremental mortality) table which is refreshed on a weekly basis
Note: Both the tables have similar format
So this week’s new records, containing both additional deceased individuals as well as updates of old data for previously delivered records. This is a single file with a column (OP_DIRECTIVE) specifying if it is an “add” or “delete” record.
Processing Weekly Files
To incorporate the weekly update file, we need to execute the following steps in order.
1. Delete rows in the master table which have a OP_DIRECTIVE = 'D' as the operation in the weekly update. For a given delete row, you should delete a single row in the master table which matches the delete record on all fields aside from the “D” operation column. Warning: please ensure you only delete, or mark as deleted, one record, even if more than one historical record fully matches this new delete record.
2. Add rows in the master table which appear in the “Add” file.
Upon completion of these steps, your master table should be the most up to date master of deaths.

(Note: THESE TABLES DOES NOT HAVE PRIMARY KEYS)
SO WHAT I TRIED:
DEL FROM MORTALITY MI
WHERE MI.DATA_SOURCE = INC_MORTALITY.DATA_SOURCE
AND MI.DD_IMP_FLAG = INC_MORTALITY.DD_IMP_FLAG
AND MI.DOB = INC_MORTALITY.DOB
AND MI.DOD = INC_MORTALITY.DOD
AND MI.DEATH_VERIFICATION = INC_MORTALITY.DEATH_VERIFICATION
AND MI.GENDER_PROBABILITY = INC_MORTALITY.GENDER_PROBABILITY
AND MI.GENDER = INC_MORTALITY.GENDER
AND MI.TOKEN_1 = INC_MORTALITY.TOKEN_1
AND MI.TOKEN_2 = INC_MORTALITY.TOKEN_2
AND MI.TOKEN_4 = INC_MORTALITY.TOKEN_4
AND MI.TOKEN_5 = INC_MORTALITY.TOKEN_5
AND MI.TOKEN_7 = INC_MORTALITY.TOKEN_7
AND MI.TOKEN_16 = INC_MORTALITY.TOKEN_16
AND MI.TOKEN_KEY = INC_MORTALITY.TOKEN_KEY
AND INC_MORTALITY.OP_DIRECTIVE = 'D'
The above Delete statement will delete all the rows satisfying the conditions, my requirement is to delete only one record even if more than one historical record fully matches this new delete record,
and if i include ROW NUMBER() stmt like below my DELETE stmt is not working
QUALIFY ROW_NUMBER() OVER (PARTITION BY MI.DATA_SOURCE,MI.DOB,MI.DOD
ORDER BY MI.DOD DESC ) = 1
Any suggestions on how to approach this scenario, Thanks!!
Approach to solution: Copy unmatched rows to a work table, then truncate the original table and replace with contents of the work table. One way to identify unmatched rows would be to tag each of the input rows in a set of duplicates with a unique number, something like this:
INSERT work_table SELECT MI.col1, MI.col2, ...
FROM
(SELECT M.*,
ROW_NUMBER() OVER (PARTITION BY <join cols> ORDER BY <some col(s)>) AS ROWNUM
FROM MORTALITY M) MI
LEFT JOIN
(SELECT I.*,
ROW_NUMBER() OVER (PARTITION BY <join cols> ORDER BY <some col(s)>) AS ROWNUM
FROM INC_MORTALITY I
WHERE OP_DIRECTIVE='D') INC
ON MI.join_col1 = INC.join_col1
AND MI.join_col2 = INC.join_col2
...
AND MI.ROWNUM = INC.ROWNUM
WHERE INC.ROWNUM IS NULL /* "anti-join" keeps only unmatched rows */
;
DELETE FROM MORTALITY;
INSERT MORTALITY SELECT * FROM work_table;
If INC_MORTALILTY never has duplicates, then you can eliminate numbering that relation and change the last join condition to MI.ROWNUM = 1 and use one of the other JOIN columns for the NULL check.
I have the following columns in a SQLite DB.
id,ts,origin,product,bid,ask,nextts
1,2016-10-18 20:20:54.733,SourceA,Dow,1.09812,1.0982,
2,2016-10-18 20:20:55.093,SourceB,Oil,7010.5,7011.5,
3,2016-10-18 20:20:55.149,SourceA,Dow,18159.0,18161.0,
How can I populate the 'next timestamp' column (nextts) with the next timestamp for the same product (ts), from the same source? I've been trying the following, but I can't seem to put a subquery in an UPDATE statement.
UPDATE TEST a SET nextts = (select ts
from TEST b
where b.id> a.id and a.origin = b.origin and a.product = b.product
order by id asc limit 1);
If I call this, I can display it, but I haven't found a way of updating the value yet.
select a.*,
(select ts
from TEST b
where b.id> a.id and a.origin = b.origin and a.product = b.product
order by id asc limit 1) as nextts
from TEST a
order by origin, a.id;
The problem is that you're using table alias for table in UPDATE statement, which is not allowed. You can skip alias from there and use unaliased (but table-name prefixed) reference to its columns (while keeping aliased references for the SELECT), like this:
UPDATE TEST
SET nextts = (
SELECT b.ts
FROM TEST b
WHERE b.id > TEST.id AND
TEST.origin = b.origin AND
TEST.product = b.product
ORDER BY b.id ASC
LIMIT 1
);
Prefixing unaliased column references with the table name is necessary for SQLite to identify that you're referencing to unaliased table. Otherwise the id column whould be understood as the id from the closest[*] possible data source, in which case it's the aliased table (as b alias), while we're interested in the unaliased table, therefore we need to explicitly tell SQLite that.
[*] Closest data source is the one listed in the same query, or parent query, or parent's parent query, etc. SQLite is looking for the first data source (going from inner part to the outside) in the query hierarchy that defines this column.
I want to select Column from Table based on Its Order
like
create Table Products
(
ProductId Int,
ProductName varchar(50)
)
lets Say I don't Know the name of the second column.
How I can get it like :
Select Col1,Col2 From Product
For SQL Server:
You can't do this in the SELECT clause. You can't select based on the order number of the column. You have to list the columns' names you need to select explicitly, otherwise, use SELECT * to list all. Me be if you are using a data reader object or any other ado.net methods to get the data from database you can do something like this, but this will be based on the column names list listed in your SQL statement.
However, you can do something like this dynamically, by reading columns' metadata ordinal_position from information_schema.columns as explained in the following answer:
Is it possible to select sql server data using column ordinal position?
But, you can do this in the ORDER BY clause. You can ORDER BY column number:
SELECT *
FROM TableName
ORDER BY 2; -- for col2
But this is not recommended to use in ORDER BY or in the SELECT (if any). Furthermore, columns order is not significant in the relational model.
Update: If you want to select at least 3 columns from any table parameter passed to your stored procedure. Try this as follows:
Your stored procedure supposed to receive a parameter #tableNameParam. The folowing code should return the first three columns from the #tablenameParam passed to the stored procedure:
DECLARE #col1 AS VARCHAR(100);
DECLARE #col2 AS VARCHAR(100);
DECLARE #col3 AS VARCHAR(100);
DECLARE #tableNameParam AS VARCHAR(50) = 'Tablename';
DECLARE #sql AS VARCHAR(MAX) ;
SELECT #col1 = column_name FROM information_schema.columns
WHERE table_name = #tableNameParam
AND ordinal_position = 1;
SELECT #col2 = column_name FROM information_schema.columns
WHERE table_name = #tableNameParam;
AND ordinal_position = 2;
SELECT #col3 = column_name FROM information_schema.columns
WHERE table_name = #tableNameParam;
AND ordinal_position = 3;
SET #sql = 'SELECT ' + col1 + ',' + col2 ' + 'col3 ' + FROM ' + #tablename;
you always can do
select * from Product
I'd like to share the following code as a solution to CRUD processing on Ordinal Position within a table. I had this problem today and it took me quite a long time to research and find a working solution. Many of the posted answers indicated that it was not possible to interact with the tables columns on an Ordinal bases but as indicated in the post above using the information_schema table will allow using the column position.
My situation was interacting with a table populated through the use of a pivot view so the columns are always changing based on the data, which is fine in a view result but when the dataset is stored into a table the columns are dynamic. The column names are a Year-Month combination such as 201801, 201802 with an Item Number as a primary key. This pivot table is to indicate manufacturing quantities by Year-Month on a rolling 12 month period so each month the column names with change/shift which changes their ordinal position when the table is rebuilt each month.
The Pivot view is used to build the Staging table, The Staging table is used to build the
Target table so the ordinal position of the staging and target tables are lined up with the same ordinal position.
Declare #colname Varchar(55) -- Column Name
Declare #ordpos INT -- Ordinal Position
Declare #Item Varchar(99) -- PK
Declare #i INT -- Counter
Declare #cnt INT -- Count
Declare #ids table(idx int identity(1,1), Item Varchar(25))
-- Item List
Insert INTO #ids Select Item From DBName.Schema.TableName
select #i = min(idx) - 1, #cnt = max(idx) from #ids
-- Row Loop
While #i < #cnt
Begin
Select #i = #i + 1
Set #ordpos=3
Set #Item = (select Item from #ids where idx = #i)
-- Column Loop
While #ordpos < 27
Begin
Select #colname =column_name From INFORMATION_SCHEMA.Columns Where table_name='TargetTable' and ordinal_position=#ordpos
Exec ('Update TargetTable set ['+#colname+']= (Select ['+#colname+'] From StagingTable Where Item='''+#Item+''') where Item='''+#Item+'''')
Set #ordpos=#ordpos + 1
End -- End Column Loop
End -- End Row Loop
The code here will loop through the Item matrix by rows and by columns and uses Dynamic SQL to build the action, in this case the action is an update but it could just as easily be a select. Each column is processed through the While Loop and then loops through the next row. This allows updates to a specific cell in the matrix by (Item X YearMonth) without actually knowing what the column name at a given position.
The one concern is that depending on the size of the data in this matrix it can be SLOW. I just wanted to show this as a way to use unknown column names in an ordinal position.
I have 2 tables, one is indexing the other.
I am querying Table#1, and it has one column (string) that has an ID in it that corresponds to a unique row in Table#2. Im trying to write a query in Sqlite that allows me to retrieve the value from Table#2 if the column value in Table#1 is not an empty string.
Kinda like:
"SELECT TMake,TModel,TTrim,IYear,[%q] AS TPart1 FROM AppGuide WHERE TPart1 != ''"
But instead of retrieving the Index value (TPart1) Id like to get the string from Table#2.
Is this possible?
Any help is appreciated.
You could use a correlated subquery:
SELECT TMake,
TModel,
...,
(SELECT stringvalue
FROM Table2
WHERE Table2.ID = Table1.TPart1)
FROM Table1
WHERE Table1.TPart1 != ''
However, these are rather slow to execute, so you'd better use a join (this returns exactly the same result):
SELECT Table1.TMake,
Table1.TModel,
...,
Table2.stringvalue
FROM Table1 LEFT JOIN Table2 ON Table1.TPart1 = Table2.ID
WHERE Table1.TPart1 != ''
If you don't want to get records from Table1 that have no matching Table2 record, drop the LEFT.