How to select gaps in my sqlite database - sqlite

I'm trying to write down a select to show me all the items that do not exist in my database. Say I have a table named TBL795 with a column named NRBEM which could not have any gaps in it.
Should be this way:
Those numbers are in sequence.
If they looked like this:
they would be wrong, because some items have not being inserted into the table.
In a table with thousand of items it would be very difficult to find out if there are any gaps and which are the missing items.
One solution would be this:
CREATE TABLE TESTE (
NRBEM VARCHAR(15))
feed it with a command like this:
INSERT INTO TESTE
WITH RECURSIVE
cnt(NRBEM) AS (VALUES(1) UNION ALL SELECT NRBEM+1 FROM cnt WHERE NRBEM <100000)
SELECT NRBEM FROM cnt A
and running this select
SELECT A.NRBEM FROM TESTE A LEFT JOIN TBL795 B
ON A.NRBEM = B.NRBEM
WHERE B.NRBEM IS NULL
I can see all the items that are missing in my table.
Since the command:
WITH RECURSIVE
cnt(NRBEM) AS (VALUES(1) UNION ALL SELECT NRBEM+1 FROM cnt WHERE NRBEM <100000)
SELECT NRBEM FROM cnt
create a virtual table I would like to run a select like this:
SELECT NRBEM FROM (
WITH RECURSIVE
cnt(NRBEM) AS (VALUES(1) UNION ALL SELECT NRBEM+1 FROM cnt WHERE NRBEM <100000)
SELECT NRBEM FROM cnt ) A LEFT JOIN TBL795 B
ON A.NRBEM = B.NRBEM
But this does not work.
This way:
SELECT X FROM (
WITH RECURSIVE
cnt(X) AS (VALUES(1) UNION ALL SELECT X+1 FROM cnt WHERE X <100000)
SELECT X FROM cnt ) A LEFT JOIN TBL795 B
ON A.X = B.NRBEM
it works, but does not select the right items.
So, how could I write this select?

It would be possible to use an outer join and filter out matches, but using set operations is simpler:
WITH RECURSIVE CNT(NRBEM) AS (...)
SELECT NRBEM
FROM CNT
WHERE NRBEM NOT IN (SELECT NRBEM
FROM tbl795);

I found out what I was doing wrong.
If I cast nrbem as number the select works.
SELECT A.X FROM (
WITH RECURSIVE
cnt(X) AS (VALUES(1) UNION ALL SELECT X+1 FROM cnt WHERE X <100000)
SELECT X FROM cnt ) A LEFT JOIN ( SELECT CAST( NRBEM AS NUMBER ) AS NRBEM FROM TBL795 ) B
ON A.X = B.NRBEM
WHERE B.NRBEM IS NULL
If I want to check the range from item 2400 to 2700 to see if there are any gaps I can do this:
SELECT A.X FROM (
WITH RECURSIVE
cnt(X) AS (VALUES(2400) UNION ALL SELECT X+1 FROM cnt WHERE X < (2700))
SELECT X FROM cnt ) A LEFT JOIN ( SELECT CAST( NRBEM AS NUMBER ) AS NRBEM FROM TBL795 WHERE NRBEM >= 2400 and nrbem <= 2700 ) B
ON A.X = B.NRBEM
WHERE B.NRBEM IS NULL
LIMIT ( 2700 - 2400 + 1 )

Related

How to use with statement in this query?

In this query, I want to use with-statement. I have a subquery that calculates A union all B and I want to use it with with-statement. But when I use with-statement I face the error that says "table or view does not exist".
what surprises me is when I replace the first part with with-statement it works correctly. But when I replace the second part, I face this error!!
select
deposit.BRNCH_COD||'-'||deposit.DP_TYPE_COD||''||deposit.CUSTOMER_NUM||'-
'||deposit.DEPOSIT_SERIAL AS DEPOSIT_NUMBER,
deposit.IBAN AS IBAN,
deposit.CURRENCY_DESC AS DEPOSIT_CURRCOD,
deposit.BRNCH_COD AS BRNCH_COD,
MAIN_7.Still_Days
AS Still_Lenght,
to_char(MAIN_7.Startdate, 'yyyy/mm/dd' ,'nls_calendar=persian') AS
START_DATE,
MAIN_7.AMOUNT
AS TOTAL_AMOUNT,
MAIN_7.TRN_Count
AS TRN_Count
from
(
select Trans_Table.DEPOSIT_KEY AS DEPOSIT_KEY,
Trans_Table.TRN_Start_DATE AS Startdate,
MAX(Active_Time_Table.EFFECTIVE_DATE) AS Lastdate,
H.PASSIVE_DAYS AS Still_Days,
SUM(Active_Time_Table.AMOUNT) AS AMOUNT,
Count(Active_Time_Table.AMOUNT) AS TRN_Count
from
(
Select F.DEPOSIT_KEY,
SUM (F.AMOUNT) AS TRN_AMOUNT,
MIN (F.EFFECTIVE_DATE) AS TRN_Start_DATE
from
(
A
union all
B
)F
Group by (F.DEPOSIT_KEY)
Having ( SUM (F.AMOUNT) >10000000000)
)Trans_Table
inner join
H
on (Trans_Table.DEPOSIT_KEY = H.DEPOSIT_KEY and
Trans_Table.TRN_Start_DATE-1 = H.EFFECTIVE_DATE)
inner join
(
A
union all
B
)Active_Time_Table
on (Trans_Table.DEPOSIT_KEY = Active_Time_Table.DEPOSIT_KEY and
Active_Time_Table.EFFECTIVE_DATE - Trans_Table.TRN_Start_DATE< 4 and
Active_Time_Table.EFFECTIVE_DATE - Trans_Table.TRN_Start_DATE>=0)
group by ( Trans_Table.DEPOSIT_KEY ,
Trans_Table.TRN_Start_DATE,H.PASSIVE_DAYS)
Having (SUM(Active_Time_Table.AMOUNT)) > 10000000000
)MAIN_7
inner join dimamldeposit deposit
on deposit.DEPOSIT_KEY = MAIN_7.DEPOSIT_KEY
***********************************************************
with rep as
(A union all B)
select
deposit.BRNCH_COD||'-'||deposit.DP_TYPE_COD||'-
'||deposit.CUSTOMER_NUM||'-'||deposit.DEPOSIT_SERIAL AS DEPOSIT_NUMBER,
deposit.IBAN AS IBAN,
deposit.CURRENCY_DESC AS DEPOSIT_CURRCOD,
deposit.BRNCH_COD AS BRNCH_COD,
MAIN_7.Still_Days AS Still_Lenght,
to_char(MAIN_7.Startdate, 'yyyy/mm/dd' ,'nls_calendar=persian') AS START_DATE,
MAIN_7.AMOUNT AS TOTAL_AMOUNT,
MAIN_7.TRN_Count AS TRN_Count
from
(
select Trans_Table.DEPOSIT_KEY AS DEPOSIT_KEY,
Trans_Table.TRN_Start_DATE AS Startdate,
MAX(rep.EFFECTIVE_DATE) AS Lastdate,
H.PASSIVE_DAYS AS Still_Days,
SUM(rep.AMOUNT) AS AMOUNT,
Count(rep.AMOUNT) AS TRN_Count
from
(
Select rep.DEPOSIT_KEY,
SUM (rep.AMOUNT) AS TRN_AMOUNT,
MIN (rep.EFFECTIVE_DATE) AS TRN_Start_DATE
from
rep
Group by (rep.DEPOSIT_KEY)
Having ( SUM (rep.AMOUNT) >10000000000)
)Trans_Table
inner join
H
on (Trans_Table.DEPOSIT_KEY = H.DEPOSIT_KEY and Trans_Table.TRN_Start_DATE-1 = H.EFFECTIVE_DATE)
inner join
rep rep
on (Trans_Table.DEPOSIT_KEY = rep.DEPOSIT_KEY and rep.EFFECTIVE_DATE - Trans_Table.TRN_Start_DATE< 4 and rep.EFFECTIVE_DATE - Trans_Table.TRN_Start_DATE>=0)
group by ( Trans_Table.DEPOSIT_KEY , Trans_Table.TRN_Start_DATE,H.PASSIVE_DAYS)
Having (SUM(rep.AMOUNT)) > 10000000000
)MAIN_7
inner join dimamldeposit deposit
on deposit.DEPOSIT_KEY = MAIN_7.DEPOSIT_KEY
That's a lot of code, but - to make it simple, I'd suggest you use WITH factoring clause as the first command, include all tables you use into it, and then - as the final SELECT - fetch data from all those CTEs. Something like this:
with
a as (select ... from ...),
b as (select ... from ...),
f as (select ... from ...),
...
select a.col1, b.col2, f.col3
from a join b on a.id = b.id
left join f on f.id = b.id
where ...

Teradata macro with volatile table and CTE to insert data into a table

I need to create a teradata macro to extract information into a volatile table first, then do CTE to extract data from this volatile table and insert into a teradata table, tried different ways all fail, appreciate help!
CREATE MACRO database.macro_insertion_tablename AS (
CREATE VOLATILE TABLE vt AS
(
SELECT
id, bu,
CONCAT(TO_CHAR(comment_date, 'yyyy-mm-dd HH24:MI:SS'), ' ', action) AS full_action,
ROW_NUMBER() OVER (PARTITION BY id ORDER BY date DESC) AS row_num,
COUNT(*) OVER (PARTITION BY id) as cnt
FROM database.table1
) WITH DATA UNIQUE PRIMARY INDEX(id, row_num) ON COMMIT PRESERVE ROWS;
WITH RECURSIVE cte (id, bu, act, rn) AS
(
SELECT
id, bu
,CAST(full_action AS VARCHAR(5000)) AS full_action
,row_num
FROM vt
WHERE row_num = cnt
UNION ALL
SELECT
vt.id, vt.bu
,cte.act || ' / ' || vt.full_action
,vt.row_num
FROM vt
JOIN cte On vt.id = cte.id AND vt.row_num = cte.rn - 1
)
INSERT INTO database.table (id, bu, full_action)
SELECT id, bu, act
FROM cte
WHERE rn = 1;
DROP TABLE vt;
);
DDL must be the only statement in a Teradata Macro.
As workaround you could switch to a Global Temporary Table which is defined once and then you simply Insert/Select into it instead of CREATE VOLATILE TABLE.
But in your case there's no need for a temp table plus inefficient recursive processing to get a "group concat":
SELECT id, max(bu) -- maybe min(bu)?
XmlAgg(Concat(To_Char(comment_date, 'yyyy-mm-dd HH24:MI:SS'), ' ', action)
ORDER BY comment_date) (VARCHAR(5000)) AS full_action
FROM database.table1
GROUP BY 1
will give you a similar result.
To follow up on my comments, you should be able to define multiple CTEs in the same statement. It may be tricky getting the RECURSIVE CTE to work, but it sounds like it's possible. Maybe something like this:
CREATE MACRO database.macro_insertion_tablename AS (
WITH vt (id, bu, full_action, row_num, cnt) AS
(
SELECT
id, bu,
CONCAT(TO_CHAR(comment_date, 'yyyy-mm-dd HH24:MI:SS'), ' ', action) AS full_action,
ROW_NUMBER() OVER (PARTITION BY id ORDER BY date DESC) AS row_num,
COUNT(*) OVER (PARTITION BY id) as cnt
FROM database.table1
),
RECURSIVE cte (id, bu, act, rn) AS
(
SELECT
id, bu
,CAST(full_action AS VARCHAR(5000)) AS full_action
,row_num
FROM vt
WHERE row_num = cnt
UNION ALL
SELECT
vt.id, vt.bu
,cte.act || ' / ' || vt.full_action
,vt.row_num
FROM vt
JOIN cte On vt.id = cte.id AND vt.row_num = cte.rn - 1
)
INSERT INTO database.table (id, bu, full_action)
SELECT id, bu, act
FROM cte
WHERE rn = 1;
);
I don't have a Teradata system to test with, so not 100% it will work as-is, but give it a try. You may need to change RECURSIVE to WITH RECURSIVE and also the ordering of the CTE queries (i.e. put the RECURSIVE one first). Take a look at these two links:
Teradata Forum - Multiple With Clause
teradata Forum - Common Table Expressions

SQLite Inner Join seems to produce duplicate rows and possibly discards some other rows

I have some data about shops that looks something like:
where the unimportant data is omitted.
I have an existing SQLite query:
select s.COMSERNO, s.COMNAME, COMALTNAME, s.COMCODE, s.COMADDR, s.COMPCODE, s.COMCITY, x.lastVisit, y.memoText, l.COMCALL
from TCOMPANY s
left outer join ( select COMSERNO, max(CALEDATE || CALESTART) as lastVisit from TCALENTR where SALMSERN='000000000000019' and (CALEDATE || CALESTART) < strftime('%Y%m%d%H%M', 'now', 'localtime') group by COMSERNO) x on (x.COMSERNO=s.COMSERNO)
left outer join ( select MEMOSERN1 as comSerno, max(MEMOTEXT) as memoText from TMEMO where MEMOTYPE='0' and 0<length(MEMOTEXT) and MEMOSERN2 in ('Notes')group by MEMOSERN1) y on (y.comSerno=s.COMSERNO)
left outer join LSALCOM l on l.COMSERNO=s.COMSERNO
and SALMSERN='000000000000019'
order by s.COMNAME, s.COMSERNO
The above returns 174 rows.
I want to show the 'Chain' for a shop, if it has one, so the data should look something like this:
I have another table, LCOMCOM and for each COMSERNO in my original result set (each shop) I need to determine if this shop is part of a chain and if it is, display that chain.
I join LCOMCOM on COMSERN2, and if the column LCOMATR1 has the value 'Chain' then it means the store COMSERN2 is a chain. I then take COMSERN1 from LCOMCOM and use the original table, TCOMPANY to look up and show the Chain name. Here is my code
case when z.lcomatr1 = 'Chain' then (select p.COMNAME from TCOMPANY p inner join LCOMCOM q on (p.COMSERNO=q.COMSERN2) where q.COMSERN1 = z.comsern1) else '' end as Chain
Here is the LCOMCOM join:
inner join LCOMCOM z on (s.COMSERNO=z.COMSERN2)
I tried adding the above two pieces into my query but it doesn't do what I expected:
select s.COMSERNO, s.COMNAME, COMALTNAME, s.COMCODE, s.COMADDR, s.COMPCODE, s.COMCITY, x.lastVisit, y.memoText, l.COMCALL,
case when z.lcomatr1 = 'Chain' then (select p.COMNAME from TCOMPANY p inner join LCOMCOM q on (p.COMSERNO=q.COMSERN2) where q.COMSERN1 = z.comsern1) else '' end as Chain
from TCOMPANY s
left outer join ( select COMSERNO, max(CALEDATE || CALESTART) as lastVisit from TCALENTR where SALMSERN='000000000000019' and (CALEDATE || CALESTART) < strftime('%Y%m%d%H%M', 'now', 'localtime') group by COMSERNO) x on (x.COMSERNO=s.COMSERNO)
left outer join ( select MEMOSERN1 as comSerno, max(MEMOTEXT) as memoText from TMEMO where MEMOTYPE='0' and 0<length(MEMOTEXT) and MEMOSERN2 in ('Notes')group by MEMOSERN1) y on (y.comSerno=s.COMSERNO)
left outer join LSALCOM l on l.COMSERNO=s.COMSERNO
inner join LCOMCOM z on (s.COMSERNO=z.COMSERN2)
and SALMSERN='000000000000019'
order by s.COMNAME, s.COMSERNO
The above returns 548 rows, not 174. Some shops are now in 4 rows, like so:
And some shops are now in 2 rows, like so:
Adding DISTINCT after the initial SELECT reduces the row count to 348 and it now looks like all the Chain stores are in 2 rows.
I'm not sure what's going on here but presumably referring to TCOMPANY twice in both the SELECTs is wrong. Can anybody tell me how to get 174 rows, with the chain, if appropriate?
The CASE statement wasn't needed and the INNER JOIN was moved into the bracketed SELECT:
select DISTINCT s.COMSERNO, s.COMNAME, COMALTNAME, s.COMCODE, s.COMADDR, s.COMPCODE, s.COMCITY, x.lastVisit, y.memoText, l.COMCALL,
(select p.comname from LCOMCOM q inner join TCOMPANY p on (p.COMSERNO=q.COMSERN1) where q.COMSERN2 = s.comserno and q.lcomatr1 = 'Chain') as Chain
from TCOMPANY s
left outer join ( select COMSERNO, max(CALEDATE || CALESTART) as lastVisit from TCALENTR where SALMSERN='000000000000019' and (CALEDATE || CALESTART) < strftime('%Y%m%d%H%M', 'now', 'localtime') group by COMSERNO) x on (x.COMSERNO=s.COMSERNO)
left outer join ( select MEMOSERN1 as comSerno, max(MEMOTEXT) as memoText from TMEMO where MEMOTYPE='0' and 0<length(MEMOTEXT) and MEMOSERN2 in ('Notes')group by MEMOSERN1) y on (y.comSerno=s.COMSERNO)
left outer join LSALCOM l on l.COMSERNO=s.COMSERNO
and SALMSERN='000000000000019'
order by s.COMNAME, s.COMSERNO

select records that not exists in another table

I would like to get all values in x that do not exists in table TableA.
I've tried following queries that is returning blank.
select x.num from
(
select '888888' as num from dual union all
select '111111' as num from dual
) x
left outer join TableA a on (a.number = x.num)
where a.number is null
select x.num
FROM
(
select '888888' as num from dual union all
select '111111' as num from dual
) x
where x.num not in
(
select a.number from TableA a where x.num = a.number
)
Table
| TableA.number |
-------------
111111
333333
The expected result is that only '888888' is returned in this case.
I can't see why this shouldn't work, where have i've done wrong?
Try this
with cte as
(
select '888888' as num from dual union all
select '111111' as num from dual
)
select cte.num
from cte
left join TableA a on (a.number = cte.num)
where a.number is null

multi table insert fecthing value from another table

I have tables - FAKE_CUST, PRE_CUST, NORMAL_CUST. Based on the constraint present in FAKE_CUST table, the value has to be brought to either PRE_CUST or NORMAL_CUST.
I am using the follwing code:
INSERT ALL
INTO PRE_CUST(CUST_ID, TOTAL_COST_TRANS) (SELECT DISTINCT CUST_ID,C FROM (
SELECT CUST_ID, SUM(COST_TRANS) OVER (PARTITION BY CUST_ID) as C FROM FAKE_CUST) WHERE C>1000)
INTO NORMAL_CUST(CUST_ID, TOTAL_COST_TRANS) (SELECT DISTINCT CUST_ID,C FROM (
SELECT CUST_ID, SUM(COST_TRANS) OVER (PARTITION BY CUST_ID) as C FROM FAKE_CUST) WHERE C<1000)
SELECT 1 FROM DUAL;
The syntax which you have used for multi-table insert is not correct to start with.
The WHEN clause, which evaluates the condition for insert, has to come before insert_intoclause.
Your query should look like this:
INSERT ALL
WHEN c > 1000 THEN
INTO pre_cust (cust_id, total_cost_trans)
WHEN c < 1000 THEN
INTO normal_cust (cust_id, total_cost_trans)
SELECT CUST_ID, SUM(COST_TRANS) OVER (PARTITION BY CUST_ID) as C
FROM FAKE_CUST

Resources