I have a SQL query in teradata that returns a results set of ~160m rows in (I guess) a reasonable time: dependent on how good a day the server is having it runs between 10-60 minutes.
I recently got access to space to save it as a table, however using my initial query and the "insert into " command I get error 2646-no more spool.
query structure is
insert into <test_DB.tablename>
with smaller_dataset as
(
select
*
from
(
select
items
,case items
from
<Database.table>
QUALIFY ROW_NUMBER() OVER (PARTITION BY A,B ORDER BY C desc , LAST_UPDATE_DTM DESC) = 1
where 1=1
and other things
) T --irrelevant alias for subquery
QUALIFY ROW_NUMBER() OVER (PARTITION BY A, B ORDER BY C desc) = 1)
, employee_table as
(
select
items
,max(J1.field1) J1_field1
,max(J2.field1) J2_field1
,max(J3.field1) J3_field1
,max(J4.field1) J4_field1
from smaller_dataset S
self joins J1,J2,J3,J4
group by
non-aggregate items
)
select
items
case items
from employee_table
;
How can I break up the return into smaller chunks to prevent this error?
If more than one ID for a match select id with max(date) .
if more than one id for above max dates select max(ID)
Go to TABLE1 to get high-dated IDS
Step 1 : identifying IDs which are having max dates
CREATE MULTISET VOLATILE TABLE TEST_VT
AS
(
SELECT
TABLE1.ID ,
TABLE1.KEY1,
TABLE1.STRT_DT,
TABLE1.TERM_DT,
Rank() OVER (PARTITION BY TABLE1.KEY1 ORDER BY TABLE1.STRT_DT DESC , TABLE1.ID DESC) RNK
FROM TABLE2
INNER JOIN TABLE1
ON TABLE2.KEY1=TABLE1.KEY1
WHERE TABLE1.TERM_DT='8888-12-31'
QUALIFY RNK=1
GROUP BY
1,2,3,4
)
WITH DATA PRIMARY INDEX(ID,KEY1) ON COMMIT PRESERVE ROWS;
Go to TABLE1 to get high-dated KEY1S associated with IDS from ABOVE STEP
Step 2 : If there is more than one max date take max ID out of all matched records in above step
SELECT
TABLE1.ID ,
TABLE1.KEY1,
TABLE1.STRT_DT,
TABLE1.TERM_DT,
RANK() OVER (PARTITION BY TABLE1.KEY1 ORDER BY TABLE1.STRT_DT DESC , TABLE1.ID DESC) AS RNK
FROM TABLE1
INNER JOIN TEST_VT
ON TEST_VT.ID=TABLE1.ID
INNER JOIN TABLE3
ON TABLE3.KEY1=TABLE1.KEY1
WHERE TABLE1.TERM_DT='8888-12-31'
QUALIFY RNK=1
GROUP BY
1,2,3,4
how are you?
I never used qualify row_number()/rank() and I have some questions.
I am trying this query in teradata:
select sit_site_id
, count(distinct shp_shipment_id)
from WHOWNER.BT_SHP_SHIPMENTS
group by sit_site_id
QUALIFY RANK() OVER (PARTITION BY sit_site_id
ORDER BY count(distinct shp_shipment_id) ) = 3
But the result is: 'No data available in table'.
I want to get the first 3 sit_site_id values with more shp_shipment_id.
Where is my error?
Thanks!
I want to get the first 3 sit_site_id values with more shp_shipment_id
If so, then you do not need to use RANK() / ROW_NUMBER(). Instead, you can just use TOP. Your query should be like below:
select TOP 3 sit_site_id
, count(distinct shp_shipment_id) shp_shipment_id_cnt
from WHOWNER.BT_SHP_SHIPMENTS
group by sit_site_id
order by shp_shipment_id_cnt DESC;
If you want to use qualify for this, then you can do:
select sit_site_id, count(distinct shp_shipment_id)
from WHOWNER.BT_SHP_SHIPMENTS
group by sit_site_id
qualify row_number() over (order by count(distinct shp_shipment_id) desc) <= 3;
However, I would recommend top as in #Clark's answer.
I have tables - FAKE_CUST, PRE_CUST, NORMAL_CUST. Based on the constraint present in FAKE_CUST table, the value has to be brought to either PRE_CUST or NORMAL_CUST.
I am using the follwing code:
INSERT ALL
INTO PRE_CUST(CUST_ID, TOTAL_COST_TRANS) (SELECT DISTINCT CUST_ID,C FROM (
SELECT CUST_ID, SUM(COST_TRANS) OVER (PARTITION BY CUST_ID) as C FROM FAKE_CUST) WHERE C>1000)
INTO NORMAL_CUST(CUST_ID, TOTAL_COST_TRANS) (SELECT DISTINCT CUST_ID,C FROM (
SELECT CUST_ID, SUM(COST_TRANS) OVER (PARTITION BY CUST_ID) as C FROM FAKE_CUST) WHERE C<1000)
SELECT 1 FROM DUAL;
The syntax which you have used for multi-table insert is not correct to start with.
The WHEN clause, which evaluates the condition for insert, has to come before insert_intoclause.
Your query should look like this:
INSERT ALL
WHEN c > 1000 THEN
INTO pre_cust (cust_id, total_cost_trans)
WHEN c < 1000 THEN
INTO normal_cust (cust_id, total_cost_trans)
SELECT CUST_ID, SUM(COST_TRANS) OVER (PARTITION BY CUST_ID) as C
FROM FAKE_CUST
I have a table which stores the check-in times of employees through Time Machine on the basis of a username. If an employee punches multiple times then there would be multiple records of his check-ins which would only have a time difference of few seconds in between. Obviously only the first record is valid. All the other entries are invalid and must be deleted from the Table. How can i do it if i can select all the checkin records of an employee for the current date?
The Data in the db is as follows.
Username Checktime CheckType
HRA001 7/29/2012 8:16:44 AM Check-In
HRA001 7/29/2012 8:16:46 AM Check-In
HRA001 7/29/2012 8:16:50 AM Check-In
HRA001 7/29/2012 8:16:53 AM Check-In
Try this:
;WITH users_CTE as (
select rank() over (partition by Username order by Checktime) as rnk from users
)
DELETE FROM users_CTE where rnk <> 1
--For your second requirement try this query
;WITH users_CTE as (
select *,rank() over (partition by Username order by Checktime) as rnk from users
)
,CTE2 as (select Username,MIN(CheckTime) as minTime,DATEADD(mi,1,MIN(CheckTime)) as maxTime from users_CTE
group by Username)
delete from users where Checktime in(
select c1.Checktime from users_CTE c1 left join CTE2 c2
on c1.Checktime > c2.minTime and c1.Checktime <= c2.maxTime
where c2.Username is not null and c1.Username in(
select c1.Username from users_CTE c1 left join CTE2 c2
on c1.Checktime > c2.minTime and c1.Checktime <= c2.maxTime
group by c1.Username,c2.Username
having COUNT(*) > 1))
--For your changed requirements pls check this query below
alter table users add flag varchar(2)
;WITH users_CTE as (
select *,rank() over (partition by Username order by Checktime) as rnk from users
)
,CTE2 as (select Username,MIN(CheckTime) as minTime,DATEADD(mi,1,MIN(CheckTime)) as maxTime from users_CTE
group by Username)
update u SET u.flag = 'd' from users_CTE u inner join (
select c1.Checktime from users_CTE c1 left join CTE2 c2
on c1.Checktime > c2.minTime and c1.Checktime <= c2.maxTime
where c2.Username is not null and c1.Username in(
select c1.Username from users_CTE c1 left join CTE2 c2
on c1.Checktime > c2.minTime and c1.Checktime <= c2.maxTime
group by c1.Username,c2.Username
having COUNT(*) > 1)) a
on u.Checktime=a.Checktime
--Check the latest query with DeletFlag
;WITH users_CTE as
(
select *,row_number() over (partition by Username order by Checktime) as row from users
)
,CTE as(
select row,Username,Checktime,CheckType,0 as totalSeconds,'N' as Delflag from users_CTE where row=1
union all
select t.row,t.Username,t.Checktime,t.CheckType,CASE WHEN (c.totalSeconds + DATEDIFF(SECOND,c.Checktime,t.Checktime)) >= 60 then 0 else (c.totalSeconds + DATEDIFF(SECOND,c.Checktime,t.Checktime)) end as totalSeconds,
CASE WHEN (c.totalSeconds + DATEDIFF(SECOND,c.Checktime,t.Checktime)) >= 60 then 'N' else 'Y' end as Delflag
--CASE WHEN c.totalSeconds <= 60 then 'Y' else 'N' end as Delflag
from users_CTE t inner join CTE c
on t.row=c.row+1
)
select Username,Checktime,CheckType,Delflag from CTE
Why don't you verify the check-ins before inserting them into db. If there exists any check-in for this user, between this date and that date then do nothing else insert it
You should be able to order all records by time, subtract the latest time from the previous time per employee and, if the result is less than a certain threshold, delete the row(s) with the most recent time.
You could try and RANK the records by checkin time and then delete all the records for each employee for each day which have RANK greater than 1.
Try this query: Delete from employee where employee.checkin in (select checkin from employee where count(checkin)>1);
http://codesimplified.com/2010/10/18/remove-duplicate-records-from-the-database-table/
Hope this will helps you.
DELETE FROM timesheet
WHERE timesheetRecordId <>(
SELECT TOP 1 timesheetRecordId from timesheet
WHERE checkInDate=todaysDate AND employeeId=empId ORDER BY checkInTime ASC
)
AND checkInDate=today's date AND empolyeeId=empId;
I don't think you can specify a Target Table, from a Delete statement, in a Subquery of that same statement. So you can't do it with one single Delete statement.
What you can do is write a stored procedure. In your Stored Procedure you should create a Temporary Table containing the PKs returned by this Query:
select cht.pkey
from CheckTimeTable as cht
where exists ( select pkey
from CheckTimeTable
where username = cht.userName
and checkType = 'check-IN'
and Checktime >= subtime(cht.Checktime, '0 0:0:15.000000')
and Checktime < cht.Checktime);
Then write another statement to delete those PKs from your original table, CheckTimeTable.
Note that the query above is for MySQL, so you'll need to find the way to subtract 15 seconds from a timestamp for your DBMS. In MySQL it's done like this:
subtime(cht.Checktime, '0 0:0:15.000000')
This query will return whichever CheckTime record that has another CheckTime record from the same user, with the type Check-In, and within 15 seconds earlier than its own checktime.