I have a SQL query in teradata that returns a results set of ~160m rows in (I guess) a reasonable time: dependent on how good a day the server is having it runs between 10-60 minutes.
I recently got access to space to save it as a table, however using my initial query and the "insert into " command I get error 2646-no more spool.
query structure is
insert into <test_DB.tablename>
with smaller_dataset as
(
select
*
from
(
select
items
,case items
from
<Database.table>
QUALIFY ROW_NUMBER() OVER (PARTITION BY A,B ORDER BY C desc , LAST_UPDATE_DTM DESC) = 1
where 1=1
and other things
) T --irrelevant alias for subquery
QUALIFY ROW_NUMBER() OVER (PARTITION BY A, B ORDER BY C desc) = 1)
, employee_table as
(
select
items
,max(J1.field1) J1_field1
,max(J2.field1) J2_field1
,max(J3.field1) J3_field1
,max(J4.field1) J4_field1
from smaller_dataset S
self joins J1,J2,J3,J4
group by
non-aggregate items
)
select
items
case items
from employee_table
;
How can I break up the return into smaller chunks to prevent this error?
Related
I'm trying to update a table with to many rows 388.000.
This is the query:
update DL_RG_ANALYTICS.SH_historico
from
(
SELECT
CAST((MAX_DIA - DIA_PAGO) AS INTEGER) AS DIAS_AL_CIERRE_1
FROM
(SELECT * FROM DL_RG_ANALYTICS.SH_historico A
LEFT JOIN
(SELECT ANO||MES AS ANO_MES, MAX(DIA) AS MAX_DIA FROM DL_RG_ANALYTICS.SH_CALENDARIO
GROUP BY 1) B
ON A.ANOMES = B.ANO_MES
) M) N
SET DIAS_AL_CIERRE = DIAS_AL_CIERRE_1;
Any help is apreciate.
This first thing I'd do is replace the SELECT * with only the columns you need. You can also remove the M derived table to make it easier to read:
UPDATE DL_RG_ANALYTICS.SH_historico
FROM (
SELECT CAST((MAX_DIA - DIA_PAGO) AS INTEGER) AS DIAS_AL_CIERRE_1
FROM DL_RG_ANALYTICS.SH_historico A
LEFT JOIN (
SELECT ANO || MES AS ANO_MES, MAX(DIA) AS MAX_DIA
FROM DL_RG_ANALYTICS.SH_CALENDARIO
GROUP BY 1
) B ON A.ANOMES = B.ANO_MES
) N
SET DIAS_AL_CIERRE = DIAS_AL_CIERRE_1;
What indexes are defined on the SH_CALENDARIO table? If there is a composite index of (ANO, MES) then you should re-write your LEFT JOIN sub-query to GROUP BY these two columns since you concatenate them together anyways. In general, you want to perform joins, GROUP BY and OLAP functions on indexes, so there will be less row re-distribution and they will run more efficiently.
Also, this query is updating all rows in the table with the same value. Is this intended, or do you want to include extra columns in your WHERE clause?
My table: CREATE TABLE T(id INT PRIMARY KEY, value INT UNIQUE)
This query, as I considered, would produce a median value of value in the table. But sqlite v.3.9.1 gives me the error no such column: ot.value to the line with group by. And it process line with where successfully, although it uses a similar expression. What's the problem of the query?
select
ot.id,
ot.value
from T as ot
where (
select count(c) > count(DISTINCT c) from (
select count(*) c from T as it
where it.value != ot.value
group by it.value < ot.value
) LessMore
)
The same query succeeds in PostgreSQL and prints what was expected. MySQL gives the error Error Code: 1054. Unknown column 'ot.value' in 'where clause'
I have a oracle query
select id from (
select ID, ROW_NUMBER() over (partition by LATEST_RECEIPT order by ID) rownumber
from Table
where LATEST_RECEIPT in
(
select LATEST_RECEIPT from Table
group by LATEST_RECEIPT
having COUNT(1) > 1
)
) t
where rownumber <> 1;
The data type of LATEST_RECEIPT was earlier varchar2(4000) and this query worked fine. Since the length of the column needs to be extended i modified it to CLOB, after which this fails. Could anyone help me fix this issue or provide a work around?
You can change your inner query to look for other rows with the same last_receipt value but a different ID (assuming ID is unique); if another row exists then that is equivalent to your count returning greater than one. But you can't simply test two CLOB values for equality, you need to use dbms_lob.compare:
select ID
from your_table t1
where exists (
select null from your_table t2
where dbms_lob.compare(t2.LATEST_RECEIPT, t1.LATEST_RECEIPT) = 0
and t2.ID != t1.ID
-- or if ID isn't unique: and t2.ROWID != t1.ROWID
);
Applying the row number filter is tricker, as you also can't use a CLOB in the analytic partition by clause. As André Schild suggested, you can use a hash; here passing the integer value 3, which is the equivalent of dbms_crypto.hash_sh1 (though in theory that could change in a future release!):
select id from (
select ID, ROW_NUMBER() over (partition by dbms_crypto.hash(LATEST_RECEIPT, 3)
order by ID) rownumber
from your_table t1
where exists (
select null from your_table t2
where dbms_lob.compare(t2.LATEST_RECEIPT, t1.LATEST_RECEIPT) = 0
and t2.ID != t1.ID
-- or if ID isn't unique: and t2.ROWID != t1.ROWID
)
)
where rownumber > 1;
It is of course possible to get a hash collision, and if that happened - you had two latest_receipt values which both appeared more than once and both hashed to the same value - then you could get too many rows back. That seems pretty unlikely, but it's something to consider.
So rather than ordering you can only look for rows which have the same lastest_receipt and a lower ID:
select ID
from your_table t1
where exists (
select null from your_table t2
where dbms_lob.compare(t2.LATEST_RECEIPT, t1.LATEST_RECEIPT) = 0
and t2.ID < t1.ID
);
Again that assumes ID is unique. If it isn't then you could still use rowid instead, but you would have less control over which rows were found - the lowest rowid isn't necessarily the lowest ID. Presumably you're using this to dine rows to delete. If you actually don't mind which row you keep and which you delete then you could still do:
and t2.ROWID < t1.ROWID
But since you are currently ordering that probably isn't acceptable, and hashing might be preferable, despite the small risk.
I have 3 Data Table Claim, Part and Labor.
In this Claim is parent table and Part and Labor is mapping tables of Claim and they have Part and Labor has the ClaimId as a Foreign Key.
Claim has data like:
Part has data Like
Labor table has data Like
Target Output would be:
Can anyone help me to achieve this in SQL server.
I have tried to solve with the Union/CTE but it did not gives the result as I want.
I got the same output (for your updated output screen) for this specific case. I don't know if any other data will work for you.
SELECT TMP.ClaimId
, CASE WHEN TMP.RowNum = 1 THEN TMP.Name ELSE NULL END AS ClaimName
, CASE WHEN TMP.RowNum = 1 THEN TMP.Note ELSE NULL END AS Note
, TMP.PartId
, TMP.PartNumber
, TMP.PartCost
, JOIN_L.LaborId
, JOIN_L.LaborCost
FROM (
SELECT C.ClaimId, C.Name, C.Note, P.PartId, P.PartNumber, P.PartCost
, ROW_NUMBER() OVER(PARTITION BY C.ClaimId ORDER BY P.PartId) AS RowNum
FROM Claim AS C
LEFT JOIN Part AS P ON C.ClaimId = P.ClaimId
)AS TMP
LEFT JOIN (
SELECT *
, ROW_NUMBER() OVER(PARTITION BY L.ClaimId ORDER BY L.ClaimId) AS RowNum
FROM Labor AS L
) AS JOIN_L ON (TMP.ClaimId = JOIN_L.ClaimId AND TMP.RowNum = JOIN_L.RowNum)
ORDER BY TMP.ClaimId
Not sure why you tried CTE here
Select C.ClaimId,C.name,C.Note,P.PartId,P.PartNumber,P.PartCost,L.LabourId,L.LabourCost
From Claim C
Left Outer Join Part P On P.ClaimId = C.ClaimId
Left Outer Join Labor L On L.ClaimId=C.ClaimId
I have tables - FAKE_CUST, PRE_CUST, NORMAL_CUST. Based on the constraint present in FAKE_CUST table, the value has to be brought to either PRE_CUST or NORMAL_CUST.
I am using the follwing code:
INSERT ALL
INTO PRE_CUST(CUST_ID, TOTAL_COST_TRANS) (SELECT DISTINCT CUST_ID,C FROM (
SELECT CUST_ID, SUM(COST_TRANS) OVER (PARTITION BY CUST_ID) as C FROM FAKE_CUST) WHERE C>1000)
INTO NORMAL_CUST(CUST_ID, TOTAL_COST_TRANS) (SELECT DISTINCT CUST_ID,C FROM (
SELECT CUST_ID, SUM(COST_TRANS) OVER (PARTITION BY CUST_ID) as C FROM FAKE_CUST) WHERE C<1000)
SELECT 1 FROM DUAL;
The syntax which you have used for multi-table insert is not correct to start with.
The WHEN clause, which evaluates the condition for insert, has to come before insert_intoclause.
Your query should look like this:
INSERT ALL
WHEN c > 1000 THEN
INTO pre_cust (cust_id, total_cost_trans)
WHEN c < 1000 THEN
INTO normal_cust (cust_id, total_cost_trans)
SELECT CUST_ID, SUM(COST_TRANS) OVER (PARTITION BY CUST_ID) as C
FROM FAKE_CUST