Teradata gets division wrong? - teradata

Question

I have a coworker that doesn't want to include null rows in a percentile rank. The default Teradata function seems to just treat null as the lowest number in the set, so I decided to do the math manually. I started using the below query to test out my equation
drop table tmp;
create multiset volatile table tmp (
num byteint
) primary index (num)
on commit preserve rows
;
insert into tmp
values (1)
;insert into tmp
values (2)
;insert into tmp
values (1)
;insert into tmp
values (4)
;insert into tmp
values (null)
;insert into tmp
values (4)
;insert into tmp
values (null)
;insert into tmp
values (2)
;insert into tmp
values (9)
;insert into tmp
values (null)
;insert into tmp
values (10)
;insert into tmp
values (10)
;insert into tmp
values (11)
;
select
num,
case
when num is null then 0
else cast(dense_rank() over (partition by case when num is not null then 1 else 2 end order by num) as number)
end as str_rnk,
q.nn,
str_rnk/q.nn as pct_rnk
from tmp
cross join (
select cast(count(num) as number) as nn from tmp
) q
order by num
;
So what I expect to see in result set is this:
num str_rnk nn pct_rnk
null 0 10 0
null 0 10 0
null 0 10 0
1 1 10 0.1
1 1 10 0.1
2 2 10 0.2
2 2 10 0.2
4 3 10 0.3
4 3 10 0.3
9 4 10 0.4
10 5 10 0.5
10 5 10 0.5
But I'm getting a result that looks like it did a regular rank instead of a dense_rank, like this:
num str_rnk nn pct_rnk
null 0 10 0
null 0 10 0
null 0 10 0
1 1 10 0.1
1 1 10 0.1
2 2 10 0.3
2 2 10 0.3
4 3 10 0.5
4 3 10 0.5
9 4 10 0.7
10 5 10 0.8
10 5 10 0.8
I know I could set the rank in a subquery and It would calculate the way I expect it to, but why isn't it doing it the way I have it now?

Answer 1

While this doesn't answer your question. It's not the division that's an issue, it seems to be some oddball issue running that CAST and Dense_Rank twice in the same SELECT.
Consider:
select
num,
case
when num is null then 0
else cast(dense_rank() over (partition by case when num is not null then 1 else 2 end order by num) as number)
end as str_rnk,
case
when num is null then 0
else cast(dense_rank() over (partition by case when num is not null then 1 else 2 end order by num) as number)
end as str_rnk2
from tmp
cross join (
select cast(count(num) as number) as nn from tmp
) q;
+--------+---------+----------+
| num | str_rnk | str_rnk2 |
+--------+---------+----------+
| 1 | 1 | 1 |
| 1 | 1 | 1 |
| 2 | 2 | 3 |
| 2 | 2 | 3 |
| 4 | 3 | 5 |
| 4 | 3 | 5 |
| 9 | 4 | 7 |
| 10 | 5 | 8 |
| 10 | 5 | 8 |
| 11 | 6 | 10 |
| <null> | 0 | 0 |
| <null> | 0 | 0 |
| <null> | 0 | 0 |
+--------+---------+----------+
Since the CAST isn't necessary here:
select
num,
case
when num is null then 0
else dense_rank() over (partition by case when num is not null then 1 else 2 END order by num)
end as str_rnk,
case
when num is null then 0
else dense_rank() over (partition by case when num is not null then 1 else 2 END order by num)
end as str_rnk2
from tmp
cross join (
select cast(count(num) as number) as nn from tmp
) q;
+--------+---------+----------+
| num | str_rnk | str_rnk2 |
+--------+---------+----------+
| 1 | 1 | 1 |
| 1 | 1 | 1 |
| 2 | 2 | 2 |
| 2 | 2 | 2 |
| 4 | 3 | 3 |
| 4 | 3 | 3 |
| 9 | 4 | 4 |
| 10 | 5 | 5 |
| 10 | 5 | 5 |
| 11 | 6 | 6 |
| <null> | 0 | 0 |
| <null> | 0 | 0 |
| <null> | 0 | 0 |
+--------+---------+----------+
Your query, with a quick rewrite:
select
num,
case
when num is null then 0
else dense_rank() over (partition by num * 0 order by num)
end as str_rnk,
str_rnk * 1.0/COUNT(*) OVER (PARTITION BY num * 0) as pct_rnk
from tmp
order by num
;
+--------+---------+---------+
| num | str_rnk | pct_rnk |
+--------+---------+---------+
| <null> | 0 | 0.0 |
| <null> | 0 | 0.0 |
| <null> | 0 | 0.0 |
| 1 | 1 | 0.1 |
| 1 | 1 | 0.1 |
| 2 | 2 | 0.2 |
| 2 | 2 | 0.2 |
| 4 | 3 | 0.3 |
| 4 | 3 | 0.3 |
| 9 | 4 | 0.4 |
| 10 | 5 | 0.5 |
| 10 | 5 | 0.5 |
| 11 | 6 | 0.6 |
+--------+---------+---------+
Or if you want to get the CASE statement out of there completely:
select
num,
dense_rank() over (partition by num * 0 order by num) * (num * 0 + 1.0) as str_rnk,
str_rnk/COUNT(*) OVER (PARTITION BY num * 0) as pct_rnk
from tmp
order by num;

Answer 2

As JNevill noted this is a bug, you should open an incident with Teradata support:
SELECT
num,
-- cast to FLOAT or DECIMAL works as expected
Cast(Dense_Rank() Over (ORDER BY num) AS NUMBER) AS a,
a AS b
FROM tmp
num a b
---- --- ---
? 1 1
? 1 1
? 1 1
1 2 4
1 2 4
2 3 6
2 3 6
4 4 8
4 4 8
9 5 10
10 6 11
10 6 11
11 7 13
But adding QUALIFY a<>b returns an empty result :-)
The original calculation for PERCENT_RANK is based on
Cast(Rank() Over (ORDER BY num) -1 AS DEC(18,6)) / Count(*) Over ()
If you want to exclude NULLs you can switch to Count(num) and NULLS LAST:
SELECT
num,
CASE
WHEN num IS NOT NULL
THEN Cast(Dense_Rank() Over (ORDER BY num NULLS LAST) AS DECIMAL(18,6))
ELSE 0
END AS str_rnk,
str_rnk / Count(num) Over ()
FROM tmp
Or using that slick num * 0 trick:
SELECT
num,
Coalesce(Dense_Rank()
Over (ORDER BY num NULLS LAST)
* (num * 0 +1.000000), 0) AS str_rnk,
str_rnk / Count(num) Over ()
FROM tmp