OLAP level-based measure drilldown issue - olap

everyone!
Following the "Pentaho Analizer Cookbook" PDF, I tried to stablish a level-based measure.
My cube has a single dimension hierarchy: Clients that have Projects that have Buildings that have Phases that have Costlines level 0 that have Costlines level 1.
So, there let be samples for both fact and dimensions:
DROP TABLE IF EXISTS TMP_DEBUG_OLAP_13_FACTS;
CREATE TABLE TMP_DEBUG_OLAP_13_FACTS AS
SELECT 1 AS client_id,1 as project_id,1 as building_id,'1_1_1_1' as phase_id,'1_1_1_1_1' as costs_line_level_0_id,'1_1_1_1_1_1' as costs_line_level_1_id,10 as amount
UNION ALL SELECT 1 AS client_id,1 as project_id,1 as building_id,'1_1_1_1' as phase_id,'1_1_1_1_2' as costs_line_level_0_id,'1_1_1_1_2_1' as costs_line_level_1_id,20 as amount
UNION ALL SELECT 1 AS client_id,1 as project_id,1 as building_id,'1_1_1_2' as phase_id,'1_1_1_2_1' as costs_line_level_0_id,'1_1_1_2_1_1' as costs_line_level_1_id,30 as amount
UNION ALL SELECT 1 AS client_id,1 as project_id,2 as building_id,'1_1_2_1' AS phase_id,'1_1_2_1_1' as costs_line_level_0_id,'1_1_2_1_1_1' as costs_line_level_1_id,40 as amount
UNION ALL SELECT 1 AS client_id,2 as project_id,3 as building_id,'1_2_3_1' AS phase_id,'1_2_3_1_1' as costs_line_level_0_id,'1_2_3_1_1_1' as costs_line_level_1_id,50 as amount
UNION ALL SELECT 1 AS client_id,2 as project_id,4 as building_id,'1_2_4_1' AS phase_id,'1_2_4_1_1' as costs_line_level_0_id,'1_2_4_1_1_1' as costs_line_level_1_id,60 as amount
UNION ALL SELECT 2 AS client_id,3 as project_id,5 as building_id,'2_3_5_1' AS phase_id,'2_3_5_1_1' as costs_line_level_0_id,'2_3_5_1_1_-1' as costs_line_level_1_id,70 as amount
;
DROP TABLE IF EXISTS TMP_DEBUG_OLAP_13_DIMENSIONS;
CREATE TABLE TMP_DEBUG_OLAP_13_DIMENSIONS AS
SELECT 1 AS client_id,'Client 1 name' AS client_name,1 as project_id,'Project 1' as project_name,1 as building_id,'Building 1' as building_name,'1_1_1_1' as phase_id,'Phase 1' as phase_name,'1_1_1_1_1' as costs_line_level_0_id,'1 blah' AS costs_line_level_0_name,'1_1_1_1_1_1' AS costs_line_level_1_id,'1.1 blah' as costs_line_level_1_name
UNION ALL SELECT 1 AS client_id,'Client 1 name' AS client_name,1 as project_id,'Project 1' as project_name,1 as building_id,'Building 1' as building_name,'1_1_1_1' as phase_id,'Phase 1' as phase_name,'1_1_1_1_2' as costs_line_level_0_id,'2 blah' AS costs_line_level_0_name,'1_1_1_1_2_1' AS costs_line_level_1_id,'2.1 blah' as costs_line_level_1_name
UNION ALL SELECT 1 AS client_id,'Client 1 name' AS client_name,1 as project_id,'Project 1' as project_name,1 as building_id,'Building 1' as building_name,'1_1_1_2' as phase_id,'Phase 2' as phase_name,'1_1_1_2_1' as costs_line_level_0_id,'1 blah' AS costs_line_level_0_name,'1_1_1_2_1_1' AS costs_line_level_1_id,'1.1 blah' as costs_line_level_1_name
UNION ALL SELECT 1 AS client_id,'Client 1 name' AS client_name,1 as project_id,'Project 1' as project_name,2 as building_id,'Building 2' as building_name,'1_1_2_1' as phase_id,'Phase 1' as phase_name,'1_1_2_1_1' as costs_line_level_0_id,'1 blah' AS costs_line_level_0_name,'1_1_2_1_1_1' AS costs_line_level_1_id,'1.1 blah' as costs_line_level_1_name
UNION ALL SELECT 1 AS client_id,'Client 1 name' AS client_name,2 as project_id,'Project 2' as project_name,3 as building_id,'Building 3' as building_name,'1_2_3_1' as phase_id,'Phase 1' as phase_name,'1_2_3_1_1' as costs_line_level_0_id,'1 blah' AS costs_line_level_0_name,'1_2_3_1_1_1' AS costs_line_level_1_id,'1.1 blah' as costs_line_level_1_name
UNION ALL SELECT 1 AS client_id,'Client 1 name' AS client_name,2 as project_id,'Project 2' as project_name,4 as building_id,'Building 4' as building_name,'1_2_4_1' as phase_id,'Phase 1' as phase_name,'1_2_4_1_1' as costs_line_level_0_id,'1 blah' AS costs_line_level_0_name,'1_2_4_1_1_1' AS costs_line_level_1_id,'1.1 blah' as costs_line_level_1_name
UNION ALL SELECT 2 AS client_id,'Client 2 name' AS client_name,3 as project_id,'Project 3' as project_name,5 as building_id,'Building 5' as building_name,'2_3_5_1' as phase_id,'Phase 1' as phase_name,'2_3_5_1_1' as costs_line_level_0_id,'1 blah' AS costs_line_level_0_name,'2_3_5_1_1_-1' AS costs_line_level_1_id,'1.1 blah' as costs_line_level_1_name
--non-crosing dimentions
UNION ALL SELECT 2 AS client_id,'Client 2 name' AS client_name,3 as project_id,'Project 3' as project_name,5 as building_id,'Building 5' as building_name,'2_3_5_1' as phase_id,'Phase 1' as phase_name,'2_3_5_1_2' as costs_line_level_0_id,'2 blah' AS costs_line_level_0_name,'2_3_5_1_2_1' AS costs_line_level_1_id,'2.1 blah' as costs_line_level_1_name
UNION ALL SELECT 2 AS client_id,'Client 2 name' AS client_name,3 as project_id,'Project 3' as project_name,5 as building_id,'Building 5' as building_name,'2_3_5_1' as phase_id,'Phase 1' as phase_name,'2_3_5_1_3' as costs_line_level_0_id,'3 blah' AS costs_line_level_0_name,'2_3_5_1_3_1' AS costs_line_level_1_id,'3.1 blah' as costs_line_level_1_name
UNION ALL SELECT 2 AS client_id,'Client 2 name' AS client_name,3 as project_id,'Project 3' as project_name,5 as building_id,'Building 5' as building_name,'2_3_5_1' as phase_id,'Phase 1' as phase_name,'2_3_5_1_4' as costs_line_level_0_id,'4 blah' AS costs_line_level_0_name,NULL AS costs_line_level_1_id,'4.1 blah' as costs_line_level_1_name
;
And my calculated measure is such as
([Measures].[My measure] , Ancestor([Dimensions].CurrentMember , [Dimensions].[Building]) )
The whole cube's xml (Mondrian) is as follows:
<Schema name="level_based_measure_unit_test">
<Dimension type="StandardDimension" visible="true" name="Dimensions">
<Hierarchy name="Dimensions hierarchy" visible="true" hasAll="true" primaryKey="costs_line_level_1_id">
<Table name="tmp_debug_olap_13_dimensions" schema="public" alias="">
</Table>
<Level name="Clients" visible="true" column="client_id" nameColumn="client_name" uniqueMembers="false">
</Level>
<Level name="Project" visible="true" column="project_id" nameColumn="project_name" uniqueMembers="false">
</Level>
<Level name="Building" visible="true" column="building_id" nameColumn="building_name" uniqueMembers="false">
</Level>
<Level name="Phase" visible="true" column="phase_id" nameColumn="phase_name" uniqueMembers="false">
</Level>
<Level name="Cost lines level 0" visible="true" column="costs_line_level_0_id" nameColumn="costs_line_level_0_name" uniqueMembers="false">
</Level>
<Level name="Cost lines level 1" visible="true" column="costs_line_level_1_id" nameColumn="costs_line_level_1_name" uniqueMembers="false">
</Level>
</Hierarchy>
</Dimension>
<Cube name="My cube" visible="true" cache="true" enabled="true">
<Table name="tmp_debug_olap_13_facts" schema="public" alias="">
</Table>
<DimensionUsage source="Dimensions" name="Dimensions" visible="true" foreignKey="costs_line_level_1_id">
</DimensionUsage>
<Measure name="My measure" column="amount" aggregator="sum" visible="true">
</Measure>
<CalculatedMember name="My leveled measure" formatString="" formula="([Measures].[My measure] , Ancestor([Dimensions].CurrentMember , [Dimensions].[Building]) )" dimension="Measures" visible="true">
</CalculatedMember>
</Cube>
</Schema>
I have 1 problem with that formula:
It displays a value at building level, but drillind-down, it keeps its value.
Picture of what I mean:
Does anyone know how should I change my calculated member, so it displays the measure [My measure] but only on Building level and below?
Thank you so much!

As the Pentaho Analizer Cookbook specifies, that Formula is for pinning a Measure to a certain level.
It is: if you drill-down, you'll get the value as the drill-up.
If what you need (if I understood correctly) is to just display nothing for upper-levels, then user the next Formula:
IIF([Dimensions].CurrentMember.Level.Ordinal < 3 , NULL , [Measures].[My measure] )

Related

MariaDB JSON_ARRAYAGG gives wrong result

I have 2 problems in MariaDB 15.1 when using JSON_ARRAYAGG
The brackets [] are omitted
Incorrect wrong result, values are duplicates or omitted
My database is the following:
user:
+----+------+
| id | name |
+----+------+
| 1 | Jhon |
| 2 | Bob |
+----+------+
car:
+----+---------+-------------+
| id | user_id | model |
+----+---------+-------------+
| 1 | 1 | Tesla |
| 2 | 1 | Ferrari |
| 3 | 2 | Lamborghini |
+----+---------+-------------+
phone:
+----+---------+----------+--------+
| id | user_id | company | number |
+----+---------+----------+--------+
| 1 | 1 | Verzion | 1 |
| 2 | 1 | AT&T | 2 |
| 3 | 1 | T-Mobile | 3 |
| 4 | 2 | Sprint | 4 |
| 5 | 1 | Sprint | 2 |
+----+---------+----------+--------+
1. The brackets [] are omitted
For example this query that gets users with their list of cars:
SELECT
user.id AS id,
user.name AS name,
JSON_ARRAYAGG(
JSON_OBJECT(
'id', car.id,
'model', car.model
)
) AS cars
FROM user
INNER JOIN car ON user.id = car.user_id
GROUP BY user.id;
Result: brackets [] were omitted in cars (JSON_ARRAYAGG has the behavior similar to GROUP_CONCAT)
+----+------+-----------------------------------------------------------+
| id | name | cars |
+----+------+-----------------------------------------------------------+
| 1 | Jhon | {"id": 1, "model": "Tesla"},{"id": 2, "model": "Ferrari"} |
| 2 | Bob | {"id": 3, "model": "Lamborghini"} |
+----+------+-----------------------------------------------------------+
However when adding the filter WHERE user.id = 1, the brackets [] are not omitted:
+----+------+-------------------------------------------------------------+
| id | name | cars |
+----+------+-------------------------------------------------------------+
| 1 | Jhon | [{"id": 1, "model": "Tesla"},{"id": 2, "model": "Ferrari"}] |
+----+------+-------------------------------------------------------------+
2. Incorrect wrong result, values are duplicates or omitted
This error is strange as the following conditions must be met:
Consult more than 2 tables
The DISTINCT option must be used
A user has at least 2 cars and at least 3 phones.
Duplicate values
for example, this query that gets users with their car list and their phone list:
SELECT
user.id AS id,
user.name AS name,
JSON_ARRAYAGG( DISTINCT
JSON_OBJECT(
'id', car.id,
'model', car.model
)
) AS cars,
JSON_ARRAYAGG( DISTINCT
JSON_OBJECT(
'id', phone.id,
'company', phone.company,
'number', phone.number
)
) AS phones
FROM user
INNER JOIN car ON user.id = car.user_id
INNER JOIN phone ON user.id = phone.user_id
GROUP BY user.id;
I will leave the output in json format and I will only leave the elements that interest.
Result: brackets [] were omitted and duplicate Verizon
{
"id": 1,
"name": "Jhon",
"phones": // [ Opening bracket expected
{
"id": 5,
"company": "Sprint",
"number": 2
},
{
"id": 1,
"company": "Verzion",
"number": 1
},
{
"id": 1,
"company": "Verzion",
"number": 1
}, // Duplicate object with the DISTINCT option
{
"id": 2,
"company": "AT&T",
"number": 2
},
{
"id": 3,
"company": "T-Mobile",
"number": 3
}
// ] Closing bracket expected
}
Omitted values
This error occurs when omit phone.id is omitted in the query
SELECT
user.id AS id,
user.name AS name,
JSON_ARRAYAGG( DISTINCT
JSON_OBJECT(
'id', car.id,
'model', car.model
)
) AS cars,
JSON_ARRAYAGG( DISTINCT
JSON_OBJECT(
--'id', phone.id,
'company', phone.company,
'number', phone.number
)
) AS phones
FROM user
INNER JOIN car ON user.id = car.user_id
INNER JOIN phone ON user.id = phone.user_id
GROUP BY user.id;
Result: brackets [] were omitted and Sprint was omitted.
Apparently this happens because it makes an OR type between the columns of the JSON_OBJECT, since the company exists in a different row and number in a other different row
{
"id": 1,
"name": "Jhon",
"phones": // [ Opening bracket expected
//{
// "company": "Sprint",
// "number": 2
//}, `Sprint` was omitted
{
"company": "Verzion",
"number": 1
},
{
"company": "AT&T",
"number": 2
},
{
"company": "T-Mobile",
"number": 3
}
// ] Closing bracket expected
}
GROUP_CONCAT instance of JSON_ARRAYAGG solves the problem of duplicate or omitted objects
However, by adding the filter WHERE user.id = 1, the brackets [] are not omitted and also the problem of duplicate or omitted objects is also solved:
{
"id": 1,
"name": "Jhon",
"phones": [
{
"id": 1,
"company": "Verzion",
"number": 1
},
{
"id": 2,
"company": "AT&T",
"number": 2
},
{
"id": 3,
"company": "T-Mobile",
"number": 3
},
{
"id": 5,
"company": "Sprint",
"number": 2
}
]
}
What am I doing wrong?
So far my solution is this, but I would like to use JSON_ARRAYAGG since the query is cleaner
-- 1
SELECT
user.id AS id,
user.name AS name,
CONCAT(
'[',
GROUP_CONCAT( DISTINCT
JSON_OBJECT(
'id', car.id,
'model', car.model
)
),
']'
) AS cars
FROM user
INNER JOIN car ON user.id = car.user_id
GROUP BY user.id;
-- 2
SELECT
user.id AS id,
user.name AS name,
CONCAT(
'[',
GROUP_CONCAT( DISTINCT
JSON_OBJECT(
'id', car.id,
'model', car.model
)
),
']'
) AS cars,
CONCAT(
'[',
GROUP_CONCAT( DISTINCT
JSON_OBJECT(
'id', phone.id,
'company', phone.company,
'number', phone.number
)
),
']'
) AS phones
FROM user
INNER JOIN car ON user.id = car.user_id
INNER JOIN phone ON user.id = phone.user_id
GROUP BY user.id;

Kusto query for getting cummulative count up to a given date [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I have a database with a set of events with a user id and timestamp, and I am trying to write a query that will give me the count of distinct users that have triggered an event up to each day. So if we have the following data:
Event | UID | Time Stamp
event 1 | 0 | 9/25/19 9:00 AM
event 2 | 1 | 9/25/19 3:00 PM
event 3 | 2 | 9/26/19 2:00 PM
event 4 | 1 | 9/28/19 5:00 PM
event 5 | 3 | 9/29/19 7:00 AM
Then the output should be:
9/25/19 : 2
9/26/19 : 3
9/27/19 : 3 (since there are no new events on the 27th)
9/28/19 : 3 (since user with UID=1 has already been counted)
9/29/19 : 4
I have a query which will get the number of events per day, but not the number of events of all days leading up to that day. Any help would be greatly appreciated!
there are several built-in user analytics plugins in Kusto/ADX: https://learn.microsoft.com/en-us/azure/kusto/query/useranalytics
one of them, for example, is the activity_engagement plugin: https://learn.microsoft.com/en-us/azure/kusto/query/activity-engagement-plugin
for example:
let T = datatable(Event:string, UID:int, Timestamp:datetime)
[
'event 1', 0, datetime(9/25/19 9:00 AM),
'event 2', 1, datetime(9/25/19 3:00 PM),
'event 3', 2, datetime(9/26/19 2:00 PM),
'event 4', 1, datetime(9/28/19 5:00 PM),
'event 5', 3, datetime(9/29/19 7:00 AM),
]
;
let min_date_time = toscalar(T | summarize startofday(min(Timestamp)));
let max_date_time = toscalar(T | summarize startofday(max(Timestamp)));
T
| evaluate activity_engagement (UID, Timestamp, 1d, 1d + max_date_time - min_date_time)
| project Timestamp, dcount_activities_outer
and, if you want to "fill the gap" for Sep-27, you can do the following:
let T = datatable(Event:string, UID:int, Timestamp:datetime)
[
'event 1', 0, datetime(9/25/19 9:00 AM),
'event 2', 1, datetime(9/25/19 3:00 PM),
'event 3', 2, datetime(9/26/19 2:00 PM),
'event 4', 1, datetime(9/28/19 5:00 PM),
'event 5', 3, datetime(9/29/19 7:00 AM),
]
;
let min_date_time = toscalar(T | summarize startofday(min(Timestamp)));
let max_date_time = toscalar(T | summarize startofday(max(Timestamp)));
range Timestamp from min_date_time to max_date_time step 1d
| join kind=leftouter (
T
| evaluate activity_engagement (UID, Timestamp, 1d, 1d + max_date_time - min_date_time)
| project Timestamp, dcount_activities_outer
) on Timestamp
| order by Timestamp asc
| project Timestamp, coalesce(dcount_activities_outer, prev(dcount_activities_outer))

How to insert xml string into Oracle database in a stored procedure?

My XML data is:
<NewDataSet>
<AotReversefeedback>
<Refid>N161144</Refid>
<DPID />
<TrdAccOpenId>92021144 </TrdAccOpenId>
<TrdAccOpenDate>May 7 2018 12:00AM </TrdAccOpenDate>
<EntryDate>25/03/2018</EntryDate>
<ITicketStatus>POA</ITicketStatus>
<LastupdatedDate>07/05/2018</LastupdatedDate>
<Status>ACTIVE</Status>
</AotReversefeedback>
<AotReversefeedback>
<Refid>N202240</Refid>
<DPID />
<TrdAccOpenId>83082240 </TrdAccOpenId>
<TrdAccOpenDate>May 7 2018 12:00AM </TrdAccOpenDate>
<EntryDate>03/05/2018</EntryDate>
<ITicketStatus>KRA</ITicketStatus>
<LastupdatedDate>07/05/2018</LastupdatedDate>
<Status>ACTIVE</Status>
</AotReversefeedback>
</NewDataSet>
and my table structure is
create table LMSDATA
(
refid nvarchar2(20),
DPID NVARCHAR2(20),
trdaccopenid number(9),
trdaccopendate nvarchar2(20),
entrydate date,
iticketstatus nvarchar2(20),
lastupdateddate date,
status nvarchar2(20)
);
The stored procedure will get input string which has xml data. What method can be used to insert XML data into table?
Would be this one:
WITH t AS
(SELECT XMLTYPE(
'<NewDataSet>
<AotReversefeedback>
<Refid>N161144</Refid>
<DPID />
<TrdAccOpenId>92021144 </TrdAccOpenId>
<TrdAccOpenDate>May 7 2018 12:00AM </TrdAccOpenDate>
<EntryDate>25/03/2018</EntryDate>
<ITicketStatus>POA</ITicketStatus>
<LastupdatedDate>07/05/2018</LastupdatedDate>
<Status>ACTIVE</Status>
</AotReversefeedback>
<AotReversefeedback>
<Refid>N202240</Refid>
<DPID />
<TrdAccOpenId>83082240 </TrdAccOpenId>
<TrdAccOpenDate>May 7 2018 12:00AM </TrdAccOpenDate>
<EntryDate>03/05/2018</EntryDate>
<ITicketStatus>KRA</ITicketStatus>
<LastupdatedDate>07/05/2018</LastupdatedDate>
<Status>ACTIVE</Status>
</AotReversefeedback>
</NewDataSet>') AS XML_DATA
FROM dual)
SELECT
refid,
DPID,
trdaccopenid,
trdaccopendate,
TO_DATE(entrydate_str, 'dd/mm/yyyy') AS entrydate,
iticketstatus,
TO_DATE(LastupdatedDate_str, 'dd/mm/yyyy') AS LastupdatedDate,
status
FROM t
CROSS JOIN XMLTABLE('/NewDataSet/AotReversefeedback' PASSING XML_DATA COLUMNS
refid NVARCHAR2(20) PATH 'Refid',
DPID NVARCHAR2(20) PATH 'DPID',
trdaccopenid NUMBER(9) PATH 'TrdAccOpenId',
trdaccopendate NVARCHAR2(20) PATH 'TrdAccOpenDate',
entrydate_str VARCHAR2(15) PATH 'EntryDate',
iticketstatus NVARCHAR2(20) PATH 'ITicketStatus',
lastupdateddate_str VARCHAR2(15) PATH 'LastupdatedDate',
status NVARCHAR2(20) PATH 'Status'
) x;
Results:
REFID DPID TRDACCOPENID TRDACCOPENDATE ENTRYDATE ITICKETSTATUS LASTUPDATEDDATE STATUS
N161144 92021144 May 7 2018 12:00AM 25.03.2018 POA 07.05.2018 ACTIVE
N202240 83082240 May 7 2018 12:00AM 03.05.2018 KRA 07.05.2018 ACTIVE
ExtractValue will also works but the function is deprecated.

Remove column in awk command

I want to remove column as for now i have command that include splitting, header and trailer in one command. I wanted to do splitting, header, trailer and remove first 3 columns in one command.
My original output is
Country Gender Plate Name Account Number Car Name
X F A Fara XXXXXXXXXX Ixora
X F b Jiha XXXXXXXXXX Saga
X M c Jiji XXXXXXXXXX Proton
My splited files:
first file
06032017
Country Gender Plate Name Account Number Car Name
X F A Fara XXXXXXXXXX Ixora
EOF 1
second file
06032017
Country Gender Plate Name Account Number Car Name
X F b Jiha XXXXXXXXXX Saga
EOF1
My desitre output:
06032017
Name Account Number Car Name
Fara XXXXXXXXXX Ixora
EOF 1
06032017
Name Account Number Car Name
Jiha XXXXXXXXXX Saga
EOF1
This is my splitted command:
awk -v date="$(date +"%d%m%Y")" -F\| 'NR==1 {h=$0; next}
{file="CAR_V1_"$1"_"$2"_"date".csv"; print (a[file]++?"": "DETAIL "date"" ORS h ORS) $0 > file}
END{for(file in a) print "EOF " a[file] > file}' HIRE_PURCHASE_testing.csv
To get an idea how to skip some fields:
$ echo "Field1 Field2 Field3 Field4 Field5 Field6" |awk '{print $0}'
Field1 Field2 Field3 Field4 Field5 Field6 #No skipping here. Printed just for comparison
$ echo "Field1 Field2 Field3 Field4 Field5 Field6" |awk '{print substr($0, index($0,$4))}'
Field4 Field5 Field6
$ echo "Field1 Field2 Field3 Field4 Field5 Field6" |awk '{$1=$2=$3="";print}'
Field4 Field5 Field6
#Mind the gaps in the beginning. You can use this if you need to "delete" particular columns.
#Actually you are not really delete columns but replace their contents with a blank value.
$ echo "Field1 Field2 Field3 Field4 Field5 Field6" |awk '{gsub($1FS$2FS$3FS$4,$4);print}'
Field4 Field5 Field6 #Columns 1-2-3-4 have been replaced by column4. No gaps here.
$ echo "Field1 Field2 Field3 Field4 Field5 Field6" |awk '{print $4,$5,$6}'
Field4 Field5 Field6
# If you have a fixed number of fields (i.e 6 fields) you can just do not print the first three fields and print the last three fields
Adapted to your existed code, the line
print (a[file]++?"": "DETAIL "date"" ORS h ORS) $0 > file
If changed to
print (a[file]++?"": "DETAIL "date"" ORS h ORS) substr($0, index($0,$4)) > file
Or to
print (a[file]++?"": "DETAIL "date"" ORS h ORS) $4,$5,$6 > file #if you have a fixed number of columns in the file)
should be enough.
I would sugggest next time you need help, simplify the problem to get help more easily. For the rest users is kind of hard to fully follow you.
Also have a look here to see how all those functions work in details: https://www.gnu.org/software/gawk/manual/html_node/String-Functions.html

Oracle Pivot - converting values into columns

I have table in Oracle 11g with 3 fields:
STUDYID | STUDY_PARAMETER | STUDY_VALUE
5268 | Age Group | ADULT (18-65)
5269 | Age Group | ADULT (18-65)
5270 | Age Group | ADULT (18-65)
5271 | Age Unit | ADULT (18-65)
1668A | Trial Type | ADULT (18-65)
5273 | Trial Type | Dispensing
5345 | Age Unit | Years
1668AC | Age Group | ADULTS (18-39)
So, what I need is to display values in this order:
STUDY_ID | AGE_GROUP | AGE_UNIT | TRIAL_TYPE
5268 | ADULT (18-65) | Years | Dispensing
5269 | ADULT (18-65) | (null) | (null)
1668AC | ADULTS (18-39)| Years | Non - Dispensing
and so on.
What I have so far is:
SELECT *
FROM (
SELECT STUDYID, STUDY_VALUE, STUDY_PARAMETER
FROM RD.STUDY_INFO
)
PIVOT (
SUM(STUDY_VALUE)
FOR (STUDY_PARAMETER)
IN (
'Age Unit' AS AGE_UNIT,
'Age Group' AS AGE_GROUP,
'Trial Type' AS TRIAL_TYPE
)
);
I learned this from examples on the net but I am not sure if I can use SUM() like this...?!
I get this error:
ORA-01722: invalid number
01722. 00000 - "invalid number"
Does anyone see what I am doing wrong?
Since the STUDY_VALUE column appears to be a string, you will need to use either the max() or min() aggregate function on the values:
SELECT *
FROM
(
SELECT STUDYID, STUDY_VALUE, STUDY_PARAMETER
FROM STUDY_INFO
)
PIVOT
(
MAX(STUDY_VALUE)
FOR (STUDY_PARAMETER) IN ('Age Unit' AS AGE_UNIT,
'Age Group' AS AGE_GROUP,
'Trial Type' AS TRIAL_TYPE)
);
See SQL Fiddle with Demo
You can try this query.
SELECT
ID,
MAX(Case When parameter='Age Group' then Value else '0' end) AS AgeGroup,
MAX(Case When parameter='Trial Type' then Value else '0'end)AS TrialType,
MAX(Case When parameter='Age Unit' then Value else '0'end)AS AgeUnit
FROM teststack
GROUP BY ID
ORDER BY ID

Resources