I have used information on this site for awhile, and now have a large project that I need some of your expertise on. I work within an Oracle 11g environment and also have Cognos Report Studio 10.1
I am building a multitab report that displays an analysis of a department's outbound orders. I have created a custom table that holds approximately 30 columns of data. From here, there are over 150 calculations that must be performed on the table daily. An example of these calculations are
1) How many orders received today?
2) Of the orders received today, how many shipped same day?
3) How many orders are on hold?
Basically 100s of 4-5 line queries on the core table. I have thought about creating a second table and with the use of WITH clauses, performing the calculations in a procedure and inserting into the table.
To get to the question, has anyone written a procedure/package that performed a large amount of calculations and is there any links/webpages that can be suggested? My searches have not led me to any examples of a report of this nature being created and am wanting to make sure that my approach is as efficient as possible. Thanks in advance for any information/resources.
A table and stored procedure is a great idea. Some considerations:
Will you be archiving these facts per day.
Ie, do you want to be able to lookup the orders that were on hold 5 days ago, or is all of the data in the 150 questions only relevant for the current date?
You can have a table with 150 columns, one for each question. That may make sense if you are archiving data and need one record per day.
The alternative is to create an order fact table with just two or three fields:
Fact_Name VARCHAR2(30)
Order_Fact NUMBER(10,2)
Last_Update_Date DATE
Your oracle stored procedure would query and update the facts one at a time:
insert into ORDER_FACTS
select "ORDERS_RECEIVED" fact_name, count(*) order_fact, sysdate
from ORDER_TABLE
where rcv_date = trunc(sysdate);
commit;
If you are only wanting to keep one record per fact, you would do an update.
If the answer to some of your questions is not a number, you may need to keep a separate table for facts with VARCHAR2 type.
If you like the sound of this solution, I can setup an example procedure tomorrow.
Edit:
Since you are storing 30 days worth of data, I would create the table at the Date detail level, and have 1 column to store each result. For my example I just included 3 columns so you could get the idea. First, create the table to hold your order facts.
create table order_facts
( DT DATE,
ORD_RCV NUMBER,
SAME_DAY_SHIPPED NUMBER,
ON_HOLD NUMBER);
I suggest the DT field stores the date alone. This will make the table easier to join with your calendar table, which you can use to join these facts to other reports easily if they are based on tables with Calendar in their star schema.
Next, create the procedure to update them:
CREATE OR REPLACE PROCEDURE CALC_ORDER_FACTS ( iDate date := trunc(sysdate), iPurgeDays number := 0)
IS
ddate DATE;
dummy DATE;
/*
Calc_order_facts
Date Author Note
04/11/2013 XXX Created this procedure
Param iDate (optional, default trunc(sysdate)
Specify Date to calculate order facts for
Param iPurgeDays number (optional, default 0)
Specify how many days to retain data. Data older than iPurgeDays will be deleted.
If 0, purging is disabled.
*/
BEGIN
ddate := iDate;
IF iPurgeDays > 0 THEN
dbms_output.put_line('Purging data more than ' || to_char(iPurgeDays) || ' days old.');
begin
delete ORDER_FACTS
where DT < trunc(ddate-iPurgeDays);
commit;
EXCEPTION
WHEN NO_DATA_FOUND THEN
dbms_output.put_line('Purge found no data.');
WHEN OTHERS THEN
-- Consider logging the error and then re-raise
dbms_output.put_line('Purged failed, rollling back...');
rollback;
END;
END IF;
-- If date does not already exist in table, insert it
begin
select dt
into dummy
from order_facts
where dt = ddate;
EXCEPTION
WHEN NO_DATA_FOUND THEN
insert into order_facts
values (ddate, null, null, null);
commit;
END;
-- ORD_RCV
-- Calculate Orders received
update order_facts
set ord_rcv =
(select count(*) ord_rcv
from ORDER_TABLE
where rcv_date = ddate)
where dt = ddate;
commit;
-- SAME_DAY_SHIPPED
-- Calculate Orders received and shipped on ddate
update order_facts
set same_day_shipped =
(select count(*) same_day_shipped
from order_table
where rcv_dt = ddate
and ship_dt = ddate)
where dt = ddate;
commit;
-- ON_HOLD
-- Total orders on_hold
-- This method applies if you are only concerned with total on hold
update order_facts
set on_hold =
(select count(*) ON_HOLD
from order_table
where status = 'HOLD')
where dt = ddate;
commit;
END CALC_ORDER_FACTS;
Related
Here is the conceptual model:
In the driver table, the totalTripMade attribute are all null, but the trip table contain all the trips the driver made, so I need to create a trigger such that when a record is inserted into trip, the trigger will update the totalTripMade by counting all the trips using the l#
create or replace trigger updateTrip
AFTER INSERT
on TRIP
for each row
declare
PRAGMA AUTONOMOUS_TRANSACTION;
totalTrips number;
begin
UPDATE DRIVER
SET TOTALTRIPMADE = (SELECT COUNT(L#) FROM TRIP WHERE L# =:NEW.L#)
WHERE L#=:NEW.L#;
SELECT COUNT(L#) into totalTrips FROM TRIP WHERE L# =:NEW.L#;
DBMS_OUTPUT.PUT_LINE(:NEW.L# || ' ' || totalTrips);
commit;
end;
/
SHOW ERRORS;
select * from DRIVER;
--
insert into TRIP values(109, 10001, 'SST005', sysdate );
insert into TRIP values(110, 10001, 'SST005', sysdate );
insert into TRIP values(111, 10001, 'SST005', sysdate );
--
select * from DRIVER;
select * from TRIP;
so the driver 10001 has made 27 trips before these 3 insert statement as there are 27 records based on the l# 10001, but after each insert, this code "DBMS_OUTPUT.PUT_LINE(:NEW.L# || ' ' || totalTrips);" keeps giving me 27 for 3 times, it should be 28, 29 then 30, may I know why is my trigger not counting the records based on the l# correctly in the trip table?
This is the output after running those insert statements:
SQL> insert into TRIP values(109, 10001, 'SST005', sysdate );
10001 27
1 row created.
SQL> insert into TRIP values(110, 10001, 'SST005', sysdate );
10001 27
1 row created.
SQL> insert into TRIP values(111, 10001, 'SST005', sysdate );
10001 27
1 row created.
I think the mistake is here:
I need to create a trigger such that when a record is inserted into trip, the trigger will update the totalTripMade by counting all the trips using the l#
I don't thing you do need to create such a trigger. If you need to count the total number of trips made by a driver, query the trips table directly using something like SELECT COUNT(*) FROM TRIP WHERE L# = <driver_number>. It won't slow the database down even if you have a lot of rows, as long as you have an index on the L# column.
There are a number of reasons why using a trigger like this isn't a good idea. Firstly, what happens if you insert multiple rows at the same time? If you insert 10 rows, should the trigger fire 10 times, or just once at the end?
Also, what happens if you update a row in the TRIP table to change the driver number, or delete a row in this table? Are you handling these cases?
But the single biggest reason why you shouldn't do something like this is that it breaks concurrency. Suppose a driver has 27 trips recorded in the database. Suppose that you are attempting to add three trips to this driver, and also someone else is attempting to add three different trips to the same driver at the same time. Neither of you can see each other's trips until you commit, both of you can only see the 27 trips that are already there. In your session, the trigger would attempt to set the total trips for this driver to 30, because your session can see the 27 existing trips and your three new trips, and in the other person's session, the trigger would also attempt to set the total trips for this driver to 30 as well, because this session sees the 27 existing trips and their three new trips, but not your three new trips. Once you have both committed, the driver then has 33 trips in the database, but their totalTripsMade is incorrectly showing as 30.
To answer your question
may I know why is my trigger not counting the records based on the l# correctly in the trip table?
I'm guessing this is because while working on this trigger you hit an ORA-04091 table is mutating error, saw a Google search result for this error that said to use PRAGMA AUTONOMOUS_TRANSACTION, so you added that to your trigger and the error went away. This pragma causes your trigger to run in an independent session: it can't see the trips you've inserted and so it prints 27 because it can only see the trips that are already there.
Compound triggers are one approach to avoiding ORA-04091 errors without using PRAGMA AUTONOMOUS_TRANSACTION. There's an example on using compound triggers here. However, you will also have to write similar triggers for UPDATE and DELETE, and compound triggers won't fix the concurrency problem I mentioned above.
However, I recommend getting rid of this trigger (and probably also the totalTripMade column in the DRIVER table as well). If you need to count the number of trips made by a driver, do this by counting rows in the TRIP table. It may come across as harsh for me to ask you to get rid of your code, but ultimately you are attempting to do something that won't work. I've seen other questions on StackOverflow where people have attempted to do similar things, so you're not alone in attempting something like this. At an intuitive level, adding these counts to the table seems like a reasonable thing to want to do. However, once you get to understand how the database works, you see the flaws in this.
I am trying to add a constraint to cust_birthdate, make sure the customer is more than 18 years old.
Here is my create table statement
create table customer(
cust_id char(5) NOT NULL,
cust_name varchar(30),
cust_birthdate date,
primary key(cust_id)
);
I found out cannot use SYSDATE in constraint, but I found a trigger online that fit my situation.
Here is the trigger I modified:
CREATE OR REPLACE TRIGGER trgCustomer
BEFORE INSERT ON customer
FOR EACH ROW
BEGIN
IF(ADD_MONTHS(:new.cust_birthdate, 18 * 12) < sysdate ) THEN
RAISE_APPLICATION_ERROR(-20001, 'Customer must be at least 18 years old.' );
END IF;
END;
/
However, the trigger will not work when I insert something like this:
insert into customer values('C0001', 'Chek Wei', TO_DATE('20-OCT-2016', 'dd-MON-yyyy'));
The 'Chek Wei' customer is less than 18 years old, but no error message is shown.
I do not yet learn trigger but I since I cannot use SYSDATE on constraint, I have no choice.
What's wrong with my trigger?
I am using Oracle Database Express Edition 11g Release 2
Any help is appreciated.
Isn't that logic reversed? you want only when you add 18 years to that date is
GREATER than sysdate to fire the error......
There you go.... :)
I am running a website using SQL Server 2008 and ASP.NET 4.0. I am trying to trace an issue down that my stored procedure is creating duplicate entries for the same date. Originally I thought this may be a couple post issue but the duplicates are recording the same date down to the milliseconds. One of the duplicates is at :'2013-04-26 15:48:28.323' All of the data is exactly the same except for the id.
#check_date is an input into the stored procedure which gives us the particular date we are looking at (entries are maid daily)
#formHeaderId is grabbed earlier in the stored procedure, getting the header ID as this is a detail table with a 1 to many relationship with the header.
The #getdate() entry is where I found the duplicate entries, there are entries with the exact getdate() values for different rows.
This doesn't occur with each entry either, it is randomly occurring in the application.
select #formHeaderId=stage2_checklist_header_id
from stage2_checklist_header
where environmental_forms_id=#envFormId
and checklist_monthyear=#inspected_month
order by start_date desc
if #formHeaderId = 0 begin
insert into stage2_checklist_header(
environmental_forms_id
,start_date
,checklist_monthyear
,st2h_load_date )
values( #envFormId
,#check_date
,#inspected_month
,getdate())
set #formHeaderId = scope_identity()
print 'inserted new header record ' + cast(#formHeaderId as varchar(50))
end
IF (NOT EXISTS(
SELECT *
FROM stage2_checklist_detail
WHERE stage2_checklist_header_id = #formHeaderId
AND check_date = #check_date
))
INSERT INTO stage2_checklist_detail
(stage2_checklist_header_id, check_date, st2_chk_det_load_date,
inspected_by)
VALUES
(#formHeaderId, #check_date, GETDATE(), #inspected_by)
SET #form_detail_id = SCOPE_IDENTITY()
PRINT 'inserted detail record ' + CAST(#form_detail_id AS VARCHAR(50))
Here is a similar case where the developer was able to track the duplicate entries to simultaneous calls from different spids (which sidestepped the EXISTS check). After experimenting with isolation levels and transactions - and trying to avoid deadlocks - it sounds like the solution in that case was to use sp_getapplock and sp_releaseapplock.
In the NOT EXISTS check, you are looking for records that have both the same ID and the same date. So, if the combination of ID AND date does not exist in the table, the row will be inserted.
In your description of the problem you state "All of the data is exactly the same except for the id". The ID being different will always cause an INSERT based on the logic you are using to check for existence.
I need to insert several rows into a SQL Server database based on Start Date and End Date textboxes.
E.G. tbStartDate.Text = "25/12/2012" and tbEndDate.Text = "29/12/2012" therefore I need to insert individual rows for the following dates:
25/12/2012
26/12/2012
27/12/2012
28/12/2012
29/12/2012
Please can you help me with the necessary T-SQL to achieve this?
As always there are a few ways. Here are some of them:
You can write code in your app that loops through the days and inserts a single record per day. (generally the worst design)
You can call some SQL script to do it all in the database.
You can wrap up your SQL script in a stored procedure and pass in the start and end date and get the stored procedure to do it for you.
You can cross join to a pre existing tally table and use it to generate your records.
If you can provide
-the version of SQL Server that you're using
-what the table looks like
-whether you're using C# or VB
then we can help further as it can be difficult to pass dates into databases. It can be particularly difficult if you do not validate them.
Anyway here is option 3 for you.
CREATE PROC dbo.t_test
#StartDate DATETIME,
#EndDate DATETIME
AS
WHILE #StartDate <= #EndDate
BEGIN
INSERT INTO YourTable(YourDateField) VALUES (#StartDate)
SET #StartDate = DATEADD(d,1,#StartDate)
END
Then you need to call this stored procedure (called dbo.t_test) from ASP.Net and pass in your two date parametes as dates.
Declare #Startdate datetime
Select #Startdate='20121025'
While #Startdate<='20121029'
begin
if not exists(select * from dummy where DATE=#Startdate)
insert into dummy (date) values (#Startdate)
set #Startdate=#Startdate + 1
end;
I have simultaneous request to a particular row in a table and PL/SQL statement is used to update the table by reading the data from master row in the same table and update the current range row and master row it read.
Algorithm is like this:-
Declare
variable declaration
BEGIN
Select (Values) into (values1) from table where <condition1> for update;
select count(*) into tempval from table where <condition2>;
if (tempval == 0) then
insert into table values(values);
else
select (values) into (values2) from table where <condition2> for update;
update table set (values1) where <condition2>;
end if
update table set (values1+incrval) where <condition1>
END;
Unfortunately the master row is updated properly with the correct sequence but the current range picks up the old value of the master range. It does the dirty read. Even though the transaction isolation level for the table is serialized.
Please could some tell me what is happening here?
This is working as designed. Oracle default, and only, read isolation lets the session see all of their own updates. If you perform:
INSERT INTO TABLE1 (col1) values (1);
COMMIT;
UPDATE TABLE1 SET col1 = 2 where col1 = 1;
SELECT col1 FROM TABLE1;
you will see 2 returned from the last query. Please read the Merge Explanation for how to use a MERGE statement to perform the insert or update based upon a single criteria.