I have a log table and the table has a varchar2 field which holds xml string like below:
In this example ClientName attribute did not change but Clientsurname changed.
I want to capture changed columns and their previous and new values.
The log table contains millions of records.
Which method can you suggest for parsing this data in an efficient way?
<r>
<columntag nameattribute="ClientName">
<new_value>Jeffrey</new_value>
<previous_value>Jeffrey</previous_value>
</columntag>
<columntag nameattribute="ClientSurname">
<new_value>Dijk</new_value>
<previous_value>Disk</previous_value>
</columntag>
</r>
Thank you
not 100% sure the below is what you are after but it should give you some ideas about how to go about it. Hope it is helpfull
CREATE TABLE "RM4SERV"."LOG_TEST" ( "TESTLOG" VARCHAR2(4000 BYTE))
Insert into RM4SERV.LOG_TEST (TESTLOG) values ('<r><columntag nameattribute="ClientName"><new_value>Jeffery</new_value><previous_value>Jeffery</previous_value> </columntag><columntag nameattribute="ClientSurname"><new_value>Dijk</new_value><previous_value>Disk</previous_value></columntag></r>');
Insert into RM4SERV.LOG_TEST (TESTLOG) values ('<r><columntag nameattribute="ClientName"><new_value>Jeffery</new_value><previous_value>Jeffery</previous_value> </columntag><columntag nameattribute="ClientSurname"><new_value>Disk</new_value><previous_value>Disk</previous_value></columntag></r>');
Insert into RM4SERV.LOG_TEST (TESTLOG) values ('<r><columntag nameattribute="ClientName"><new_value>Jeffery</new_value><previous_value>Jim</previous_value> </columntag><columntag nameattribute="ClientSurname"><new_value>Dijks</new_value><previous_value>Diskett</previous_value></columntag></r>');
declare
v_logrec varchar2(4000) := null;
v_recnum number := 0;
cursor c_logs is
select testlog from log_test;
cursor c_records is
select extractValue(x.column_value, '/columntag/#nameattribute') as column_name,
extractValue(x.column_value, '/columntag/new_value') as new_value,
extractValue(x.column_value, '/columntag/previous_value') as previous_value
from TABLE(XMLSequence(extract(xmltype.createxml(v_logrec), '//columntag'))) x
where extractValue(x.column_value, '/columntag/new_value') != extractValue(x.column_value, '/columntag/previous_value');
begin
for v_log in c_logs loop
v_logrec := v_log.testlog;
v_recnum := v_recnum + 1;
dbms_output.put_line(v_recnum);
for v_rec in c_records loop
SYS.dbms_output.put_line(v_rec.column_name || ' : *' || v_rec.new_value || '* : *' || v_rec.previous_value || '*');
end loop;
end loop;
end;
This would give you the below output (so Surname different in first record, nothing different in the second and both different in the third)...
1
ClientSurname : Dijk : Disk
2
3
ClientName : Jeffery : Jim
ClientSurname : Dijks : Diskett
Related
I am deleting a massive table and want to delete the table in batches. I am deleting records older than 467 days. I want to insert the dbms_output status in the following procedure to be written to a table that has two columns such as the number of records to be deleted with timestamp columns:
CREATE OR REPLACE PROCEDURE delete_tab (tablename IN VARCHAR2, nrows IN NUMBER ) IS
sSQL1 VARCHAR2(2000);
sSQL2 VARCHAR2(2000);
nCount NUMBER;
BEGIN
DBMS_OUTPUT.enable (100000);
nCount := 0;
sSQL1:='delete from '|| tablename ||
' where ROWNUM < ' || nrows || ' and where cast(time_stamp as date) < sysdate - 467';
sSQL2:='select count(ROWID) from ' || tablename ||
' where cast(time_stamp as date) < sysdate - 467';
EXECUTE IMMEDIATE sSQL2 INTO nCount;
LOOP
EXECUTE IMMEDIATE sSQL1;
nCount := nCount-nrows;
DBMS_OUTPUT.PUT_LINE('Existing records to be deleted: ' || to_char(nCount));
commit;
EXIT WHEN nCount = 0;
END LOOP;
END delete_tab;
/
Let me know how I can add an insert statement within the block to write the progress.
Besides the missing Insert there are a couple other minor issues and a major issue with this code.
ssql1:='delete from '|| tablename ||
' where ROWNUM < ' || nrows || ' and where cast(time_stamp as date) < sysdate - 467';
If the above were valid then when executed it nrows-1 (not nrows as I suspect you are thinking) every time as long as that many rows still exist. However, it is not valid; the where clause is invalid. The proper format for the where clause is
"Where <condition> and <condition> ..."; do not repeat where.
ssql2:='select count(ROWID) from ' || tablename ||
' where cast(time_stamp as date) < sysdate - 467';
There is no reason to count the rowids, every row has exactly 1. ROWID is pseudo column telling Oracle basically where on the disk the row is located. You have probably heard that it is the fastest way to retrieve a specific - that is true - but it is not true selecting rowid itself is faster, it will if anything, be slower. It would require a full table scan and calculation of its value for every row in the table. The optimizer may notice what is happening and change this to select count() but why hope for that just start with count().
Now we get to the worst issue. Unless the total number of rows to delete is an exact multiple of the parameter nrows then the procedure becomes a never ending loop until it throws the exception ORA-22054: underflow error.
This is because result as the only exit condition is "exit when ncount = 0". Suppose you have 1002 rows to delete and nrows is 1000. The first iteration deletes 999 then reduces ncount to 2 (1002-1000). Then the second iteration deletes another 999 and reduces ncount to -998 (2-1000). The procedure continues iterating until eventually ncount gets so small it cannot be held any longer (something like -1*10^39 - 1).
You could change the exit condition to "exit when nCount < 1". But even that is not necessary, no need for the code to calculate the exit condition at all. Let Oracle tell it. The delete statement returns SQL%rowcount that contains the number of rows processed by the last DML statement. If no are processed it returns 0. So the exist condition becomes a simple "exit when sql%rowcount = 0". Lesson: always be careful with equal 0 conditions.
Taking all this into account the procedure becomes. (Also changed variable names as I do not like ssql1, ssql2 and reusing a variable for 2 things. When using such I get confused as to which is which, esp on large procedures.):
create or replace procedure delete_tab
( tablename in varchar2
, nrows in number
)
is
delete_sql varchar2(2000);
count_sql varchar2(2000);
deleted_count number;
expect_delete number;
begin
dbms_output.enable (100000);
count_sql := 'select count(*) from ' || tablename ||
' where time_stamp < current_timestamp - 467';
dbms_output.put_line('COUNT_SQL: ' || count_sql
);
execute immediate count_sql into expect_delete;
dbms_output.put_line('Existing records to be deleted: ' || to_char(expect_delete));
delete_sql := 'delete from '|| tablename ||
' where rownum <= ' || nrows ||
' and cast(time_stamp as date) < sysdate - 467';
dbms_output.put_line('DELETE_SQL: '
|| delete_sql
);
deleted_count := 0;
loop
execute immediate delete_sql;
exit when sql%rowcount = 0;
deleted_count := deleted_count+sql%rowcount;
commit;
end loop;
insert into delete_log( table_name, date_deleted, rows_deleted)
values (tablename, sysdate, deleted_count) ;
commit; -- commit the deletes in final loop and insert
end delete_tab;
See demo (includes example of danger with ncount = 0 and procedure has additional trace information). BTW in a production run my commit interval (nRows) would something like 500000 or larger.
I want to do something like this:
type tab_imb is table of (cod number, job varchar2(10));
but SQL doesn't let me. Can I declare a nested table with multiple columns in PL/SQL?
Well Yes you can, sort of, you cannot do it directly. What you need is create an object that defines your "inner" column definitions then create a collection (table) of that object type.
So at the schema level:
create type imb_obj_type as object(cod number, job varchar2(20));
create type tab_imb is table of imb_obj_type;
declare
v_imb tab_imb := tab_imb();
begin
v_imb.extend(2);
v_imb(1) := imb_obj_type(1,'Lead');
v_imb(2) := imb_obj_type(2,'Developer');
dbms_output.put_line('----- Job List -----');
for indx in 1 .. v_imb.count
loop
dbms_output.put_line('cod=>' || v_imb(indx).cod || ', ' ||
'Job=>' || ', ' || v_imb(indx).job
);
end loop;
end ;
At the procedure or anonymous block you can use the above or define a record:
declare
type imb_rec is record(cod number, job varchar2(20));
type tab_imb is table of imb_rec;
v_imb tab_imb := tab_imb();
begin
v_imb.extend(2);
v_imb(1).cod := 1;
v_imb(1).job := 'Lead';
v_imb(2).cod := 2;
v_imb(2).job := 'Developer';
dbms_output.put_line('----- Job List -----');
for indx in 1 .. v_imb.count
loop
dbms_output.put_line('cod=>' || v_imb(indx).cod || ', ' ||
'Job=>' || ', ' || v_imb(indx).job
);
end loop;
end ;
I want to put single clob data column value in 2 varchar2 columns by checking the length of CLOB column, but i am getting error in case statement, line is marked in * *, it says syntex error , what am i doing wrong
DECLARE
v_tot_rows NUMBER (3);
rqst_xml_1 ISG.CERT_TEST_CASE_GTWY_TXN.RQST_XML_1_TX%TYPE;
rqst_xml_2 ISG.CERT_TEST_CASE_GTWY_TXN.RQST_XML_2_TX%TYPE;
CURSOR req_res_populate_cur
IS
SELECT scptc.SWR_CERT_PRJCT_TEST_CASE_ID,
orb_txn.MIME_HEAD_TX,
orb_txn.RSPNS_XML_TX,
orb_msg.RQST_GNRL_VLD_JSON_TX,
orb_msg.RQST_TEST_CASE_VLD_JSON_TX,
orb_msg.MRCH_ID
(
CASE
WHEN DBMS_LOB.GETLENGTH (orb_txn.RQST_XML_TX) <= 4000 THEN
rqst_xml_1 := CAST ( orb_txn . RQST_XML_TX AS VARCHAR2 ( 4000 ) ) * ,
rqst_xml_2 := ''
WHEN DBMS_LOB.GETLENGTH(orb_txn.RQST_XML_TX)>4000 THEN
rqst_xml_1:=CAST(substr(orb_txn.RQST_XML_TX,1,4000) AS VARCHAR2(4000)),
rqst_xml_2:=CAST(substr(orb_txn.RQST_XML_TX,4001)
END
)
FROM ISG.online_messages msg
JOIN ISG.SWR_CERT_PRJCT_TEST_CASE scptc
ON msg.online_message_id = scptc.TXN_ID,
ISG.GTWY_PLTFM_TXN_MSG orb_msg
JOIN ISG.GTWY_PLTFM_TXN orb_txn
ON orb_msg.GTWY_PLTFM_TXN_ID = orb_txn.GTWY_PLTFM_TXN_ID
WHERE msg.SPEC_ID = 60;;
BEGIN
FOR req_res IN req_res_populate_cur
LOOP
DBMS_OUTPUT.PUT_LINE (req_res.SWR_CERT_PRJCT_TEST_CASE_ID,
req_res.MIME_HEAD_TX,
req_res.rqst_xml_1,
req_res.rqst_xml_2,
req_res.RSPNS_XML_TX,
req_res.RQST_GNRL_VLD_JSON_TX,
req_res.RQST_TEST_CASE_VLD_JSON_TX,
req_res.MRCH_ID);
END LOOP;
END;
Your problem is your invalid SELECT-statement. You're trying to set variables (of your plsql-block) within a query. That's not intended or allowed.
You need to select the values into columns. Here i added two columns. One for each xml-value.
SELECT scptc.SWR_CERT_PRJCT_TEST_CASE_ID,
orb_txn.MIME_HEAD_TX,
orb_txn.RSPNS_XML_TX,
orb_msg.RQST_GNRL_VLD_JSON_TX,
orb_msg.RQST_TEST_CASE_VLD_JSON_TX,
orb_msg.MRCH_ID,
CASE --Column-Start
WHEN DBMS_LOB.GETLENGTH (orb_txn.RQST_XML_TX) <= 4000
THEN
CAST (orb_txn.RQST_XML_TX AS VARCHAR2 (4000))
WHEN DBMS_LOB.GETLENGTH (orb_txn.RQST_XML_TX) > 4000
THEN
CAST (
SUBSTR (orb_txn.RQST_XML_TX, 1, 4000) AS VARCHAR2 (4000))
END
AS my_rqst_xml_1, -- Column-End. In this column you'll have the value for xml_1
CASE --Column-Start
WHEN DBMS_LOB.GETLENGTH (orb_txn.RQST_XML_TX) <= 4000
THEN
''
WHEN DBMS_LOB.GETLENGTH (orb_txn.RQST_XML_TX) > 4000
THEN
CAST (SUBSTR (orb_txn.RQST_XML_TX, 4001) AS VARCHAR2 (4000))
END
AS my_rqst_xml_2 -- Column-End. In this column you'll have the value for xml_12
FROM ISG.online_messages msg
JOIN ISG.SWR_CERT_PRJCT_TEST_CASE scptc
ON msg.online_message_id = scptc.TXN_ID,
ISG.GTWY_PLTFM_TXN_MSG orb_msg
JOIN ISG.GTWY_PLTFM_TXN orb_txn
ON orb_msg.GTWY_PLTFM_TXN_ID = orb_txn.GTWY_PLTFM_TXN_ID
WHERE msg.SPEC_ID = 60
Afterwards you can work with the result and get the values from it.
BEGIN
FOR req_res IN req_res_populate_cur
LOOP
DBMS_OUTPUT.PUT_LINE (req_res.SWR_CERT_PRJCT_TEST_CASE_ID,
req_res.MIME_HEAD_TX,
req_res.my_rqst_xml_1, -- here we can see the values
req_res.my_rqst_xml_2, -- here too
req_res.RSPNS_XML_TX,
req_res.RQST_GNRL_VLD_JSON_TX,
req_res.RQST_TEST_CASE_VLD_JSON_TX,
req_res.MRCH_ID);
-- And here we could store the values into variables or call some procedures etc.
rqst_xml_1 := req_res.my_rqst_xml_1;
rqst_xml_2 := req_res.my_rqst_xml_2;
END LOOP;
END;
I've to guess, but it seems you didn't want to declare the variables:
rqst_xml_1 ISG.CERT_TEST_CASE_GTWY_TXN.RQST_XML_1_TX%TYPE;
rqst_xml_2 ISG.CERT_TEST_CASE_GTWY_TXN.RQST_XML_2_TX%TYPE;
This would be only needed if you want to work with the values.
Getting this error when ever I tried to execute the entire procedure. Below is the piece of code from the procedure.
SELECT
DISTINCT SUBSTRING (a.GL06001,19,10) as ProjectNo,
--PR01040 UD_10,
SUBSTRING (a.GL06001,1,8) as AccountNo,
a.GL06002 as TransNo,
a.GL06004 as TransAmount,
a.GL06003 as TransDate,
a.GL06005 as TransDesc,
'GL' as SourceType,
' ' as ResourceCode,
' ' as TransLine,
0 as CostPR,
'000000' as PRTransNo,
a.GL06027 as SubprojNo,
a.GL06028 as ActiLineNo,
a.GL06012 as TransType,
a.GL06016 as Counter
from ScalaMX.dbo.GL06PA17 a where a.GL06003 between '2017-02-21 00:00:00.000' and '2017-03-01 00:00:00.000'
There are actually 18000+ rows and 15 columns. Any hint on how to track which column has B value?
I downloaded the result in Excel and ctrl+f 'B' But still no clue and I couldn't find it.
convert into stored procedure, see below sample
declare
n integer;
m integer;
begin
for i in ( select primarykeycolumn, a,b from table )
loop
begin
n := i.a;
m := i.b;
exception
when others then
dbms_output.put_line(i.primarykeycolumn);
end
end loop;
end;
When conversion error happens it will catch the exception hence find the primary key.
I have this tables:
CREATE TABLE departments (
dep_na number(2) NOT NULL PRIMARY KEY,
dname VARCHAR2(15),
loc VARCHAR2(15)
);
INSERT INTO departments VALUES (20,'CONTABILITY','SEVILLA');
INSERT INTO departments VALUES (30,'INVEST','MADRID');
COMMIT;
CREATE TABLE employees (
emp_nu number(4) NOT NULL PRIMARY KEY,
surname VARCHAR2(10),
oficio VARCHAR2(10),
dir number(4),
date_a DATE ,
salar number(6,2),
comis number(6,2),
dep_na number(2) NOT NULL REFERENCES departments(dept_no)
);
ALTER SESSION SET NLS_DATE_FORMAT='DD/MM/YYYY';
INSERT INTO employees VALUES (7369,'SANCHEZ','EMPLEADO',7902,'17/12/1990',1040,NULL,20);
INSERT INTO employees VALUES (7499,'ARROYO','VENDEDOR',7698,'20/02/1990',1500,390,30);
COMMIT;
create or replace
TYPE TDEP AS OBJECT(
dep_na NUMBER(2),
dname VARCHAR2(15),
loc VARCHAR2(15)
);
CREATE OR REPLACE TYPE TEMPLE AS OBJECT(
emp_nu number(4),
surname VARCHAR2(10),
oficio VARCHAR2(10),
dir number(4),
date_a DATE,
salar number(6,2),
comision number(6,2),
dep_na TDEP
);
CREATE OR REPLACE TYPE VEMPLE AS VARRAY(20) OF TEMPLE;
I created the following table pack with varray and types, but I have the problem that when I insert the contents of the employees and departments tables in the table pack the computer gives me error.
CREATE TABLE pack(
array_employees VEMPLE,
departme TDEP
);
I have problems with this code:
DECLARE
T VEMPLE;
A TDEP
CURSOR C1 is select * from departamentos order by dep_na;
CURSOR C2(DEPAR NUMBER) is select * from empleados where dep_na = depar;
j integer := 1;
BEGIN
for i in C1 LOOP
DBMS_OUTPUT.PUT_LINE(i.dep_na);
T := NEW VEMPLE();
A := NEW TDEP();
j := 1;
for x in C2(i.dep_na) loop
if j < T.LIMIT THEN
DBMS_OUTPUT.PUT_LINE(x.apellido || ' - ' || i.dep_na);
T.extend;
T(j) := NEW TEMPLE(NULL, NULL, NULL, NULL);
T(j).departments := NEW TDEP(i.loc, NULL, NULL);
j := j + 1;
end if;
end loop;
INSERT INTO Grupos VALUES(i.A, T);
end loop;
end;
/
I need help with this query in Oracle. I have problems with the cursor.
The immediate problem is that you're just missing a semicolon after A TDEP in the declare section, which is causing your ORA-06550: line 4, column 1: PLS-00103: Encountered the symbol "CURSOR" when expecting ... error. Which you haven't shown, but you said it's a cursor error, so that seems to line up.
But you have lots of other errors, some from inconsistent naming which I think is from partially changing things for posting, but partly from how you're creating object instances. I think this is pretty much clean:
DECLARE
T VEMPLE;
A TDEP;
CURSOR C1 is select * from departments order by dep_na;
CURSOR C2(DEPAR NUMBER) is select * from employees where dep_na = depar;
j integer := 1;
BEGIN
for i in C1 LOOP
DBMS_OUTPUT.PUT_LINE(i.dep_na);
T := NEW VEMPLE();
-- supply values to constructor
A := NEW TDEP(i.dep_na, i.dname, i.loc);
j := 1;
for x in C2(i.dep_na) loop
if j < T.LIMIT THEN
DBMS_OUTPUT.PUT_LINE(x.surname|| ' - ' || i.dep_na);
T.extend;
-- supply values to constructor
T(j) := NEW TEMPLE(x.emp_nu, x.surname, x.oficio,
x.dir, x.date_a, x.salar, x.comis, A);
-- no idea what the next line is doing
-- T(j).departments := NEW TDEP(i.loc, NULL, NULL);
j := j + 1;
end if;
end loop;
-- elements were wrong way round, which wouldn't matter if you
-- included the column names - which is good practice anyway
INSERT INTO pack VALUES(T, A);
end loop;
end;
/
SQL Fiddle. (Don't try to select from the table in the Fiddle though, it struggles with object types; this just shows it doesn't error...)
I assume this is an exercise. There are probably better and more efficient ways to convert from a relational to an object schema.