I am writing a PL/SQL parser to identify the operations(Select,Insert,Delete) performed on the Table when I run Procedure, Function or Package.
GOAL:I Goal of this tool is to identify which all the tables will be affected by running the procedure,Fun to prepare with better test case.
Any better ideas or tool will really help a lot.
INPUT:
some SQL file with procedure
or proc file.
OUTPUT required is:
SELECT from: First_table, secondTable
-> In procedure XYZ --This is if the procedure is calling one more procedure
INSERT into: SomeTable
INSERT into: SomeDiffTable
-> END of procedure XYZ --End of one more procedure.
DELETE from: xyzTable
INSERT into: OnemoreTable
My requirement is When I am parsing porc1 if it calls another proc2. I have to go inside that proc2 to find out what all the operation is performed in that and come back to proc1 and continue.:
For this I have to store the all procedure some where and while parsing I have to check each token(word with space) in the tempStorage to find out if it is procedure or not.
As my logic's takes lot of time. Can any body suggest better logic to achieve my GOAL.
There's also the possiblity of triggers being involved. That adds an additional layer of complexity.
I'd say you're better off mining DBA_DEPENDENCIES with a recursive query to determine impact analysis in the abstract; it won't capture dynamic SQL, but nothing will 100% of the time. In your case, proc1 depends on proc2, and proc2 depends on whatever it depends on, and so forth. It won't tell you the nature of the dependency - INSERT, UPDATE, DELETE, SELECT - but it's a beginning.
If you're really interested in determining the actual impact of a single-variable-value run of a procedure, implement it in a non-production system, and then turn auditing on your system up to 11:
begin
for i in (select owner, object_type, object_name from dba_objects
where owner in ([list of application schemas]
and object_type in ('TABLE', 'PACKAGE', 'PROCEDURE', 'FUNCTION', 'VIEW')
loop
execute immediate 'AUDIT ALL ON ' || i.owner || '.' || i.object_type ||
' BY SESSION';
end loop;
end;
/
Run your test, and see what objects got touched as a result of the exectution by mining the audit trail. It's not bulletproof, as it only audits objects that got touched by that execution, but it does tell you how they got touched.
Related
I have been asked to create an SP which creates temporary table and insert some records.
I am preparing some sample code for the same as mentioned below but the output is not displayed.
create or replace procedure Test
is
stmt varchar2(1000);
stmt2 varchar2(1000);
begin
stmt := 'create global temporary table temp_1(id number(10))';
execute immediate stmt;
insert into temp_1(id) values (10);
execute immediate 'Select * from temp_1';
execute immediate 'Drop table temp_1';
commit;
end;
When i am executing the SP by (Exec Test) desired O/P is not displayed.
I am expecting O/P of "Select * from temp_1" to be displayed. But it is not happening. Please suggest where i am doing wrong.
But i am interesting in knowing why ( execute immediate 'Select * from temp_1';) do not yield any result
For two reasons. Firstly because as #a_horse_with_no_name said PL/SQL won't display the result of a query. But more importantly here, perhaps, the query is never actually executed. This behaviour is stated in the documentation:
If dynamic_sql_statement is a SELECT statement, and you omit both into_clause and bulk_collect_into_clause, then *execute_immediate_statement( never executes.
You would have to execute immediate into a variable, or more likely a collection if your real scenario has more than one row, and then process that data - iterating over the collection in the bulk case.
There is not really a reliable way to display anything from PL/SQL; you can use dbms_output but that's more suited for debugging than real output, and you usually have no guarantee that the client will be configured to show whatever you put into its buffer.
This is all rather academic since creating and dropping a GTT on the fly is not a good idea and there are better ways to accomplish whatever it is you're trying to do.
The block you showed shouldn't actually run at all; as you're creating temp_1 dynamically, the static SQL insert into temp_1 will error as that table does not yet exist when the block is compiled. The insert would have to be dynamic too. Any dynamic SQL is a bit of a warning sign you're maybe doing something wrong, though it is sometimes necessary; having to do everything dynamically suggests the whole approach needs a rethink, as does creating objects at runtime.
Why do we use "Execute immediate" in plsql? I know we use it to execute dynamic sql statements. But still I'm unable to convince interviewer. Could anybody give me an exact and proper answer for this? Though I use it everyday, but still unable to explain it. One thing I know is, it's used in DML statements and to retrieve multiple rows through select statement. Please give an exact definition for using "Execute immediate".
EXECUTE IMMEDIATE enables execution of a DML or DDL statement which is held as a string and only evaluated at runtime. This enables one to dynamically create the statement based on program logic. EXECUTE IMMEDIATE is also the only way you can execute DDL within a PL/SQL block. See the Oracle Manual for a complete and thorough review of these features.
While this is a very useful facility it should be used with care. Unless there is an explicit need for dynamic sql then it is better to directly declare the sql within your PL/SQL code. This will enable Oracle to parse the SQL at compile time for validity and also reduce overhead when executing the pre-compiled statement. Also you need to be very careful to avoid SQL injection attacks when dynamically building SQL.
For example, let's say you have some kind of application where users define their company and all the work they do. If you want to make it dynamically for users, then you can give them option to create their own models(tables). Since you do not know how many tables they will have, and how many attributes this tables will have you can use execute immediate.
DDL Statement in Procedures or Anonymous PLSQL blocks
Here is an example showing how to use dynamic DDL to create, drop and re-create a table:
BEGIN
EXECUTE IMMEDIATE 'create table abcd (efgh NUMBER)';
EXECUTE IMMEDIATE 'drop table abcd';
EXECUTE IMMEDIATE 'create table abcd (efgh VARCHAR2(10))';
END;
You can use this method to execute any DDL.
Though it's not suggested to use Execute Immediate for DDL, rather Global Temp Table should be used.
To execute DML statements in Procedures or Anonymous PLSQL blocks
To execute DML commands more often than DDL. With dynamic SQL you can issue inserts, updates and deletes just as you can with static SQL:
BEGIN
EXECUTE IMMEDIATE 'INSERT INTO abcd (efgh) VALUES (:text_string)'
USING 'ijkl';
EXECUTE IMMEDIATE 'INSERT INTO abcd (efgh) VALUES (:text_string)'
USING 'mnop';
EXECUTE IMMEDIATE 'UPDATE abcd ' ||
'SET efgh = :text_string WHERE efgh = :second_string'
USING 'qrst', 'mnop';
EXECUTE IMMEDIATE 'DELETE FROM abcd ' ||
'WHERE efgh = :text_string '
USING 'qrst';
END;
In Select queries with bind variables
As useful as DDL and DML are, a database is not very useful if you can't get your data out. You can also use execute immediate to select your data back out.
DECLARE
v_data abcd.efgh%TYPE;
v_data_row abcd%ROWTYPE;
BEGIN
EXECUTE IMMEDIATE 'SELECT efgh FROM abcd WHERE efgh = :text_string'
INTO v_data
USING 'ijkl';
DBMS_OUTPUT.PUT_LINE( 'Column Variable: ' || v_data );
EXECUTE IMMEDIATE 'SELECT * FROM abcd WHERE efgh = :text_string'
INTO v_data_row
USING 'ijkl';
DBMS_OUTPUT.PUT_LINE( 'Row Variable: ' || v_data_row.efgh );
END;
INFO: Column Variable: ijkl
INFO: Row Variable: ijkl
I have a procedure to insert records through a select statement. I need my procedure to perform something if the insert fails or completes. How can I let my procedure know if an insert is completed or failed?
Currently I am matching the count between fetched data and loaded data. If they match, then I take it as completed.(not sure if this approach is the most efficient)
What if the load fails for whatever reason, how I do trap that?
Thanks
You need to trap exceptions and provide a useful handler. Perhaps something like the following:
BEGIN
INSERT INTO SOME_TABLE (A_COLUMN) VALUES ('XYZ');
EXCEPTION
WHEN OTHERS THEN
DBMS_OUTPUT.PUT_LINE('Error: ' || SQLCODE || ':' || SQLERRM);
RAISE;
END;
The exception handler begins with the line containing EXCEPTION.
There are many pre-defined exceptions that either you or the Oracle system can raise, and you can define your own exceptions and use them. The link given in the comment above should be helpful in understanding this concept.
Share and enjoy.
I would like to ask you how would you increase the performance on Insert cursor in this code?
I need to use dynamic plsql to fetch data but dont know how to improve the INSERT in best way. like Bulk Insert maybe?
Please let me know with code example if possible.
// This is how i use cur_handle:
cur_HANDLE integer;
cur_HANDLE := dbms_sql.open_cursor;
DBMS_SQL.PARSE(cur_HANDLE, W_STMT, DBMS_SQL.NATIVE);
DBMS_SQL.DESCRIBE_COLUMNS2(cur_HANDLE, W_NO_OF_COLS, W_DESC_TAB);
LOOP
-- Fetch a row
IF DBMS_SQL.FETCH_ROWS(cur_HANDLE) > 0 THEN
DBMS_SQL.column_value(cur_HANDLE, 9, cont_ID);
DBMS_SQL.COLUMN_VALUE(cur_HANDLE, 3, proj_NR);
ELSE
EXIT;
END IF;
Insert into w_Contracts values(counter, cont_ID, proj_NR);
counter := counter + 1;
END LOOP;
You should do database actions in sets whenever possible, rather than row-by-row inserts. You don't tell us what CUR_HANDLE is, so I can't really rewrite this, but you should probably do something like:
INSERT INTO w_contracts
SELECT ROWNUM, cont_id, proj_nr
FROM ( ... some table or joined tables or whatever... )
Though if your first value there is a primary key, it would probably be better to assign it from a sequence.
Solution 1) You can populate inside the loop a PL/SQL array and then just after the loop insert the whole array in one step using:
FORALL i in contracts_tab.first .. contracts_tab.last
INSERT INTO w_contracts VALUES contracts_tab(i);
Solution 2) if the v_stmt contains a valid SQL statement you can directly insert data into the table using
EXECUTE IMMEDIATE 'INSERT INTO w_contracts (counter, cont_id, proj_nr)
SELECT rownum, 9, 3 FROM ('||v_stmt||')';
"select statement is assembled from a website, ex if user choose to
include more detailed search then the select statement is changed and
the result looks different in the end. The whole application is a web
site build on dinamic plsql code."
This is a dangerous proposition, because it opens your database to SQL injection. This is the scenario in which Bad People subvert your parameters to expand the data they can retrieve or to escalate privileges. At the very least you need to be using DBMS_ASSERT to validate user input. Find out more.
Of course, if you are allowing users to pass whole SQL strings (you haven't provided any information regarding the construction of W_STMT) then all bets are off. DBMS_ASSERT won't help you there.
Anyway, as you have failed to give the additional information we actually need, please let me spell it out for you:
will the SELECT statement always have the same column names from the same table name, or can the user change those two?
will you always be interested in the third and ninth columns?
how is the W_STMT string assembled? How much control do you have over its projection?
I have stored procedure that insanely times out every single time it's called from the web application.
I fired up the Sql Profiler and traced the calls that time out and finally found out these things:
When executed the statements from within the MS SQL Management Studio, with same arguments (in fact, I copied the procedure call from sql profile trace and ran it): It finishes in 5~6 seconds avg.
But when called from web application, it takes in excess of 30 seconds (in trace) so my webpage actually times out by then.
Apart from the fact that my web application has its own user, every thing is same (same database, connection, server etc)
I also tried running the query directly in the studio with the web application's user and it doesn't take more than 6 sec.
How do I find out what is happening?
I am assuming it has nothing to do with the fact that we use BLL > DAL layers or Table adapters as the trace clearly shows the delay is in the actual procedure. That is all I can think of.
EDIT I found out in this link that ADO.NET sets ARITHABORT to true - which is good for most of the time but sometime this happens, and the suggested work-around is to add with recompile option to the stored proc. In my case, it's not working but I suspect it's something very similar to this. Anyone knows what else ADO.NET does or where I can find the spec?
I've had a similar issue arise in the past, so I'm eager to see a resolution to this question. Aaron Bertrand's comment on the OP led to Query times out when executed from web, but super-fast when executed from SSMS, and while the question is not a duplicate, the answer may very well apply to your situation.
In essence, it sounds like SQL Server may have a corrupt cached execution plan. You're hitting the bad plan with your web server, but SSMS lands on a different plan since there is a different setting on the ARITHABORT flag (which would otherwise have no impact on your particular query/stored proc).
See ADO.NET calling T-SQL Stored Procedure causes a SqlTimeoutException for another example, with a more complete explanation and resolution.
I also experience that queries were running slowly from the web and fast in SSMS and I eventually found out that the problem was something called parameter sniffing.
The fix for me was to change all the parameters that are used in the sproc to local variables.
eg. change:
ALTER PROCEDURE [dbo].[sproc]
#param1 int,
AS
SELECT * FROM Table WHERE ID = #param1
to:
ALTER PROCEDURE [dbo].[sproc]
#param1 int,
AS
DECLARE #param1a int
SET #param1a = #param1
SELECT * FROM Table WHERE ID = #param1a
Seems strange, but it fixed my problem.
Not to spam, but as a hopefully helpful solution for others, our system saw a high degree of timeouts.
I tried setting the stored procedure to be recompiled by using sp_recompile and this resolved the issue for the one SP.
Ultimately there were a larger number of SP's that were timing-out, many of which had never done so before, by using DBCC DROPCLEANBUFFERS and DBCC FREEPROCCACHE the incident rate of timeouts has plummeted significantly - there are still isolated occurrences, some where I suspect the plan regeneration is taking a while, and some where the SPs are genuinely under-performant and need re-evaluation.
Could it be that some other DB calls made before the web application calls the SP is keeping a transaction open? That could be a reason for this SP to wait when called by the web application. I say isolate the call in the web application (put it on a new page) to ensure that some prior action in the web application is causing this issue.
You can target specific cached execution plans via:
SELECT cp.plan_handle, st.[text]
FROM sys.dm_exec_cached_plans AS cp
CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS st
WHERE [text] LIKE N'%your troublesome SP or function name etc%'
And then remove only the execution plans causing issues via, for example:
DBCC FREEPROCCACHE (0x050006003FCA862F40A19A93010000000000000000000000)
I've now got a job running every 5 minutes that looks for slow running procedures or functions and automatically clears down those execution plans if it finds any:
if exists (
SELECT cpu_time, *
FROM sys.dm_exec_requests req
CROSS APPLY sys.dm_exec_sql_text(sql_handle) AS sqltext
--order by req.total_elapsed_time desc
WHERE ([text] LIKE N'%your troublesome SP or function name etc%')
and cpu_time > 8000
)
begin
SELECT cp.plan_handle, st.[text]
into #results
FROM sys.dm_exec_cached_plans AS cp
CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS st
WHERE [text] LIKE N'%your troublesome SP or function name etc%'
delete #results where text like 'SELECT cp.plan_handle%'
--select * from #results
declare #handle varbinary(max)
declare #handleconverted varchar(max)
declare #sql varchar(1000)
DECLARE db_cursor CURSOR FOR
select plan_handle from #results
OPEN db_cursor
FETCH NEXT FROM db_cursor INTO #handle
WHILE ##FETCH_STATUS = 0
BEGIN
--e.g. DBCC FREEPROCCACHE (0x050006003FCA862F40A19A93010000000000000000000000)
print #handle
set #handleconverted = '0x' + CAST('' AS XML).value('xs:hexBinary(sql:variable("#handle"))', 'VARCHAR(MAX)')
print #handleconverted
set #sql = 'DBCC FREEPROCCACHE (' + #handleconverted + ')'
print 'DELETING: ' + #sql
EXEC(#sql)
FETCH NEXT FROM db_cursor INTO #handle
END
CLOSE db_cursor
DEALLOCATE db_cursor
drop table #results
end
Simply recompiling the stored procedure (table function in my case) worked for me
like #Zane said it could be due to parameter sniffing. I experienced the same behaviour and I took a look at the execution plan of the procedure and all the statements of the sp in a row (copied all the statements form the procedure, declared the parameters as variables and asigned the same values for the variable as the parameters had). However the execution plan looked completely different. The sp execution took 3-4 seconds and the statements in a row with the exact same values was instantly returned.
After some googling I found an interesting read about that behaviour: Slow in the Application, Fast in SSMS?
When compiling the procedure, SQL Server does not know that the value of #fromdate changes, but compiles the procedure under the assumption that #fromdate has the value NULL. Since all comparisons with NULL yield UNKNOWN, the query cannot return any rows at all, if #fromdate still has this value at run-time. If SQL Server would take the input value as the final truth, it could construct a plan with only a Constant Scan that does not access the table at all (run the query SELECT * FROM Orders WHERE OrderDate > NULL to see an example of this). But SQL Server must generate a plan which returns the correct result no matter what value #fromdate has at run-time. On the other hand, there is no obligation to build a plan which is the best for all values. Thus, since the assumption is that no rows will be returned, SQL Server settles for the Index Seek.
The problem was that I had parameters which could be left null and if they were passed as null the would be initialised with a default value.
create procedure dbo.procedure
#dateTo datetime = null
begin
if (#dateTo is null)
begin
select #dateTo = GETUTCDATE()
end
select foo
from dbo.table
where createdDate < #dateTo
end
After I changed it to
create procedure dbo.procedure
#dateTo datetime = null
begin
declare #to datetime = coalesce(#dateTo, getutcdate())
select foo
from dbo.table
where createdDate < #to
end
it worked like a charm again.
--BEFORE
CREATE PROCEDURE [dbo].[SP_DEMO]
(
#ToUserId bigint=null
)
AS
BEGIN
SELECT * FROM tbl_Logins WHERE LoginId = #ToUserId
END
--AFTER CHANGING TO IT WORKING FINE
CREATE PROCEDURE [dbo].[SP_DEMO]
(
#ToUserId bigint=null
)
AS
BEGIN
DECLARE #Toid bigint=null
SET #Toid=#ToUserId
SELECT * FROM tbl_Logins WHERE LoginId = #Toid
END