I am trying to change the type of a column from VARCHAR to INT.
During the deployment, the database project will stop the deployment due to the "data loss" error:
RAISERROR (N'Rows were detected. The schema update is terminating because data loss might occur.', 16, 127)
I know the data is convertible and if I run a manual ALTER TABLE script it will be fine. However, I cannot integrate that properly with this scenario to avoid the error during the deployment.
What is your solution to resolve my problem?
Is there a method
to override this behaviour in a database project and for this
particular case, use a custom script?
One way in such scenario is using PreDeployment script and deploy twice.
Change data type column in table definition as usual
Add in Predeploy script:
-- this script has to be idempotent, and removed after some time
IF EXISTS (SELECT 1
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'table_name'
AND TABLE_SCHEMA = 'schema_name'
AND COLUMN_NAME = 'column_name
AND DATA_TYPE != 'INT'
)
BEGIN
ALTER TABLE schema_name.table_name ALTER COLUMN Column_name INT NULL/NOT NULL;
END
First publish will change the data type during PreDeploy, and deploy will fail with Potential Data loss error.
Second publish will omit the part of PreDeploy(if condition), and schema compare does not detect any changes, meaning it has been changed.
Next step should be removing the manual part from PreDeployment script.
Related
I am creating a book tracking database for myself that holds information about my books and allows me to keep track of who is borrowing them. I am trying to create a trigger on my Checkouts table that runs if a record is added or updated that will determine if a checkout data has been entered or if a checkin date has been entered and change the "available" field in my Books table to "Y" or "N".
I have created a trigger called "update_book_availablility" on my Checkouts table but I keep getting this error:
"PLS-00103: Encountered the symbol 'end-of-file' when expecting one of the following: ( begin case declare and exception exit for goto if loop mod null pragma raise return select update while with <<continue close current delete fetch lock insert open rollback savepoint set sql execute commit forall merge standard pipe purge json_object
Errors: check compiler log"
Here is my trigger code:
CREATE OR REPLACE NONEDITIONABLE TRIGGER "UPDATE_BOOK_AVAILABILITY"
AFTER INSERT OR UPDATE OF ISBN, PersonID, checkout_date, checkin_date
ON Checkouts
FOR EACH ROW
BEGIN
IF :NEW.checkout_date = NULL
THEN
UPDATE Book
SET available = 'N'
WHERE ISBN IN (SELECT :NEW.ISBN FROM Checkouts);
END IF;
END;
Here is an image of my ERD:
ERD
I have been looking into and double checking my trigger syntax, If condition syntax, subquery syntax, and googling this error but have found nothing that has helped. I am new to PL/SQL and would appreciate any help in understanding what I have done wrong or missed.
PLS-00103: Encountered the symbol end-of-file error is SYNTAX ERROR
Copied your trigger and adjusted it to one of my test tables - it works. I removed NONEDITIONABLE and changed trigger table name as well as column names and table/column beeing updated by trigger.
To Do:
Check your syntax again or write the trigger from scratch once more
"...WHERE ISBN IN (SELECT :NEW.ISBN FROM Checkouts)..." selects one fixed value (FOR EACH ROW) :NEW.ISBN of triggering table, better ->> "... WHERE ISBN = :NEW.ISBN ..."
Prety sure that you don't need NONEDITIONABLE trigger for your books tracking app...
Regards...
I am planning to use flywayDb to automate SQL script migration part of my project for the same purpose i tried to test it but it failed one one SQL script saying some Invalid Character error which I am unable to identify where as same script is working fine inside AQT/SQL Developer.
Do we have any restrictions or standards need to follow while using command line migrate?
I have doubt on some values which are using single quote and slash symbols... like below
'Family Member''s' OR SOME \ TEXT
Please suggest.
I was able to find issue problem was with one merge statement in which field value and field name was not having space
hence flyway DB was throwing error...I added one space and it worked. Where as without removing space works fine in AQT and sqldeveloper
MERGE INTO TEST.question ques USING
(SELECT '2004'question_id,'Details (Header)' question_text FROM DUAL) S
ON (ques.Question_id = S.Question_id)
WHEN MATCHED THEN UPDATE
SET
ques.text=S.text;
Correct one
MERGE INTO TEST.question ques USING
(SELECT '2004' question_id,'Details (Header)' question_text FROM DUAL) S
ON (ques.Question_id = S.Question_id)
WHEN MATCHED THEN UPDATE
SET
ques.text=S.text;
I'm trying to find a way to have a database object change based on the publish profile used, or a sqlcmd variable. Specifically, if I deploy the database project to Server A & B; on Server A I want one version of a stored procedure, on Server B I want a different version of the stored procedure. For dependency reasons the stored procedure must have the same name on both servers. I know of a way to do this using post deployment scripts and dynamic sql. I am looking for a better option.
I am not sure that it is actual for question starter, however it might help someone. You can do following:
Create variable, for example $(DeployType) and set it to unique
value in all publishing configs
Alter your database with following code:
.
CREATE PROCEDURE p_test
AS
BEGIN
IF '$(DeployType)' = 'Production'
BEGIN
/*Your code for production database*/
END
IF '$(DeployType)' = 'Dev'
BEGIN
/*Your code for devdatabase*/
END
END
In that case it supposed to be exactly what you want. With views/functions you can do something like:
CREATE VIEW v_test AS
SELECT 1
FROM table
WHERE '$(DeployType)' = 'Production'
UNION ALL
SELECT 1
FROM table2
WHERE '$(DeployType)' = 'Dev'
The performance is not supposed to decrease as optimizer checks WHERE condition and see that 1 part of union will allways be NULL so it does not execute it.
I have stored procedure that insanely times out every single time it's called from the web application.
I fired up the Sql Profiler and traced the calls that time out and finally found out these things:
When executed the statements from within the MS SQL Management Studio, with same arguments (in fact, I copied the procedure call from sql profile trace and ran it): It finishes in 5~6 seconds avg.
But when called from web application, it takes in excess of 30 seconds (in trace) so my webpage actually times out by then.
Apart from the fact that my web application has its own user, every thing is same (same database, connection, server etc)
I also tried running the query directly in the studio with the web application's user and it doesn't take more than 6 sec.
How do I find out what is happening?
I am assuming it has nothing to do with the fact that we use BLL > DAL layers or Table adapters as the trace clearly shows the delay is in the actual procedure. That is all I can think of.
EDIT I found out in this link that ADO.NET sets ARITHABORT to true - which is good for most of the time but sometime this happens, and the suggested work-around is to add with recompile option to the stored proc. In my case, it's not working but I suspect it's something very similar to this. Anyone knows what else ADO.NET does or where I can find the spec?
I've had a similar issue arise in the past, so I'm eager to see a resolution to this question. Aaron Bertrand's comment on the OP led to Query times out when executed from web, but super-fast when executed from SSMS, and while the question is not a duplicate, the answer may very well apply to your situation.
In essence, it sounds like SQL Server may have a corrupt cached execution plan. You're hitting the bad plan with your web server, but SSMS lands on a different plan since there is a different setting on the ARITHABORT flag (which would otherwise have no impact on your particular query/stored proc).
See ADO.NET calling T-SQL Stored Procedure causes a SqlTimeoutException for another example, with a more complete explanation and resolution.
I also experience that queries were running slowly from the web and fast in SSMS and I eventually found out that the problem was something called parameter sniffing.
The fix for me was to change all the parameters that are used in the sproc to local variables.
eg. change:
ALTER PROCEDURE [dbo].[sproc]
#param1 int,
AS
SELECT * FROM Table WHERE ID = #param1
to:
ALTER PROCEDURE [dbo].[sproc]
#param1 int,
AS
DECLARE #param1a int
SET #param1a = #param1
SELECT * FROM Table WHERE ID = #param1a
Seems strange, but it fixed my problem.
Not to spam, but as a hopefully helpful solution for others, our system saw a high degree of timeouts.
I tried setting the stored procedure to be recompiled by using sp_recompile and this resolved the issue for the one SP.
Ultimately there were a larger number of SP's that were timing-out, many of which had never done so before, by using DBCC DROPCLEANBUFFERS and DBCC FREEPROCCACHE the incident rate of timeouts has plummeted significantly - there are still isolated occurrences, some where I suspect the plan regeneration is taking a while, and some where the SPs are genuinely under-performant and need re-evaluation.
Could it be that some other DB calls made before the web application calls the SP is keeping a transaction open? That could be a reason for this SP to wait when called by the web application. I say isolate the call in the web application (put it on a new page) to ensure that some prior action in the web application is causing this issue.
You can target specific cached execution plans via:
SELECT cp.plan_handle, st.[text]
FROM sys.dm_exec_cached_plans AS cp
CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS st
WHERE [text] LIKE N'%your troublesome SP or function name etc%'
And then remove only the execution plans causing issues via, for example:
DBCC FREEPROCCACHE (0x050006003FCA862F40A19A93010000000000000000000000)
I've now got a job running every 5 minutes that looks for slow running procedures or functions and automatically clears down those execution plans if it finds any:
if exists (
SELECT cpu_time, *
FROM sys.dm_exec_requests req
CROSS APPLY sys.dm_exec_sql_text(sql_handle) AS sqltext
--order by req.total_elapsed_time desc
WHERE ([text] LIKE N'%your troublesome SP or function name etc%')
and cpu_time > 8000
)
begin
SELECT cp.plan_handle, st.[text]
into #results
FROM sys.dm_exec_cached_plans AS cp
CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS st
WHERE [text] LIKE N'%your troublesome SP or function name etc%'
delete #results where text like 'SELECT cp.plan_handle%'
--select * from #results
declare #handle varbinary(max)
declare #handleconverted varchar(max)
declare #sql varchar(1000)
DECLARE db_cursor CURSOR FOR
select plan_handle from #results
OPEN db_cursor
FETCH NEXT FROM db_cursor INTO #handle
WHILE ##FETCH_STATUS = 0
BEGIN
--e.g. DBCC FREEPROCCACHE (0x050006003FCA862F40A19A93010000000000000000000000)
print #handle
set #handleconverted = '0x' + CAST('' AS XML).value('xs:hexBinary(sql:variable("#handle"))', 'VARCHAR(MAX)')
print #handleconverted
set #sql = 'DBCC FREEPROCCACHE (' + #handleconverted + ')'
print 'DELETING: ' + #sql
EXEC(#sql)
FETCH NEXT FROM db_cursor INTO #handle
END
CLOSE db_cursor
DEALLOCATE db_cursor
drop table #results
end
Simply recompiling the stored procedure (table function in my case) worked for me
like #Zane said it could be due to parameter sniffing. I experienced the same behaviour and I took a look at the execution plan of the procedure and all the statements of the sp in a row (copied all the statements form the procedure, declared the parameters as variables and asigned the same values for the variable as the parameters had). However the execution plan looked completely different. The sp execution took 3-4 seconds and the statements in a row with the exact same values was instantly returned.
After some googling I found an interesting read about that behaviour: Slow in the Application, Fast in SSMS?
When compiling the procedure, SQL Server does not know that the value of #fromdate changes, but compiles the procedure under the assumption that #fromdate has the value NULL. Since all comparisons with NULL yield UNKNOWN, the query cannot return any rows at all, if #fromdate still has this value at run-time. If SQL Server would take the input value as the final truth, it could construct a plan with only a Constant Scan that does not access the table at all (run the query SELECT * FROM Orders WHERE OrderDate > NULL to see an example of this). But SQL Server must generate a plan which returns the correct result no matter what value #fromdate has at run-time. On the other hand, there is no obligation to build a plan which is the best for all values. Thus, since the assumption is that no rows will be returned, SQL Server settles for the Index Seek.
The problem was that I had parameters which could be left null and if they were passed as null the would be initialised with a default value.
create procedure dbo.procedure
#dateTo datetime = null
begin
if (#dateTo is null)
begin
select #dateTo = GETUTCDATE()
end
select foo
from dbo.table
where createdDate < #dateTo
end
After I changed it to
create procedure dbo.procedure
#dateTo datetime = null
begin
declare #to datetime = coalesce(#dateTo, getutcdate())
select foo
from dbo.table
where createdDate < #to
end
it worked like a charm again.
--BEFORE
CREATE PROCEDURE [dbo].[SP_DEMO]
(
#ToUserId bigint=null
)
AS
BEGIN
SELECT * FROM tbl_Logins WHERE LoginId = #ToUserId
END
--AFTER CHANGING TO IT WORKING FINE
CREATE PROCEDURE [dbo].[SP_DEMO]
(
#ToUserId bigint=null
)
AS
BEGIN
DECLARE #Toid bigint=null
SET #Toid=#ToUserId
SELECT * FROM tbl_Logins WHERE LoginId = #Toid
END
I would like to find out if it is possible to find out which package or procedure in a package is updating a table?
Due to a certain project being handed over (the person who handed over the project has since left) without proper documentation, data that we know we have updated always go back to some strange source point.
We are guessing that this could be a database job or scheduler that is running the update command without our knowledge. I am hoping that there is a way to find out where the source code is calling from that is updating the table and inserting the source as a trigger on that table that we are monitoring.
Any ideas?
Thanks.
UPDATE: I poked around and found out
how to trace a statement back to its
owning PL/SQL object.
In combination with what Tony mentioned, you can create a logging table and a trigger that looks like this:
CREATE TABLE statement_tracker
( SID NUMBER
, serial# NUMBER
, date_run DATE
, program VARCHAR2(48) null
, module VARCHAR2(48) null
, machine VARCHAR2(64) null
, osuser VARCHAR2(30) null
, sql_text CLOB null
, program_id number
);
CREATE OR REPLACE TRIGGER smb_t_t
AFTER UPDATE
ON smb_test
BEGIN
INSERT
INTO statement_tracker
SELECT ss.SID
, ss.serial#
, sysdate
, ss.program
, ss.module
, ss.machine
, ss.osuser
, sq.sql_fulltext
, sq.program_id
FROM v$session ss
, v$sql sq
WHERE ss.sql_address = sq.address
AND ss.SID = USERENV('sid');
END;
/
In order for the trigger above to compile, you'll need to grant the owner of the trigger these permissions, when logged in as the SYS user:
grant select on V_$SESSION to <user>;
grant select on V_$SQL to <user>;
You will likely want to protect the insert statement in the trigger with some condition that only makes it log when the the change you're interested in is occurring - on my test server this statement runs rather slowly (1 second), so I wouldn't want to be logging all these updates. Of course, in that case, you'd need to change the trigger to be a row-level one so that you could inspect the :new or :old values. If you are really concerned about the overhead of the select, you can change it to not join against v$sql, and instead just save the SQL_ADDRESS column, then schedule a job with DBMS_JOB to go off and update the sql_text column with a second update statement, thereby offloading the update into another session and not blocking your original update.
Unfortunately, this will only tell you half the story. The statement you're going to see logged is going to be the most proximal statement - in this case, an update - even if the original statement executed by the process that initiated it is a stored procedure. This is where the program_id column comes in. If the update statement is part of a procedure or trigger, program_id will point to the object_id of the code in question - you can resolve it thusly:
SELECT * FROM all_objects where object_id = <program_id>;
In the case when the update statement was executed directly from the client, I don't know what program_id represents, but you wouldn't need it - you'd have the name of the executable in the "program" column of statement_tracker. If the update was executed from an anonymous PL/SQL block, I'm not how to track it back - you'll need to experiment further.
It may be, though, that the osuser/machine/program/module information may be enough to get you pointed in the right direction.
If it is a scheduled database job then you can find out what scheduled database jobs exist and look into what they do. Other things you can do are:
look at the dependencies views e.g. ALL_DEPENDENCIES to see what packages/triggers etc. use that table. Depending on the size of your system that may return a lot of objects to trawl through.
Search all the database source code for references to the table like this:
select distinct type, name
from all_source
where lower(text) like lower('%mytable%');
Again that may return a lot of objects, and of course there will be some "false positives" where the search string appears but isn't actually a reference to that table. You could even try something more specific like:
select distinct type, name
from all_source
where lower(text) like lower('%insert into mytable%');
but of course that would miss cases where the command was formatted differently.
Additionally, could there be SQL scripts being run through "cron" jobs on the server?
Just write an "after update" trigger and, in this trigger, log the results of "DBMS_UTILITY.FORMAT_CALL_STACK" in a dedicated table.
The purpose of this function is exactly to give you the complete call stack of al the stored procedures and triggers that have been fired to reach your code.
I am writing from the mobile app, so i can't give you more detailed examples, but if you google for it you'll find many of them.
A quick and dirty option if you're working locally, and are only interested in the first thing that's altering the data, is to throw an error in the trigger instead of logging. That way, you get the usual stack trace and it's a lot less typing and you don't need to create a new table:
AFTER UPDATE ON table_of_interest
BEGIN
RAISE_APPLICATION_ERROR(-20001, 'something changed it');
END;
/