I'm working with the Microsoft Parallel Data Warehouse appliance and attempting to use flyway to handle table migrations in that environment. The issue I'm running into is that the default script for establishing the schema_version table fails.
Here is the default script as far as I can tell that is being executed upon calling baseline().
CREATE TABLE [dbo].[dbresult_migration] (
[installed_rank] INT NOT NULL,
[version] NVARCHAR(50),
[description] NVARCHAR(200),
[type] NVARCHAR(20) NOT NULL,
[script] NVARCHAR(1000) NOT NULL,
[checksum] INT,
[installed_by] NVARCHAR(100) NOT NULL,
[installed_on] DATETIME NOT NULL DEFAULT GETDATE(),
[execution_time] INT NOT NULL,
[success] BIT NOT NULL
);
ALTER TABLE [dbo].[dbresult_migration] ADD CONSTRAINT [dbresult_migration_pk] PRIMARY KEY ([installed_rank]);
CREATE INDEX [dbresult_migration_s_idx] ON [dbo].[dbresult_migration] ([success]);
Specifically the Microsoft Parallel Data Warehouse (MS PDW or APS as it is now known) doesn't support expressions with default constraints.
Msg 104338, Level 16, State 1, Line 1
An expression cannot be used with a default constraint. Specify only constants for a default constraint.
Which causes an error when GETDATE() is used as the default for the installed_on column.
The ALTER TABLE statement will also fail as PRIMARY KEYs and INDICES are managed differently in the environment.
Is there a way to override the default initialization script for the schema_version?
UPDATE
Further investigation reveals that the next failure occurs when attempting to insert records into the schema_version table. Specifically the current implementation attempts to identify the current user based on a call to dbSupport.getCurrentUserFunction(). For SQL Server this function is SUSER_SNAME(). While this function is available on both the standard SQL Server and the Parallel Data Warehouse the current implementation of the Parallel Data Warehouse does not allow for function calls within the values portion of an insert statement. As such, the following error is returned:
Insert values statement can contain only constant literal values or variable references.
When the query that is attempted is logged as:
INSERT INTO [dbo].[dbresult_migration] ([installed_rank],[version],[description],[type],[script],[checksum],[installed_by],[execution_time],[success]) VALUES (#P0, #P1, #P2, #P3, #P4, #P5, SUSER_SNAME(), #P6, #P7)
UPDATE 2
I now have a fork of flyway-core that correctly identifies if you are connecting to SQL Server vs SQL Server parallel data warehouse. Another issue that I have identified is that SQL Server PDW does not allow DDL within transactions and so an attempt to baseline fails as this appears to be attempted from within a transaction template. Ultimately this is evolving from a question of understanding how to modify an initialization script to a need for support of a new database platform. I've submitted this as a new issue on the flyway repo on github here.
Related
I wanted to 'Call' MariaDB Procedure from Azure Data Factory.
How can this be achieved, are there any other service which can be integrated with ADF to call this MariaDB procedures
I tried calling the procedure by writing the query using lookup activity.
It fails while showing this error.
ErrorCode=InvalidParameter,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The value of the property 'columns' is invalid: 'Value cannot be null.
Parameter name: columns'.,Source=,''Type=System.ArgumentNullException,Message=Value cannot be null.
Parameter name: columns,Source=Microsoft.DataTransfer.Common,'
Lookup activity reads and returns the content of the query. I tried to repro this by creating three stored procedures in Azure SQL database for Maria DB.
First Stored procedure is written to update the data in the table.
DELIMITER $$
CREATE PROCEDURE update_inventory()
BEGIN
UPDATE inventory SET quantity = 150
WHERE id = 1;
END$$
DELIMITER ;
When this procedure is called in ADF lookup activity, error occurs.
Second stored procedure is written with select query.
DELIMITER $$
CREATE PROCEDURE select_inventory()
BEGIN
select * from inventory;
END$$
DELIMITER ;
When this SP is called, ADF pipeline is executed successfully.
In order to execute the stored procedure with update statements (or any statements), a select statement is added in the Stored procedure.
DELIMITER $$
CREATE PROCEDURE update_select_inventory()
BEGIN
UPDATE inventory SET quantity = 150
WHERE id = 1;
select * from inventory;
END$$
DELIMITER ;
When this stored procedure is called through Lookup activity, it got executed successfully.
Try adding select statement in the stored procedure and execute it in Lookup activity. Or add Select statement after Call stored procedure statement.
By selecting the 'query' option, you can call the stored procedure using lookup activity. From your error message, it looks like you are missing the parameter columns while calling the stored procedure.
Did you try executing the same code using the client tools like MySQL workbench? If you can execute the stored proc from other client tools, then you should be able to execute the same using the lookup activity.
I tested from my end and was able to execute the Stored procedure using lookup activity. Please see the below screenshot for your reference.
I am trying to change the type of a column from VARCHAR to INT.
During the deployment, the database project will stop the deployment due to the "data loss" error:
RAISERROR (N'Rows were detected. The schema update is terminating because data loss might occur.', 16, 127)
I know the data is convertible and if I run a manual ALTER TABLE script it will be fine. However, I cannot integrate that properly with this scenario to avoid the error during the deployment.
What is your solution to resolve my problem?
Is there a method
to override this behaviour in a database project and for this
particular case, use a custom script?
One way in such scenario is using PreDeployment script and deploy twice.
Change data type column in table definition as usual
Add in Predeploy script:
-- this script has to be idempotent, and removed after some time
IF EXISTS (SELECT 1
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'table_name'
AND TABLE_SCHEMA = 'schema_name'
AND COLUMN_NAME = 'column_name
AND DATA_TYPE != 'INT'
)
BEGIN
ALTER TABLE schema_name.table_name ALTER COLUMN Column_name INT NULL/NOT NULL;
END
First publish will change the data type during PreDeploy, and deploy will fail with Potential Data loss error.
Second publish will omit the part of PreDeploy(if condition), and schema compare does not detect any changes, meaning it has been changed.
Next step should be removing the manual part from PreDeployment script.
I'm new to sqlite and sql in gerneral so I don't know if my approach is reasonable.
I want to model inventory items that can be created, lent, returned and discarded.
I want to model this using two tables, one for items, containing an id, a name and a reference to the last transaction (created, lent, returned, ...) and a table of transactions containing an id transaction type, date, and a reference to the item.
Since creating only one table leaves the database in an inconsitent state with one table referencing a non existant table I thought of using a transaction to crate both tables at once, and defining the foreign keys as deferrable. Creation of a new item would have to be done together in one transaction with creating a "created" event to leave the database in a consistent state.
The following code gives me the error Query Error: not an error Unable to execute multiple statements at a time in sqliteman on linux.
PRAGMA foreign_keys = ON;
begin transaction;
create table items (
id integer primary key,
name char(30),
foreign key (last_transaction) references transactions(transaction_id) DEFERRABLE INITIALLY DEFERRED
);
create table transactions(
transaction_id integer primary key,
text char(100)
foreign key (item) references items(id) DEFERRABLE INITIALLY DEFERRED
);
commit transaction;
Does my approach make sense at all?
If yes, why does the code not work? (Did I make a mistake somewhere, or is what I'm trying impossible in mysql?)
Note: simply creating the tables in one transaction without the foreign key constraints gives the same error. (Could this be a similar Problem to: this question)
I have an ASP.NET web application that populates the SQL Server 2008 database table like this:
INSERT INTO tblName1 (col1, col2, col3)
VALUES(1, 2, 3)
I also have a separate service application that processes the contents of that table (on the background) by first renaming that table, and then by creating an empty table as such:
SET XACT_ABORT ON
BEGIN TRANSACTION
--Rename table
EXEC sp_rename 'tblName1', 'temp_tblName1'
--Create new table
CREATE TABLE tblName1(
id INT NOT NULL IDENTITY(1,1) PRIMARY KEY,
col1 INT,
col2 INT,
col3 INT
)
COMMIT
SET XACT_ABORT OFF
--Begin working with the 'temp_tblName1' table
What I am not sure is which SQL lock do I need to use in this situation on the tblName1 table?
PS. To give you a frequency with which these two code samples run: first may run several times a second (although most times, less frequently), and the second one -- twice a day.
As some of the comments have suggested, consider doing this differently. You may benefit from using the snapshot isolation level. Using snapshot isolation requires ALLOW_SNAPSHOT_ISOLATION to be set to ON on the database. This setting is off by default, so you'll want to check whether you can turn it on.
Once you are able to use snapshot isolation, you would not need to change your INSERT statement, but your other process could change to something like:
SET XACT_ABORT ON
SET TRANSACTION ISOLATION LEVEL SNAPSHOT
BEGIN TRANSACTION
-- Do whatever this process does, but don't rename the table.
-- If you want to get rid of the old records:
DELETE [tblName1] WHERE 1 = 1
-- Then
COMMIT TRANSACTION
In case you really do need to create a new non-temporary table for some reason, you may need to do so before entering the transaction, as there are some limits on what you are allowed to do during snapshot isolation.
I wrote some sql scripts to create a database and store data. I just noticed that the new tables and data are going to the master database.
I found that I can address the correct database if I scope out the database as so:
CREATE TABLE Practice1.dbo.Experiments
(
ID int IDENTITY (100,1) PRIMARY KEY,
CompanyName nvarchar (50)
)
but I'd rather not have to scope out each command. Is there a way to set the database in the script so I don't have to scope everything out?
INSERT INTO Practice1.dbo.EXPERIMENTS
VALUES
(
'hello world'
)
SELECT * FROM Practice1.dbo.EXPERIMENTS
You have a drop down list on your toolbar that allows you to select what database you want the script to execute on. Also, you can state the database to use at the top of your script.
Example:
Syntax
USE {database}
http://doc.ddart.net/mssql/sql70/ua-uz_7.htm
On SQL Server 2005, to switch the database context, use the command:
USE DatabaseName
in the samples above, the database name is Practice1, hence:
USE Practice1