Does FreeTDS ODBC + pyodbc support table-valued parameters (TVPs)? - pyodbc

I'm working on unix/rhel7 system. I have installed require drivers for FreeTDS, unixODBC and pyodbc.Other query is working fine but when I'm trying execute stored proc with TVP (table valued parameter), its giving me error. Is there any way to connect SQL Server using windows service account from python?
Example:
import pyodbc;
cnxn = pyodbc.connect('DRIVER=FreeTDS;SERVER=SERVERNAME;PORT=1234;UID=USERNAME;PWD=PASSWORD;DATABASE=DBNAME')
cnxn.cursor()
param_array = []
for i in range(3):
param_array.append(['abc', 'adi', '/somepath/', '2021-01-04', 'NEW'])
result_array = cursor.execute("EXEC abc.stored_proc_name ?", [param_array]).fetchall()
cursor.commit()
cnxn.close()
Error:
pyodbc.Error: ('HY004', '[HY004] [FreeTDS][SQL Server]Invalid data type (0) (SQLBindParameter)')
So Is there any other way to connect SQL service account from python which supports TVP? Or Is there any solution in above example?

FreeTDS ODBC does not directly support table-valued parameters (TVPs) as discussed here. However we can use a temporary table and an anonymous code block to work around the issue. For a user-defined table type
USE [myDb]
GO
/****** Object: UserDefinedTableType [dbo].[dboListInt] Script Date: 2021-02-18 10:53:17 ******/
CREATE TYPE [dbo].[dboListInt] AS TABLE(
[Id] [int] NOT NULL,
PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (IGNORE_DUP_KEY = OFF)
)
GO
and a stored procedure that accepts that table type
USE [myDb]
GO
/****** Object: StoredProcedure [dbo].[dboPyOdbcTestTvp] Script Date: 2021-02-18 10:41:43 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[dboPyOdbcTestTvp](#tvp [dbo].dboListInt READONLY)
AS
BEGIN
SET NOCOUNT ON;
SELECT * FROM #tvp
END
we can call the stored procedure and retrieve the results like so:
import pyodbc
cnxn = pyodbc.connect(
"DRIVER=FreeTDS_1.2.18;"
"SERVER=192.168.0.179;"
"PORT=49242;"
"DATABASE=myDb;"
"UID=sa;PWD=_whatever_;"
)
crsr = cnxn.cursor()
crsr.execute("CREATE TABLE #tvp_data (Id int)")
tvp_data = [(123, ), (234, ), (345, )]
crsr.executemany(
"INSERT INTO #tvp_data (Id) VALUES (?)",
tvp_data
)
crsr.execute("""\
SET NOCOUNT ON;
DECLARE #tvp dbo.dboListInt;
INSERT INTO #tvp (Id)
SELECT Id FROM #tvp_data;
EXEC dbo.dboPyOdbcTestTvp #tvp;
""")
print(crsr.fetchall())
# [(123, ), (234, ), (345, )]

Related

.sql database import not working. (stored procedure error)

I am new to MySQL.
I want to import the database from .sql file.
Tables are imported successfully.
but it gives the below error regarding procedures :
CREATE DEFINER=`dev`#`%` PROCEDURE `sp_PlaceBet`(
IN `pLogInId` INT,
IN `pUserId` INT,
IN `pParantId` INT,
IN `pMatchId` INT,
IN `pSelectionId` INT,
IN `pStack` INT,
IN `pMarketId` VARCHAR(100),
IN `pselectionName` VARCHAR(100),
IN `pMstDate` DATETIME,
IN `pOdds` DECIMAL(10, 2),
IN `pP_L` DECIMAL(10, 2),
IN `pisBack` INT,
IN `pIsMatched` INT,
IN `pNarration` VARCHAR(200),
IN `pdeviceInfo` VARCHAR(100),
IN `pIP_ADDESSS` VARCHAR(100),
IN `pInPlayStack` INT,
IN `pIsApp` INT,
IN `pType` VARCHAR(100)
)
BEGIN
DECLARE LID INTEGER;
DECLARE lCtr integer;
DECLARE resultV INT;
DECLARE retMess VARCHAR(500);
DECLARE checkBal decimal(50, 2);
DECLARE EXIT HANDLER FOR SQLEXCEPTION
BEGIN
set resultV = -1;
GET STACKED DIAGNOSTICS CONDITION 1 #sqlstate = RETURNED_SQLSTATE, #errno = MYSQL_ERRNO, #text = MESSAGE_TEXT;
SET retMess = CONCAT("ERROR ", #errno, " (", #sqlstate, "): ", #text);
ROLLBACK;
[...]
MySQL said: Documentation
#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'STACKED DIAGNOSTICS CONDITION 1 #sqlstate = RETURNED_SQLSTATE, #errno = MYSQL...' at line 31
Please help me to solve this error.
Thanks!
The problem is that you are using mariadb, not mysql. If you check out the error message, it tells you to fefer to the mariadb manual.
Mariadb's get diagnostics statement does not have stacked option, so you must remove it if you want to use the code in a mariadb environment or you need to migrate to a proper mysql server.

Using sqlcmd to execute deploy scripts generated by DAC in a transaction

We have a DACPAC (sqlproj) solution which has some tables and a post-deployment script which runs some DML queries.
If the DML query fails (I'm raising an error with severity=20), I would like to rollback all changes - including the DDL changes done by the dacpac and the post-deployment file changes. This is especially useful when I would be upgrading an existing target database.
I'm striving for an atomic DB upgrade when the DACPAC is published - all DDL changes mentioned in the DACPAC solution should be published only when everything in the post deployment script is successful.
Since DACPAC DDL changes are committed before it invokes post-deployment script, I thought generating all the DAC changes as a single script file using DacServices.GenerateDeployScript will help. Doesnt look so straight forward.
Has anyone tried something like this (and failed/passed)?
I'm facing many challenges like...
Create/alter Database shouldnt be in a transaction.
Rollback not happening at all.
[Edit 10Nov]: Pasting the deploy script generated by the dacpac here, so that I can explain my issue better (hopefully)
/*
Deployment script for 9Nov
This code was generated by a tool.
Changes to this file may cause incorrect behavior and will be lost if
the code is regenerated.
*/
GO
SET ANSI_NULLS, ANSI_PADDING, ANSI_WARNINGS, ARITHABORT, CONCAT_NULL_YIELDS_NULL, QUOTED_IDENTIFIER ON;
SET NUMERIC_ROUNDABORT OFF;
GO
:setvar INSTALL_DIR "D:\EDW_9Nov\"
:setvar DatabaseName "9Nov"
:setvar DefaultFilePrefix "9Nov"
:setvar DefaultDataPath "C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\DATA\"
:setvar DefaultLogPath "C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\DATA\"
GO
:on error exit
GO
/*
Detect SQLCMD mode and disable script execution if SQLCMD mode is not supported.
To re-enable the script after enabling SQLCMD mode, execute the following:
SET NOEXEC OFF;
*/
:setvar __IsSqlCmdEnabled "True"
GO
IF N'$(__IsSqlCmdEnabled)' NOT LIKE N'True'
BEGIN
PRINT N'SQLCMD mode must be enabled to successfully execute this script.';
SET NOEXEC ON;
END
GO
USE [master];
GO
IF (DB_ID(N'$(DatabaseName)') IS NOT NULL)
BEGIN
ALTER DATABASE [$(DatabaseName)]
SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
DROP DATABASE [$(DatabaseName)];
END
GO
PRINT N'Creating $(DatabaseName)...'
GO
CREATE DATABASE [$(DatabaseName)]
ON
PRIMARY(NAME = [$(DatabaseName)], FILENAME = N'$(DefaultDataPath)$(DefaultFilePrefix)_Primary.mdf')
LOG ON (NAME = [$(DatabaseName)_log], FILENAME = N'$(DefaultLogPath)$(DefaultFilePrefix)_Primary.ldf') COLLATE SQL_Latin1_General_CP1_CI_AS
GO
IF EXISTS (SELECT 1
FROM [master].[dbo].[sysdatabases]
WHERE [name] = N'$(DatabaseName)')
BEGIN
ALTER DATABASE [$(DatabaseName)]
SET ANSI_NULLS ON,
ANSI_PADDING ON,
ANSI_WARNINGS ON,
ARITHABORT ON,
CONCAT_NULL_YIELDS_NULL ON,
NUMERIC_ROUNDABORT OFF,
QUOTED_IDENTIFIER ON,
ANSI_NULL_DEFAULT ON,
CURSOR_DEFAULT LOCAL,
RECOVERY SIMPLE,
CURSOR_CLOSE_ON_COMMIT OFF,
AUTO_CREATE_STATISTICS ON,
AUTO_SHRINK OFF,
AUTO_UPDATE_STATISTICS ON,
RECURSIVE_TRIGGERS OFF
WITH ROLLBACK IMMEDIATE;
ALTER DATABASE [$(DatabaseName)]
SET AUTO_CLOSE OFF
WITH ROLLBACK IMMEDIATE;
END
GO
IF EXISTS (SELECT 1
FROM [master].[dbo].[sysdatabases]
WHERE [name] = N'$(DatabaseName)')
BEGIN
ALTER DATABASE [$(DatabaseName)]
SET ALLOW_SNAPSHOT_ISOLATION OFF;
END
GO
IF EXISTS (SELECT 1
FROM [master].[dbo].[sysdatabases]
WHERE [name] = N'$(DatabaseName)')
BEGIN
ALTER DATABASE [$(DatabaseName)]
SET READ_COMMITTED_SNAPSHOT OFF
WITH ROLLBACK IMMEDIATE;
END
GO
IF EXISTS (SELECT 1
FROM [master].[dbo].[sysdatabases]
WHERE [name] = N'$(DatabaseName)')
BEGIN
ALTER DATABASE [$(DatabaseName)]
SET AUTO_UPDATE_STATISTICS_ASYNC OFF,
PAGE_VERIFY NONE,
DATE_CORRELATION_OPTIMIZATION OFF,
DISABLE_BROKER,
PARAMETERIZATION SIMPLE,
SUPPLEMENTAL_LOGGING OFF
WITH ROLLBACK IMMEDIATE;
END
GO
IF IS_SRVROLEMEMBER(N'sysadmin') = 1
BEGIN
IF EXISTS (SELECT 1
FROM [master].[dbo].[sysdatabases]
WHERE [name] = N'$(DatabaseName)')
BEGIN
EXECUTE sp_executesql N'ALTER DATABASE [$(DatabaseName)]
SET TRUSTWORTHY OFF,
DB_CHAINING OFF
WITH ROLLBACK IMMEDIATE';
END
END
ELSE
BEGIN
PRINT N'The database settings cannot be modified. You must be a SysAdmin to apply these settings.';
END
GO
IF IS_SRVROLEMEMBER(N'sysadmin') = 1
BEGIN
IF EXISTS (SELECT 1
FROM [master].[dbo].[sysdatabases]
WHERE [name] = N'$(DatabaseName)')
BEGIN
EXECUTE sp_executesql N'ALTER DATABASE [$(DatabaseName)]
SET HONOR_BROKER_PRIORITY OFF
WITH ROLLBACK IMMEDIATE';
END
END
ELSE
BEGIN
PRINT N'The database settings cannot be modified. You must be a SysAdmin to apply these settings.';
END
GO
ALTER DATABASE [$(DatabaseName)]
SET TARGET_RECOVERY_TIME = 0 SECONDS
WITH ROLLBACK IMMEDIATE;
GO
IF EXISTS (SELECT 1
FROM [master].[dbo].[sysdatabases]
WHERE [name] = N'$(DatabaseName)')
BEGIN
ALTER DATABASE [$(DatabaseName)]
SET FILESTREAM(NON_TRANSACTED_ACCESS = OFF),
CONTAINMENT = NONE
WITH ROLLBACK IMMEDIATE;
END
GO
IF EXISTS (SELECT 1
FROM [master].[dbo].[sysdatabases]
WHERE [name] = N'$(DatabaseName)')
BEGIN
ALTER DATABASE [$(DatabaseName)]
SET AUTO_CREATE_STATISTICS ON(INCREMENTAL = OFF),
MEMORY_OPTIMIZED_ELEVATE_TO_SNAPSHOT = OFF,
DELAYED_DURABILITY = DISABLED
WITH ROLLBACK IMMEDIATE;
END
GO
USE [$(DatabaseName)];
GO
IF fulltextserviceproperty(N'IsFulltextInstalled') = 1
EXECUTE sp_fulltext_database 'enable';
GO
PRINT N'Creating [EDW_INTERNAL]...';
GO
CREATE SCHEMA [EDW_INTERNAL]
AUTHORIZATION [dbo];
GO
PRINT N'Creating [EDW_INTERNAL].[DB_VERSIONS]...';
GO
CREATE TABLE [EDW_INTERNAL].[DB_VERSIONS] (
[ID] BIGINT IDENTITY (1, 1) NOT NULL,
[MODULE] VARCHAR (30) NOT NULL,
[FROM_VERSION] VARCHAR (20) NOT NULL,
[TO_VERSION] VARCHAR (20) NOT NULL,
[UPGRADE_DML_APPLIED_YN] VARCHAR (1) NOT NULL,
CONSTRAINT [PK_DB_VERSIONS] PRIMARY KEY CLUSTERED ([MODULE] ASC, [FROM_VERSION] ASC, [TO_VERSION] ASC)
);
GO
PRINT N'Creating [EDW_INTERNAL].[DML_UPGR_SCRIPT_MASTER]...';
GO
CREATE TABLE [EDW_INTERNAL].[DML_UPGR_SCRIPT_MASTER] (
[MODULE] VARCHAR (30) NOT NULL,
[FROM_VERSION] VARCHAR (20) NOT NULL,
[TO_VERSION] VARCHAR (20) NOT NULL,
[APPLY_ORDER] INT NOT NULL,
[UPGR_SCRIPT_FILEPATH] VARCHAR (1024) NOT NULL,
CONSTRAINT [PK_DML_UPGR_SCRIPT_MASTER] PRIMARY KEY CLUSTERED ([APPLY_ORDER] ASC, [TO_VERSION] ASC, [FROM_VERSION] ASC, [MODULE] ASC)
);
GO
PRINT N'Creating unnamed constraint on [EDW_INTERNAL].[DB_VERSIONS]...';
GO
ALTER TABLE [EDW_INTERNAL].[DB_VERSIONS]
ADD DEFAULT 'N' FOR [UPGRADE_DML_APPLIED_YN];
GO
PRINT N'Creating [EDW_INTERNAL].[UPGRADE_DML]...';
GO
CREATE PROCEDURE EDW_INTERNAL.UPGRADE_DML
#Module VARCHAR(30)
AS
BEGIN
DECLARE #Failure bit = 1;
IF #Failure = 1
BEGIN
RAISERROR
(N'One or more database upgrade query statements have failed. Please check the DML Upgrade Log table for details.',
20, -- Severity.
1 -- State
) WITH LOG;
END
END
GO
/*
Post-Deployment Script Template
--------------------------------------------------------------------------------------
This file contains SQL statements that will be appended to the build script.
Use SQLCMD syntax to include a file in the post-deployment script.
Example: :r .\myfile.sql
Use SQLCMD syntax to reference a variable in the post-deployment script.
Example: :setvar TableName MyTable
SELECT * FROM [$(TableName)]
--------------------------------------------------------------------------------------
*/
-- Reference to load the Version Upgrade tables with rows
--:r VersionUpgradeRowsPopulate.sql
-- execute
EXEC [EDW_INTERNAL].[UPGRADE_DML] #MODULE = 'Test_Common'
GO
GO
DECLARE #VarDecimalSupported AS BIT;
SELECT #VarDecimalSupported = 0;
IF ((ServerProperty(N'EngineEdition') = 3)
AND (((##microsoftversion / power(2, 24) = 9)
AND (##microsoftversion & 0xffff >= 3024))
OR ((##microsoftversion / power(2, 24) = 10)
AND (##microsoftversion & 0xffff >= 1600))))
SELECT #VarDecimalSupported = 1;
IF (#VarDecimalSupported > 0)
BEGIN
EXECUTE sp_db_vardecimal_storage_format N'$(DatabaseName)', 'ON';
END
GO
PRINT N'Update complete.';
GO

How can I use IF statements in Teradata without using BTEQ

I'm trying to create some deployment tools and I don't want to use BTEQ. I've been trying to work with the Teradata.Client.Provider in PowerShell but I'm getting syntax errors on the creation of a table.
[Teradata Database] [3706] Syntax error: expected something between
';' and the 'IF' keyword.
SELECT * FROM DBC.TablesV WHERE DatabaseName = DATABASE AND TableName = 'MyTable';
IF ACTIVITYCOUNT > 0 THEN GOTO EndStep1;
CREATE MULTISET TABLE MyTable ,
NO FALLBACK ,
NO BEFORE JOURNAL,
NO AFTER JOURNAL,
CHECKSUM = DEFAULT,
DEFAULT MERGEBLOCKRATIO
(
MyColId INTEGER GENERATED ALWAYS AS IDENTITY
(START WITH 1
INCREMENT BY 1
MINVALUE 0
MAXVALUE 2147483647
NO CYCLE)
NOT NULL,
MyColType VARCHAR(50) NULL,
MyColTarget VARCHAR(128) NULL,
MyColScriptName VARCHAR(256) NULL,
MyColOutput VARCHAR(64000) NULL,
isMyColException BYTEINT(1) NULL,
ExceptionOutput VARCHAR(64000) NULL,
MyColBuild VARCHAR(128) NULL,
MyColDate TIMESTAMP NOT NULL
)
PRIMARY INDEX PI_MyTable_MyColLogId(MyColLogId);
LABEL EndStep1;
I would rather not use BTEQ as I've not found it has worked well in other deployment tools we have created and requires a bit of hacks. Is there anything I can use that would avoid using that tool?
What Parse error?
The CREATE will fail due to double INTEGER in MyColId and VARCHAR(max) in ExceptionOutput, it's an unknown datatype in Teradata.

Unable to execute dynamic sql Error:global_names parameter must be set to TRUE for this operation

I have a remote table with blob column accessed via a db link. I want to insert a blob from my local table to remote table blob column.I am executing dynamic sql like follows
declare
theblob blob;
theclob clob;
thenumber number;
begin
select base64encode2(image) into theclob from per_images where image_id = 113077;
execute immediate 'insert into image#APPSERP2ERPAPPS(column1,column2,column3) values((select null from dual),(select base64encode2(image) from per_images where image_id = 113077),(select ceil(5.4) from dual))';
commit;
end;
When i run the sql i get ORA-02069: global_names parameter must be set to TRUE for this operation.
If i do ALTER SESSION SET GLOBAL_NAMES = true then i get database link APPSERP2ERPAPPS.CSN.EDU.PK connects to TEST.CSN.EDU.PK error while inserting into blob.
Kindly tell me how can i insert blob into remote table blob column.
Thanks
To be able to insert over a dblink the insert sentence must match this format
Insert into table2#dblink select * from Table1
here more info.

RODBC command 'sqlQuery' has problems with table variables in t-SQL

I am using the RODBC package which I am applying on a Microsoft SQL Server 2012.
Now I have discovered a phenomenon that puzzles me.
If I run the following query with the RODBC command sqlQuery, then, in R, I will get back an empty data frame with the columns Country, CID, PriceID and WindID.
DECLARE #tbl_IDs TABLE
(
Country nvarchar(30),
CID nvarchar(5),
PriceID int,
WindID int
)
SELECT * FROM #tbl_Ids
So far, everything is fine.
However, if I try to write a record to the table variable and execute
DECLARE #tbl_IDs TABLE
(
Country nvarchar(30),
CID nvarchar(5),
PriceID int,
WindID int
)
INSERT INTO #tbl_IDs
VALUES
('Germany', 'DE', 112000001, 256000002);
SELECT * FROM #tbl_Ids
Then, in R, the result will be an empty character instead of a dataframe with one record. Still the same query works perfectly with SQL Server Management Studio.
Also, we have traced the behaviour of the DB Server while the R-Query is executed and it seems the server handles it perfectly. It seems that the RODBC interface has a problem to return the result to the R console.
Does anybody have an idea how this issue can be resolved?
Try toggling NOCOUNT as below:
old_qry <- "
DECLARE #tbl_IDs TABLE
(
Country nvarchar(30),
CID nvarchar(5),
PriceID int,
WindID int
)
INSERT INTO #tbl_IDs
VALUES
('Germany', 'DE', 112000001, 256000002);
SELECT * FROM #tbl_Ids
"
##
new_qry <- "
SET NOCOUNT ON;
DECLARE #tbl_IDs TABLE
(
Country nvarchar(30),
CID nvarchar(5),
PriceID int,
WindID int
);
INSERT INTO #tbl_IDs
VALUES
('Germany', 'DE', 112000001, 256000002);
SET NOCOUNT OFF;
SELECT * FROM #tbl_Ids
"
R> sqlQuery(tcon, gsub("\\n", " ", old_qry))
#character(0)
R> sqlQuery(tcon, gsub("\\n", " ", new_qry))
# Country CID PriceID WindID
#1 Germany DE 112000001 256000002
Basically you want to SET NOCOUNT ON at the beginning of your code, and SET NOCOUNT OFF just before the final SELECT statement.
Since database server handles query correctly, save the multiple line action TSQL query as a SQL Server Stored Procedure and have R call it retrieving the resultset.
Do note you can even pass parameters in the EXEC sp line from R to MSSQL. Also as mentioned, include the SET NOCOUNT ON declaration in the query to avoid undesired result character(0):
library("RODBC");
conn <- odbcConnect("DSN Name",uid="***",pwd="***"); # WITH DSN
#conn <-odbcDriverConnect('driver={SQL Server};server=servername;database=databasename;
#trusted_connection=yes;UID=username; PWD=password') # WITH DRIVER
df<-sqlQuery(conn, "EXEC dbo.StoredProcName");

Resources