Ok - I have a situation in which I must execute a dynamically built stored procedure against tables that may, or may not be in the database. The data retrieved is then shunted to a VB.Net backed ASP based report page. By design, if the tables are not present in the database, the relevant data is automatically hidden on the report page. Currently, I'm doing this by checking for the inevitable error, and hiding the div in the catch block. A bit kludgy, but it worked.
I can't include the VB code-behind, but the relevant stored procedure is included below.
However, a problem with this method was recently brought to my attention when, for no apparent reason, the div was being hidden even though the proper data was available. As it turned out, the user trying to select the table in the dynamic SQL call didn't have the proper select permissions, an easy enough fix once I could track it down.
So, two fold question. First and foremost - is there a better way to check for a missing table than through catching the error in the VB.Net codebehind? All things considered, I'd rather save the error checking for an actual error. Secondly, is there a preferred method to squirrel out a particular OLE DB error out of the general object caught by the try->catch block other than just checking the actual stack trace string?
SQL Query - The main gist of the code is that, due to the design of the database, I have to determine the name of the actual table being targeted manually. The database records jobs in a single table, but each job also gets its own table for processing data on the items processed in that job, and it's data from those tables I have to retrieve. Absolutely nothing I can do about this setup, unfortunately.
DECLARE #sql NVarChar(Max),
#params NVarChar(Max),
#where NVarChar(Max)
-- Retained for live testing of stored procedure.
-- DECLARE #Table NvarChar(255) SET #Table = N'tblMSGExportMessage_10000'
-- DECLARE #AcctID Integer SET #AcctID = 10000
-- DECLARE #Type Integer SET #Type = 0 -- 0 = Errors only, 1 = All Messages
-- DECLARE #Count Integer
-- Sets our parameters for our two dynamic SQL calls.
SELECT #params = N'#MsgExportAccount INT, #cnt INT OUTPUT'
-- Sets our where clause dependent upon whether we want all results or just errors.
IF #Type = 0
BEGIN
SELECT #where =
N' AND ( mem.[MSGExportStatus_OPT_CD] IN ( 11100, 11102 ) ' +
N' OR mem.[IngestionStatus_OPT_CD] IN ( 11800, 11802, 11803 ) ' +
N' OR mem.[ShortcutStatus_OPT_CD] IN ( 11500, 11502 ) ) '
END
ELSE
BEGIN
SELECT #where = N' '
END
-- Retrieves a count of messages.
SELECT #sql =
N'SELECT #cnt = Count( * ) FROM dbo.' + QuoteName( #Table ) + N' AS mem ' +
N'WHERE mem.[MSGExportAccount_ID] = #MsgExportAccount ' + #where
EXEC sp_executesql #sql, #params, #AcctID, #cnt = #Count OUTPUT
To avoid an error you could query the sysobjects table to find out if the table exists. Here's the SQL (replace YourTableNameHere). If it returns > 0 then the table exists. Create a stores procedure on the server that runs this query.
select count(*)
from sysobjects a with(nolock)
where a.xtype = 'U'
and a.name = 'YourTableNameHere'
Related
I am using ASP.NET and a SQL Server database.
I have options to search values from gridview base on some criteria. The criteria are mostly independent from each other.
For example:
"where ProductType = " + Convert.ToInt32(recordType.persoRecord) +
" and AccountNumber like '%" + SearchValue + "%'";
or
"where fileid=" + File_ID + ShowSearch +
" and lower(j.CardHolderName) like '%" + SearchValue.Trim().ToLower() + "%'
There are a lot of options to search by user. I have millions of rows of data in this table, in order to fetch the data and bind it fast to gridview, I have created a stored procedure.
It works fine for fetching and binding but for searching, it's hard to manage. Due to I don't have much time, i want to configure the stored procedure to 'if there's a searching' fetch the searched data only.
Here is my stored procedure:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[SetJobsGrid]
(#FileID varchar(50),
#PageIndex int = 1,
#PageSize int = 1,
#DynamicQuery NVARCHAR(MAX),
#SearchFlag bit,
#RecordCount int output)
AS
BEGIN
DECLARE #SearchQuerySQL as nvarchar(MAX)
SET NOCOUNT ON;
select ROW_NUMBER()OVER (
order by j.creationdate asc,
j.id desc
) AS RowNumber
, j.id as 'Serial #', j.jobname as 'File Record Name',
j.RecordNumber as 'Record Number',i.issuername as'Issuer Name',p.ProductName as 'Product Name',p.productNumber as 'Product Number',
(j.AccountNumber) as AccountNumber ,j.CardHolderName as 'Card Holder Name', j.CardholdersBranchName as 'Card Holder Branch Name',
j.ShipmentBranchName as 'Shipment Branch Name', j.EmbossingCardholderName as 'Embossing Card Holder Name', j.MaskedPAN as 'PAN',
j.creationdate as 'File Record Creation Date', j.status as 'File Record Status', j.chipdatastatuserror as 'ChipDataStatus Erro',
j.chipdatastatuserrormessage as 'Error Message', j.chipdatastatus as 'Data Prepared',j.isduplicaterecord as 'isduplicate',j.isduplicatefromsamefile as 'IsDuplicateFromSameFile',
j.validationerrors , j.isworkordercreatedForCard,j.isworkordercreatedForPin,j.isworkordercreatedForCarrier,j.PersoMachineId,j.PinMachineId,j.CarrierMachineId
INTO #Results
FROM jobs j join issuer i on j.issuerid=i.id join Product p on p.id=j.productid WHERE fileid = #FileID
IF(#SearchFlag = 1)
begin
select #SearchQuerySQL = 'SELECT ' + #RecordCount + ' = COUNT(*) FROM #Results ' + #DynamicQuery
EXEC(#SearchQuerySQL)
select #SearchQuerySQL = 'SELECT * FROM #Results ' + #DynamicQuery + ' and RowNumber BETWEEN(#PageIndex -1) * #PageSize + 1 AND(((#PageIndex -1) * #PageSize + 1) + #PageSize) - 1'
EXEC(#SearchQuerySQL)
end
ELSE
begin
SELECT #RecordCount = COUNT(*)
FROM #Results
SELECT * FROM #Results
WHERE RowNumber BETWEEN(#PageIndex -1) * #PageSize + 1 AND(((#PageIndex -1) * #PageSize + 1) + #PageSize) - 1
end
DROP TABLE #Results
END
When the SearchFlag is set to true from ASP.NET, I want to fetch only searched value. #DynamicQuery set from asp for example:
WHERE AccountNumber LIKE '%" + SearchValue + "%'"
or with many different case.
When I run this stored procedure as in the above, I get an exception:
Conversion failed when converting the varchar value 'select ' to data type int
Regards
To make the stored procedure use your dynamic where statement, it is better to use sp_executesql
example:
EXEC sp_executesql
N'select * from Employees where Id = #param1',
N'#param1 int'
,#param1 = 1
for more information about dynamic query refer to the following site
SQL Server Dynamic SQL
Too long for a comment but you need to understand why the error occurs to fix it. You have this snippet in your code
select #SearchQuerySQL = 'SELECT ' + #RecordCount +
First, if you intend to assign a scalar value (e.g., #SearchQuerySQL) then you should use SET, not SELECT. That's another topic you can research at your leisure. The assignment expression that follows is where you intended to do string concatenation. Unfortunately, the interpretation of the plus sign operator varies according to datatypes involved.
What happens when the database engine encounters an operator that involves 2 different datatypes. Like any other language, one (or both) must be converted to the same type in order to perform the expressed operation. How does the engine do that? There are rules for datatype precedence. In this case, your int parameter has higher precedence and so those strings in that expression are converted to int. That fails with the error that you encountered.
If you want to write dynamic sql, you need to have an advanced understanding of tsql. You should also consider searching the internet first before trying to reinvent the wheel. Maybe Aaron's article on dynamic pagination might help - but it might be a bit much for you at this point.
And while you're mucking about with things, add some comments within the procedure declaration about the usage you intend to support. No one should have to read your code to understand what it does and how it should be used.
When using a stored procedure to dynamically generate a table based on configuration and return a result set (SELECT) with the records in that table, the second call to that procedure to generate a different table structure returns no records and it reports a missing column from a previous temporary table of the same name.
I tried this with MariaDB 10.3 and 10.1.21 and received the same result. I have minimized my code here to the minimum to demonstrate the error after trying several variations of single and multiple sub-procedures.
I also tried using some transaction control with COMMITS after executing the process, before trying to start the process with a different parameter, but got the same results.
DROP PROCEDURE IF EXISTS CreateATable;
DELIMITER $$
CREATE PROCEDURE CreateATable( _TableType tinyint )
BEGIN
DROP TEMPORARY TABLE IF EXISTS aTable;
IF _TableType = 1 THEN
SET #SQL_Statement :=
CONCAT(
'CREATE TEMPORARY TABLE aTable (',
'the_id bigint, ',
'the_column varchar(100) ',
') engine=INNODB',
';');
ELSE
SET #SQL_Statement :=
CONCAT(
'CREATE TEMPORARY TABLE aTable (',
'the_id bigint, ',
'the_other_column varchar(100) ',
') engine=INNODB',
';');
END IF;
PREPARE stmtCreateTable FROM #SQL_Statement;
EXECUTE stmtCreateTable;
DEALLOCATE PREPARE stmtCreateTable;
SET #SQL_Statement := NULL;
END$$
DELIMITER ;
DROP PROCEDURE IF EXISTS GetATable;
DELIMITER $$
CREATE PROCEDURE GetATable()
BEGIN
CALL CreateATable( 1 );
SELECT * FROM aTable;
CALL CreateATable( 2 );
SELECT * FROM aTable;
END$$
DELIMITER ;
DROP PROCEDURE IF EXISTS GetATable2;
DELIMITER $$
CREATE PROCEDURE GetATable2(_TableType tinyint)
BEGIN
CALL CreateATable( _TableType );
SELECT * FROM aTable;
END$$
DELIMITER ;
/*
Test execution script starts here
*/
-- Just CALL Create for one and Select
CALL CreateATable( 1 );
DESCRIBE aTable;
SELECT * FROM aTable;
CALL CreateATable( 2 );
DESCRIBE aTable;
SELECT * FROM aTable;
-- -> no errors
-- now CALL procedure to Create and Select from two different temp tables
CALL GetATable();
-- -> no errors
-- now CALL procedure to CREATE AND SELECT from ONE temp table definition using a parameter to select
CALL GetATable2(1);
CALL GetATable2(2);
-- Error Code: 1054. Unknown column 'mySchema.aTable.the_column' in 'field list'
I would expect that I can pass a parameter to a stored procedure to generate a temporary table, and return the records of that temporary table. Even if I call that same procedure multiple times with different parameters on the same session.
The actual results are that when the stored procedure is called to generate the temporary table with a different table structure, it returns this error complaining about the column missing from the temporary table created in the previous invocation of that same stored procedure.
Error Code: 1054. Unknown column 'mySchema.aTable.the_column' in 'field list'
The only way I have found to prevent this error is
a. ending the jdbc connection and ending the server session
b. recompiling one of the stored procedures in the call stack
Recompiling is not viable. And ending the session seems unreasonable.
This seems like a defect. But would be interested to find if there is some way to get this to work.
This seems like a bug and you can report it directly to the MariaDB team at MariaDB bugs database.
A temporary solution is to use a prepared statement in the stored procedure GetATable2 (my test on MariaDB 10.3.16 to use EXECUTE IMMEDIATE):
...
CREATE PROCEDURE `GetATable2`(`_TableType` TINYINT)
BEGIN
CALL CreateATable(`_TableType`);
-- SELECT * FROM `aTable`;
EXECUTE IMMEDIATE 'SELECT * FROM `aTable`';
END$$
...
See dbfiddle.
create or replace procedure sample
as
ID VARCHAR(20);
BEGIN
execute immediate
'CREATE GLOBAL TEMPORARY TABLE UPDATE_COLUMN_NO_TP
(
NAME VARCHAR2(256)
)';
INSERT INTO UPDATE_COLUMN_NO_TP
SELECT SRC_PK_COLUMNS.PK_KEY
FROM SRC_PK_COLUMNS
WHERE NOT EXISTS (
SELECT 1
FROM TGT_PK_COLUMNS
WHERE TGT_PK_COLUMNS.ID = SRC_PK_COLUMNS.ID);
END;
Error is:
The table is no exist.
So, I want a best solution for this scenario. In my stored procedure I have 10 temporary tables. All are all dynamic creations and inserts.
Table UPDATE_COLUMN_NO_TP not exists at compile time, so you got the error.
If you created a table dynamically, you should access it dynamically.
And pay attention to Mat's comment about essence of GTT.
execute immediate '
INSERT INTO UPDATE_COLUMN_NO_TP
SELECT SRC_PK_COLUMNS.PK_KEY
FROM SRC_PK_COLUMNS
WHERE NOT EXISTS (
SELECT 1
FROM TGT_PK_COLUMNS
WHERE TGT_PK_COLUMNS.ID = SRC_PK_COLUMNS.ID
)
';
We have an application that creates a table with a randomly generated name. I would like to create a trigger on this table.Since I do not know the name of the tabe I would like to get it from the all_table view. How can I go about achieveing something like this?
create or replace trigger t1
after insert or update on (select table_name from all_tables where owner = 'CustomAPP' and table_name like 'STAGE_%')
-- for each row
declare
-- local variables here
begin
end t1;
The SQL above obviously gives an error because of the select clause after the create trigger instead of a table name. Please advise
You would need to make the entire CREATE TRIGGER dynamic in order to do this. Something like this should work. You probably want to make the trigger name depend on the name of the table assuming there could be multiple tables that your query against ALL_TABLES might return multiple rows. And you certainly want the trigger to do something rather than having an empty body.
SQL> create table stg_12345( col1 number );
Table created.
SQL> begin
2 for x in (select *
3 from user_tables
4 where table_name like 'STG%')
5 loop
6 execute immediate
7 'create or replace trigger trg_foo ' ||
8 ' before insert on ' || x.table_name ||
9 ' for each row ' ||
10 'begin ' ||
11 ' null; ' ||
12 'end;';
13 end loop;
14 end;
15 /
PL/SQL procedure successfully completed.
SQL> select count(*) from user_triggers where trigger_name = 'TRG_FOO';
COUNT(*)
----------
1
Of course, the idea of an application that creates tables on the fly is one that frightens me to the core. If you have any control over that, I would strongly suggest reconsidering the architecture.
Solution 1:
If the problem is "poor performance due to lack of statistics", perhaps changing the OPTIMIZER_DYNAMIC_SAMPLING parameter at a system or session level can help. See the Performance Tuning Guide for a more thorough discussion, but I've found the default of 2 (64 blocks) to be insufficient, especially for large data sets where keeping optimizer statistics current is impractical.
Solution 2:
If you really want to automatically create a trigger after a table's been created, you'll need to create a DDL trigger for the schema. The SQL below demonstrates that.
CREATE OR REPLACE TRIGGER MAKE_ME_A_TRIGGER
AFTER CREATE ON CUSTOM_APP_SCHEMA
AS
l_trigger_sql varchar2(4000);
BEGIN
if l_ora_obj_dict_type = 'TABLE'
then
l_trigger_sql := 'create or replace trigger ' || ora_dict_obj_name
' before insert on ' || ora_dict_obj_type||
' for each row ' ||
'begin ' ||
' null; ' ||
'end;'
execute immediate l_sql;
end if;
END;
/
You can use EXECUTE IMMEDIATE to dynamically execute SQL, including DDL scripts, provided the active connection has appropriate permissions on the database. Use PL/SQL to build the full DDL statement via string concatenation, and then you can execute it dynamically.
Docs:
http://docs.oracle.com/cd/B12037_01/appdev.101/b10807/13_elems017.htm
More Docs:
http://docs.oracle.com/cd/B28359_01/appdev.111/b28370/dynamic.htm
The title, while long, pretty much says it all.
What I have is a master table with a bunch of supporting table relations through foreign keys. For a few of the foreign tables, upon attempting to insert a record into the master table where one of the foreign keys doesn't exist, the data would be passed to the foreign table to create the record first, thereby making the constraint valid and passing the key to the created record back to the insert procedure of the master table.
This data comes from a form in String form, but naturally the foreign key will be an int. The process would look something like this:
-- ASP.NET Web Form --
Requestor Name: _____________ (combobox)
Request: _____________ (dropdownlist)
Date: _____________ (datepicker)
This is a super simplified version, but assume we have a master table with the above data, where both names are foreign keys to a People table. The name fields are comboboxes with a populated list of names linking to People. However, if I wanted to enter a person who didn't yet exist in the People table, the procedure should first create the Person, then use the ID from that new record as the foreign key in the Master table containing columns for the above.
I'm using SQL Server and ASP.NET with VB.NET codebehind. I've been scratching my head over this one for awhile, how to pass data (in different forms such as a foreign key or string) between the web server and DB server, as well as where to validate / transform the data.
It seems the entered name will be passed as an ID if the foreign key exists, and a String if not.
This is my most perplexing problem so far, and no idea where else to look. I've read up on Scott Mitchell's site and others.
MY SOLUTION (?)
The best I can come up with is to pass the user input from the user as a string and convert it to int in the T-SQL procedure. If the value was selected from the drop down, it should match precisely with a valid foreign key. If it doesn't match, then create a new Person and return a foreign key. Is this best practice?
This seems complicated because it is. You have to get your hands dirty. If you need a relational database with ACID support, there's no auto-magical way of getting around it.
Relational databases 101: The primary key must exist before the foreign key can be populated (This is the reason why data warehouse developers populate the dimension table before the fact table). You'll have to design the logic to validate that the primary key exists, insert and get the key if not, and just get the key if exists.
Here's my implementation. I don't know if it's the best, but it worked well for me. Basically I take the values from the controls; in the case of the combobox I need the values from both the TextBox and DropDownList. I then pass those values to the following function in my codebehind:
'This method determines if the name selected already exists in the selection
' options and if so assigns the corresponding ID value to an object variable,
' if not it assigns the value of the `TextBox` to the variable.
Protected Function _ValidateValues(ByRef ddl As DropDownList, ByRef cb As TextBox) As Object
'Ensures the selected value is valid by checking against the entered value in the textbox
If Not String.IsNullOrEmpty(cb.Text) Then
If ddl.Items.Count > 0 Then
If StrComp(cb.Text, ddl.SelectedItem.ToString) = 0 Then
Return ddl.Items.Item(ddl.SelectedIndex).Value 'Returns the index of dropdown selected name
End If
End If
'This counts the capital letters in the entered value and if fewer than 2
' auto capitalizes the first letters. This also allows for project code
' names such as "DOORS" and people names such as "Allen McPherson" etc.
' Be careful though because if "allen McPherson" is entered, it will NOT
' be corrected, though it displays correctly.
Dim rg As New Regex("[A-Z]")
Dim mc As MatchCollection = rg.Matches(cb.Text)
If mc.Count < 2 Then
Return StrConv(cb.Text, VbStrConv.ProperCase)
Else : Return cb.Text
End If
End If
'Returns a SQL DB NULL object if an empty string is submitted
Return DBNull.Value
End Function
Then my stored procedure handles the values something like so...
(Forgive me if I neglected to replace some of the values. I tried to catch them all.)
CREATE PROCEDURE spInsertUser
#User nvarchar(50) = NULL,
#Role nvarchar(50) = NULL,
#RecordID int output -- Returned Value
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- CHECK IF USER EXISTS
-- Insert new record to Users table if the requested user doesn't exist
-- Needed to ensure that the foreign keys are relevant
IF #User = '' OR #User IS NULL BEGIN SET #User = NULL SET #RecordID = NULL END --Ensures that an empty string cannot be submitted, thereby causing an error.
ELSE BEGIN
declare #forename varchar(50), #surname varchar(50)
declare #uid table (ID int)
declare #users table (ID smallint, Name nvarchar(50))
insert into #users
select ID, Name from Users
--If the value can be converted into an int, we need go no further.
BEGIN TRY SET #RecordID = CONVERT(smallint, #User) END TRY
BEGIN CATCH
BEGIN TRY --Otherwise, attempt to parse the name
Set #User = LTRIM(RTRIM(#User)) --Trim the extra space at the beginning and end. This ensures the following "IF" test will evaluate properly.
IF NOT CHARINDEX(' ', #User) > LEN(#User) AND CHARINDEX(' ', #User) > 0 BEGIN -- Confirm First & Last Name exist
Set #forename = RTRIM(LEFT(#User, CHARINDEX(' ',#User,0)-1))
Set #surname = LTRIM(RIGHT(#User, LEN(#User) - CHARINDEX(' ',#User,0)))
Set #User = #forename + ' ' + #surname --Ensure that there is a valid First & Last name
IF LEN(#forename) > 1 AND LEN(#surname) > 1 BEGIN -- Confirm First & Last Name exist
--First ensure that the User doesn't already exist, and if
-- so use their ID, if not insert the new User.
IF NOT EXISTS (select Name from #users where Name like #User) BEGIN --Check if the user already exists
INSERT INTO Users (Name, Forename, Surname) OUTPUT INSERTED.ID INTO #uid Values (#User, -- If not, insert them
#forename, #surname) --Nicely manicured first, last, and full names
SET #RecordID = CONVERT(smallint, (select MAX(ID) from #uid)) END -- Now set the Role to the ID of the new user
ELSE BEGIN --Otherwise if the user already exists, set the Role to the ID of that user
SET #RecordID = (select ID from #users where Name like #User) END
IF NOT EXISTS (select * from rUsersInRoles where UserID = #RecordID) BEGIN
--Do some string manipulation to increase the chances of matching the role
SET #Role = REPLACE(REPLACE(REPLACE(LTRIM(RTRIM(#Role)), ' ', '%'), '.', '%'), '#', '%') --Trims & replaces spaces & periods with wildcards
INSERT INTO rUsersInRoles (UserID, UserRoleID) VALUES
(#RecordID, (select top 1 ID from rUserRoles where Role like #Role)) END
END
END
END TRY
BEGIN CATCH END CATCH
END CATCH
END
END
This stored procedure deals with the case of User Roles as well. If the more simple case of Users only is needed, simply remove the clauses dealing with the checking and insertion of User Roles. :)