I have a .aspx page that has query to and informix database. This query is done via an odbc connection and is put into a datatable. Then this datatable is used as the datasource for a radio button group.
My problem is that for whatever reason the time is being appended to the radio button as "12:00:00 AM". This is odd because the informix field is a date field that does not include the time. If I were to run the query outside of the webpage it returns it without the time... "2012-06-15"
So in summary... what I am getting is: "6/15/2012 12:00:00 AM" and what I want is "06/15/2012"
The query is as follows:
"select DATE(attend_date) as attend_date from soar_major_table where major =? and active<>'N'"
The code that creates the datatable:
string connString;
connString = ConfigurationManager.ConnectionStrings [ "ERP" ].ConnectionString;
OdbcConnection conn = new OdbcConnection ( );
conn.ConnectionString = connString;
string sql = "select DATE(attend_date) as attend_date from soar_major_table where major =? and active<>'N' ";
OdbcCommand command = new OdbcCommand ( );
command.CommandText = sql;
command.Parameters.Add ( new OdbcParameter ( "major", major ) );
command.Connection = conn;
DataTable dt = new DataTable ( );
OdbcDataAdapter dataAdapter = new OdbcDataAdapter ( );
dataAdapter.SelectCommand = command;
try
{
conn.Open ( );
dataAdapter.Fill ( dt );
}
finally
{
if ( conn != null && conn.State == ConnectionState.Open )
{
command.Dispose ( );
dataAdapter.Dispose ( );
conn.Close ( );
}
}
return dt;
And lastly the population of the radio btn group:
if ( dt.Rows.Count > 0 )
{
rdoDate.DataSource = dt;
rdoDate.DataTextField = "attend_date";
rdoDate.DataValueField = "attend_date";
rdoDate.DataBind ( );
}
The problem is upstream of the Informix data server, I believe.
When you execute:
SELECT DATE(attend_date) ...
the server will return that value as a 4-byte integer representing the number of days since 1899-12-31 (so 1900-01-01 was day 1), which is the internal representation of a DATE in Informix.
Something in a higher layer is then treating it as a 'date + time' value and assuming midnight is the time since there was no time component in the date, and is then rubbing salt in the wound by formatting it in am/pm notation.
This will involve client-side tracing of what's going on. My suspicion (not founded on anything except limited knowledge of the ODBC drivers) is that the problem is occurring in the .NET layers rather than the ODBC driver. However, you're way outside my area of expertise once you're above the ODBC layer (and I don't claim great expertise in ODBC).
You may be able to isolate the problem to the client code by using SQLIDEBUG=2:xyz in the environment (you might need to set that with SETNET32 for Windows). If it works at all on Windows (it does on Unix), then you'll end up with a file with a name starting xyz_ followed by various groups of digits and letters. That file can be analyzed by sqliprint and will show you what was sent to the Informix data server and returned to your client. Assuming the SQL was not hacked en route to the server, then you'll see the date returned as a simple date, and the problem is definitively client-side. If the SQL is hacked en route, then that too is a client-side problem.
It at least gives a starting point for debugging.
Look very carefully at the data types of the types your code is using. In particular, some DBMS have a DATE type that includes time information, and you may need to avoid that interpretation. The SQL standard has DATE (no time), TIME (no date) and TIMESTAMP (date and time) types.
Related
I'm using Classic ASP. I have a wrapper function for database queries that accepts a query string and an array of parameters, and auto-creates the proper query object and runs the query. Very handy and has been working great.
Here's my problem: When testing, I often want to see the exact text of the query being passed to SQL. Back in the "bad old days" of assembling queries through concatenation I could just write out the string. Now that I'm using parameterization it's a bit more tricky.
How do I take a peek at the fully-assembled query string just before it's passed to the database connection?
Here is the function I'm using, simplified. (The actual function doesn't assume string, for example.)
Public Function pquery( strQuery, params )
Dim cmd, param, thisParam, rs
Set cmd = Server.CreateObject( "ADODB.Command" )
cmd.ActiveConnection = MyConn
cmd.CommandText = strQuery
If IsArray( params ) then
Dim adVarChar : adVarChar = 200
For Each param In params
Set thisParam = cmd.CreateParameter( "#p", adVarChar, , len( param ), param )
cmd.Parameters.Append thisParam
Next
End If
Set rs = cmd.Execute
Set pquery = rs
End Function
I would consider using Sql Query Profiler, as it'll allow you to view the sql text as well as the values being passed in. It'll allow you to set breakpoints, as well as see how long it takes to run a query. However, this requires the query to be sent to the actual database (you had asked for before).
To do it beforehand, you would need to loop through the parameters collection in the command object, then do a find/replace with the key/value pairs in the command text property. it would be hackish at best, if you can use Profiler, go with that.
I have a strange phenomenon, while investigating the performance of a query, I noticed that it takes 6 seconds to run the query in sql server management studio, but it only takes .6 seconds when invoked from within a vb.net application.
I am getting the time stats from "set statistics time" in SSMS, and using System.Diagnostics.Stopwatch in vb.net application.
I have done everything I can think of to rule out caching, and verifying that the query within vb.net code is not deferred somehow. I've verified both are using the same database. I've restarted sql server, I've restarted the vb.net application, I've used
USE <dbname>;
GO
CHECKPOINT;
GO
DBCC DROPCLEANBUFFERS;
GO
When running the query in SSMS, flushing the cache (as above) makes no difference, indicating that the query does not benefit from any caching. The query takes abou 6 seconds every time. The same query takes .6 seconds when executed from vb.net via a SqlClient.SqlConnection. It goes that fast even after restarting both the sql server and the vb.net app.
I don't think the code of the query is terribly relevant, other than to note that it calls a function which in turn uses sql's version of "recursion".
Another note is that when executing the query from SSMS, I can speed it up by adding a needed index, after which it runs in about 1.4 seconds. When executed from vb.net app, it runs in .23 seconds with the index.
The question is, what could sql server be doing differently depending on whether a query is invoked from ssms vs. vb.nt, to result in this performance difference?
---edit to provide code---
Here is the query called in vb.net, yes it is embedded in a linq query in a rather otdd way, and yes I have verified that the linq query is not deferred for purposes of timing it.
fnName = "AT.fn_Plan_Item_Level_Sort_Reverse"
isParent = 1
foo = (From d In DBConn.sqlSelect(
"SELECT DENSE_RANK() OVER (ORDER BY SUBSTRING(Sort,1,3)) AS [GroupNumber]," & _
"Plan_ID, Master_Item_ID, Item_ID, Level, Sort " & _
"FROM " & fnName & "(#Plan_ID) ORDER BY Sort;" _
, GetCommandItem("#Plan_ID", SqlDbType.BigInt, _planId)).OfType(Of DataRowView)() _
Select New PlanItemRaw With { _
.PlanID = Wrapper(Of Long)(d.Item("Plan_ID")) _
, .MasterItemID = Wrapper(Of Long)(d.Item("Master_Item_ID")) _
, .ItemID = Wrapper(Of Long)(d.Item("Item_ID")) _
, .GroupNumber = Wrapper(Of Integer)(d.Item("GroupNumber")) _
, .Level = Wrapper(Of Integer)(d.Item("Level")) _
, .IsParent = (Wrapper(Of Integer)(d.Item("Level")) = isParent) _
, .Sort = Wrapper(Of String)(d.Item("Sort")) _
}).ToArray
Note that isParent is irrelevant to the SQL query, only used by the linq query.
DBConn.sqlSelect is homebrew, and a wrapper for the following, (items() is nothing in this case):
Public Function sqlSelect(ByVal sqlCommand As String, ByVal commandTimeout As Integer,
ByVal ParamArray items() As SQLCommandItem) As DataView
Using conn As New SqlClient.SqlConnection(_conn)
Using cmd As New SqlClient.SqlCommand(sqlCommand, conn)
cmd.CommandTimeout = commandTimeout
Call AddParamesterstoCommand(cmd, items)
Using da As New SqlClient.SqlDataAdapter
da.SelectCommand = cmd
Call logFullSQLStringForDebugging("sqlSelect", cmd)
Using custDS As New DataSet
da.Fill(custDS)
If custDS.Tables.Count = 0 Then
sqlSelect = Nothing
Else
sqlSelect =
custDS.Tables(custDS.Tables.Count-1).DefaultView
End If
End Using
End Using
End Using
conn.Close()
End Using
End Function
Now here is the query in SSMS:
set statistics time on
select DENSE_RANK() over (order by substring(sort,1,3)) as [groupnumber],
Plan_ID, Master_Item_ID, Item_ID, [level], Sort
from at.fn_Plan_Item_Level_Sort_Reverse(3538)
order by Sort
set statistics time off
Note that the parameter 3538 is identical to the value passed via #Plan_Id in the vb.net code. Here is the code for the function. This is basically using a recursive common table expression to collect and sort hierarchical data. It does call a couple other functions which are both quite simple.
RETURN
(
WITH Items
as
(
SELECT i.Plan_ID, mi.Item_ID [Master_Item_ID], i.Item_ID, Sort
FROM AT.Item i
LEFT OUTER JOIN AT.Mapping_Item mi ON i.Plan_ID = mi.Plan_ID AND i.Item_ID = mi.Master_Item_ID
WHERE i.Plan_ID = #Plan_ID
),
tmp
as
(
SELECT i.*, replicate('0',3-len(i.Sort)) + convert(varchar(max), i.Sort) 'Good_Sort', 0 as 'Level'
FROM AT.fn_Plan_Item_Leaves(#Plan_ID) r
JOIN Items i ON r.Plan_ID = i.Plan_ID AND r.Item_ID = i.Item_ID
UNION ALL
SELECT i.*, convert(varchar(max), ti.Good_Sort) + ' ' + replicate('0',3-len(i.Sort)) + convert(varchar(max), i.Sort) 'Good_Sort', ti.level + 1 as 'Level'
FROM Items i
JOIN tmp ti ON ti.Plan_ID = i.Plan_ID AND ti.Item_ID = i.Master_Item_ID
)
SELECT t.Plan_ID, t.Master_Item_ID, t.Item_ID, t.level,
CASE WHEN l.Plan_ID IS NOT NULL THEN convert(bit,1) ELSE convert(bit,0) END IsRoot, Good_Sort 'Sort'
FROM tmp t
LEFT OUTER JOIN AT.fn_Plan_Item_Roots(#Plan_ID) l ON t.Plan_ID=l.Plan_ID AND t.Item_ID=l.Item_ID
)
Thanks in advance for anyone's help. This is mystery which is driving me crazyyyy :(.
IF I run this following stored procedure directly on SQL server 2008R2, it returns the desired rows. But if I call this via ASP.net(3.5) it returns empty data from the last Select statement in SP.
Is there any scoping involved in this regarding the temp table #_CalendarDate?
Stored Procedure:
USE[DB]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [WC].[spsGetDayCyclePeriod]
(
#Param_StartDate datetime,
#NumberOfDayRange int,
#Campus_Type varchar(2)
)
AS
DECLARE #DateRangeStart datetime
DECLARE #DateRangeEnd datetime
DECLARE #_CalendarDate TABLE (CollegeDate datetime)
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
SET DATEFIRST 1
SELECT #DateRangeStart=max(CalendarDate) FROM [X].dbo.CalendarEvents
WHERE CalendarDate <= #Param_StartDate and left(CalendarType,1)= #Campus_Type
and (CalendarType <> #Campus_Type+'_H' and CalendarType<>'H'
and convert(INT, right(CalendarType, len(CalendarType)-3))>0)
SELECT #DateRangeEnd=min(CalendarDate) FROM [X].dbo.CalendarEvents
WHERE CalendarDate >= dateadd(day, #NumberOfDayRange-1, #Param_StartDate)
and left(CalendarType,1)= #Campus_Type and (CalendarType <> #Campus_Type+'_H'
and CalendarType<>'H' and convert(INT, right(CalendarType, len(CalendarType)-3))=0)
--Get all Dates within range
;WITH CollegeDate AS
(
SELECT #DateRangeStart AS DateValue
union all
SELECT dateadd(day, 1, DateValue)
FROM CollegeDate
WHERE dateadd(day, 1, DateValue) <= #DateRangeEnd
)
INSERT INTO #_CalendarDate (CollegeDate)
SELECT DateValue FROM CollegeDate OPTION (MAXRECURSION 0)
SELECT * from #_CalendarDate
END
ASP.Net code:
DataTable dayCycle = new DataTable();
var dateTimestr = startDate.ToString("yyyy-MM-dd HH:mm:ss");
using (SqlCommand sqlCommand = new SqlCommand("WC.spsGetDayCyclePeriod", new SqlConnection(Connection)))
{
sqlCommand.CommandType = CommandType.StoredProcedure;
var range = endDate.Subtract(startDate).Days;
sqlCommand.Parameters.Add(new SqlParameter("#Param_StartDate", dateTimestr));
sqlCommand.Parameters.Add(new SqlParameter("#NumberOfDayRange", range));
sqlCommand.Parameters.Add(new SqlParameter("#Campus_Type", campus));
//dayCycle = SqlHelper.GetDataTableUsingSqlCommand(sqlCommand);
try
{
SqlDataAdapter _dap = new SqlDataAdapter(sqlCommand);
_dap.Fill(dayCycle);
}
catch (Exception ex)
{ throw new Exception(ex.ToString()); }
return dayCycle;
Pass #Param_StartDate as a date time object rather than a string.
Thanks everyone for help.
Solved the problem myself ! Hopefully it will help someone else as well in future. Here is the answer:
In above ASP.net code:
sqlCommand.Parameters.Add(new SqlParameter("#Campus_Type", campus));
campus is a enum type and when I was calling the above method with enum type I was actually passing the int value instead of string. So this is what I change to. This was confusing because when I was debugging my code I was using cursor on top of the #campus which basically calls toString so I was seeing the right value (which was wrong) the actual value was passed was number of enum.
sqlCommand.Parameters.Add(new SqlParameter("#Campus_Type", campus.ToString()));
And the above change solved the problem.
What I learned from this is Always...Always confirm that the arguments you intend to pass to SP is what SP is receiving so retrieve back your passed arguments by running the following before you do anything with Stored procedures...
Select #Your_Param1, #Your_Param2
And then check on code side that you are receiving what you are expecting.
DbDataAdapter.Fill() is extremly slow when performing parameters!
I have a query with 2 parameters inside, and when I put those parameters hardcoded in the query it takes 1 second to execute (in a 470k table rows, returning only 20 rows).
I found many posts similars here and I tried all those solutions (set arithabort, option recompile, option optimize for, ...) with no luck.
I just perform a query (sql server 2008) and not a stored procedure, so the query with arithabort is like this:
string strSql = #"set ARITHABORT ON;
select TOP 20 ....
Also I tried to call set arithabort in the same transaction but performing that query first..
I don't know if I'm doing something wrong, but the sensation is the ado.net is performing a very bad execution plan in ado.net when I have defined parameters on it.
As a result of this bad choice, the execution time in SSMS is 1 second (after being cached) but in asp is like 9 seconds!
The query is something like this:
strSQL #="
select *
from Table1
where Name like #name";
And then:
DbProviderFactory factory = DbProviderFactories.GetFactory(mProvider);
DbCommand dbcmd = factory.CreateCommand();
if (CommandTimeout != null)
dbcmd.CommandTimeout = CommandTimeout.Value;
if(this.transaccion != null)
dbcmd.Transaction = this.transaccion;
dbcmd.Connection = dbc;
dbcmd.CommandText = strSQL;
if (parametros != null)
dbcmd.Parameters.AddRange(parametros);
DbDataAdapter dbda = factory.CreateDataAdapter();
dbda.SelectCommand = dbcmd;
DataTable dt = new DataTable();
dbda.Fill(dt);
return dt;
EDIT 14/01/2013 (18:44)
I'm not longer retrieve the connection from DbProviderFactory, insted I'm using directly SqlConnection and SqlCommand. I know DbCommand and DbProvider are a base clase... but I think there is something more in there.. because the performance drasticaly increase like 300%!
It's not the fill method, because I already tried in the code shown before..
Anyway, I don't know the reason why but using a SqlConnection is much faster! Any idea? Maybe isn't making that bad execution plan made before?
SqlCommand objCmd = new SqlCommand(strSQL, sqlConn);
if (CommandTimeout != null)
objCmd.CommandTimeout = CommandTimeout.Value;
if (this.transaccion != null)
objCmd.Transaction = SQLtransaccion;
if (parametros != null)
objCmd.Parameters.AddRange(parametros);
DbDataReader dbReader = objCmd.ExecuteReader();
DataTable dt = new DataTable();
dt.Load(dbReader);
dbReader.Close();
return dt;
Any help will be greatly appreciated,
Thanks,
I found the solution!
It was parameters!
I was using a wrong type in the the List!
Parametross.Add(bd.MakeParameter("#val", "%" + txtFind.Text + "%",
DbType.String));
DbType.String vs. DbType.AnsiString
Although both DbType.String and DbType.AnsiString deal with character data, these datatypes are processed differently, and using the wrong data type can have a negative effect on the application’s performance. DbType.String identifies the parameter as a 2-byte Unicode value and is sent to the server as such.DbType.AnsiString causes the parameter to be sent as a multibyte character string. To avoid excessive string conversions, use:
DbType.AnsiString for char or varchar columns and parameters.
DbType.String for unichar and univarchar columns and parameters.
Source:
http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc20066.0115/html/adonet/adonet49.htm
In my query there is a:
....
where Table.Col1 like #val
But the column type was varchar and I should use DbType.AnsiString, instead of DbType.String
Parametross.Add(bd.MakeParameter("#val", "%" + txtFind.Text + "%",
DbType.AnsiString));
In my huge table I was making a lot of unnecesary casts and this is the reason why the performance drastically fall down!
Hope this will help someone,
We're trying to insert data into a clipper database file (DBF file with NTX index file).
For a variety of reasons, we cannot change the database format.
We are currently seeing two issues:
With our existing code, we are not able to update or utilize the NTX index file (I believe). We would like to be able to do this. Do you know of an OLE or ODBC driver that can do this?
We are able to insert a row into the clipper database file (DBF) as long as we do not include a date. If we include a date, in any format, we get an error.
Some example code:
OleDbConnection con = new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + path + ";Extended Properties=dBASE IV");
string sql = "insert into TABLE (E, J, DATE, STARTTIME, ENDTIME) values ('1', '2', '2010-01-13' ,'08:12:12', '18:12:12')";
OleDbCommand myCommand = new OleDbCommand(sql);
myCommand.Connection = con;
con.Open();
myCommand.ExecuteNonQuery();
myCommand.Connection.Close();
and the exception is something like:
01/15/2010 12:50:31 {ERROR} ASITranslator.GUI.ASITranslatorGUI.insertSCH - Error in: Syntax error in INSERT INTO statement.-- StackTrace: at System.Data.OleDb.OleDbCommand.ExecuteCommandTextErrorHandling(OleDbHResult hr)
at System.Data.OleDb.OleDbCommand.ExecuteCommandTextForSingleResult(tagDBPARAMS dbParams, Object& executeResult)
at System.Data.OleDb.OleDbCommand.ExecuteCommandText(Object& executeResult)
at System.Data.OleDb.OleDbCommand.ExecuteCommand(CommandBehavior behavior, Object& executeResult)
at System.Data.OleDb.OleDbCommand.ExecuteReaderInternal(CommandBehavior behavior, String method)
at System.Data.OleDb.OleDbCommand.ExecuteNonQuery()
Again, without the DATE column, it works fine.
Is there a better provider to use for Clipper files (that provider works great for other DBF files).
Any ideas?
Seems the issue is primarily related to the OLE DBF / dbase driver is unable to write to the Clipper native format, which is a modified version of dbase III.
To write to the Clipper format, this string needs to be used:
Provider=MSDASQL.1;Persist Security Info=False;Mode=ReadWrite;Extended Properties="CollatingSequence=ASCII;DBQ=C:\DATA\8110FULL;DefaultDir=C:\DATA\8110FULL;Deleted=1;Driver={Microsoft dBase Driver (*.dbf)};DriverId=21;FIL=dBase III;FILEDSN=C:\Program Files\Common Files\ODBC\Data Sources\test.dsn;MaxBufferSize=2048;MaxScanRows=8;PageTimeout=600;SafeTransactions=0;Statistics=0;Threads=3;UID=admin;UserCommitSync=Yes;";Initial Catalog=C:\DATA\8110FULL
This will allow one to write to the file, including the DATE format.
However, this does NOT use the NTX index files (nor does it update them). For that, it would appear that we would need to use the CodeBase (or similar) Clipper driver.
First, it appears you are trying to add 'Text values' for the date columns regardless of them being in a date format. Additionally, if used in web-based applications where you would be using variables, you'd be best to use parameterized queries.
String sql = "insert into YourTable ( fld1, fld2, DateFld1, DateFld2 ) "
+ "value ( ?, ?, ?, ? )";
OleDbCommand myCommand = new OleDbCommand(sql);
OleDbParameter NewParm = new OleDbParameter( "parmFld1", 1 );
NewParm.DbType = DbType.Int32;
myCommand.Parameters.Add( NewParm );
NewParm = new OleDbParameter( "parmFld2", 2 );
NewParm.DbType = DbType.Int32;
myCommand.Parameters.Add( NewParm );
NewParm = new OleDbParameter( "parmDate1", DateTime.Now );
NewParm.DbType = DbType.DateTime;
myCommand.Parameters.Add( NewParm );
NewParm = new OleDbParameter( "parmDate2", DateTime.Now );
NewParm.DbType = DbType.DateTime;
myCommand.Parameters.Add( NewParm );
Then continue with your connection, open, execute and close...
DSN-less connection : (Note the driver and FIL string changed to what called in Windows 7)
It use Microsoft OLE DB Provider for ODBC Drivers (MSDASQL).
The code use the .Net Framework Data Provider for ODBC (System.Data.Odbc),
Not the .Net Framework Data Provider for OLEDB (System.Data.OleDb).
Clipper type N-> OdbcType Double
"Provider=MSDASQL.1;Persist Security Info=False;Mode=ReadWrite;Extended Properties=\"CollatingSequence=ASCII;DBQ=F:\\Folder;DefaultDir=F:\\Folder;Deleted=1;DRIVER=Microsoft Access dBASE Driver (*.dbf, *.ndx, *.mdx);DriverId=21;FIL=dBASE III;MaxBufferSize=2048;MaxScanRows=8;PageTimeout=600;SafeTransactions=0;Statistics=0;Threads=3;UID=admin;UserCommitSync=Yes;\";Initial Catalog=F:\\Folder";