InvalidOperationException: Timeout - recursive DB access - asp.net

currently I have a serious problem with one of my web applications which runs into a Timeout Exception around half a dozen times a day.
Error: "The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached".
After a lot of googling I found out that the problem has something to do with unclosed connections. So I checked all functions that access the database in any way until I stumbled upon this one:
Private Sub getOrgas(ByVal orgID As String)
Dim Id = orgID
orgColl.Add(Id)
While (Not IsNothing(Id))
Dim conn = Database.DbWrapper.GetConnection(1, Integration.Mandanten.DatabaseType.AddonSQL)
Dim paras As New HashSet(Of System.Data.Common.DbParameter)
Dim orgatmp As String
paras.Add(New SqlClient.SqlParameter("#Id", orgID))
Dim dr = Database.DbWrapper.GetDataReaderFromStoredProcedure("stp_Orgas_Get", paras, conn)
While dr.Read
If Not valueInColl(CStr(dr(0))) Then
orgatmp = dr(0).ToString
orgColl.Add(orgatmp)
getOrgas(orgatmp)
End If
End While
dr.Close()
conn.Close()
Id = Nothing
End While
End Sub
As you can see this function executes a stored procedure and runs the results through a while loop where it calls the function again if a specific condition -valueInColl-. Now in that way it is possible that there are 20 or more open connections. It has nothing to do with the timeout-value which is set via the GetDataReaderFromStoredProcedure to 600 which actually should be enough. To be sure I doubled the value and will roll it out this evening. I'll see whether that helped within the next day then.
I believe the problem is that there are too many open connections at the same time, because of the recursive function, but I have no clue how to solve this.
I couldn't find anything as to how to edit the max connections. I'm not even entirely sure where have to set it. Is it the IIS, the DB itself or is it a programming-parameter (VB.net/ASP.NET).
Would be nice if you guys could help me out here.
[EDIT]
Ok, somebody had the idea to reuse the connection variable, but this won't work as the datareader is still running. As long as it is not closed I can't reuse the connection in any way and I can't close the datareader, because I might lose data if I do so. The while-loop for dr.read hasn't ended, yet ..
On the other hand I deleted the (pretty much useless) outer while and used an If-clause in exchange:
Private Sub getOrgas(ByVal orgID As String, ByVal con As DbConnection)
Dim Id = orgID
Dim conn As DbConnection
Dim tmpOrga As String
orgColl.Add(Id)
If Not IsNothing(Id) Then
If IsNothing(con) Then
conn = Database.DbWrapper.GetConnection(1, Integration.Mandanten.DatabaseType.AddonSQL)
Else
conn = con
End If
Dim paras As New HashSet(Of System.Data.Common.DbParameter)
paras.Add(New SqlClient.SqlParameter("#Id", orgID))
Dim dr = Database.DbWrapper.GetDataReaderFromStoredProcedure("stp_Orgas_Get", paras, conn)
While dr.Read
If Not valueInColl(CStr(dr(0))) Then
tmpOrga = dr(0).ToString
orgColl.Add(tmpOrga)
getOrgas(tmpOrga, conn)
End If
End While
dr.close()
conn.Close()
Id = Nothing
End If
End Sub

Is there any reason you cannot refactor things so that each recursion uses the same db connection?
I am not a VB coder but I would tackle it as follows
Change getOrgas() to take a connection parameter defaulting to 'nothing'.
Change the Dim conn line to if IsNothing(connParameter) conn = GetConnection() else conn := connParameter;
Change your recursion line to getOrgas(orgatmp, conn);
Test the F%%%% out of it.
I have just noticed the outer While Loop. Is it there just to confuse you ? How many times will it execute ? ...
I did wonder about the datareader -
try this - I see that your datareader needs to close before you recurse, so close it.
in pseudocode -
dim locallist = new list();
while dr.read
{
LocalList.Add dr.thing;
}
dr.close;
foreach(thing in locallist)
{
if Not ValueInColl(thing) Then
CallYourFunctionTRecursively()
end if;
}
Are you with me ?
If you are trying to put together all the members of a family then it depends which database system you are using how it is done, but look up 'Heirarchical queries' in your documentation.

Related

SqlDataAdaptor update and close connection - how can I close connection via data Adaptor

I have the following code:
Dim da As SqlDataAdapter
Dim rst As DataRow = MyrstEdit("SELECT * FROM tblHotels WHERE ID = 19",, da).Rows(0)
rst("HotelName") = "My Cool"
rst("FirstName") = "Albert"
rst("City") = "Edmonton"
da.Update(rst.Table)
So, the above is nice and short. And it works rather nice.
And of course this being asp.net, then centralizing things like connection code (that I don't have to create over and over) is a also rather nice. And why bother with a connection during coding, so above reduces coding workload.
How then can I ensure the connection object is disposed and closed correctly?
From reading, since I do not open the conneciton, then
The Fill() does open, then close.
And I have to assume that the da.Update() ALSO must then by logic open, then close the conneciton.
However, I still should clean up the connection object after above is done.
Question:
Is disposing of the sql data adaptor object sufficient here to also dispose the connection object that the data adaptor is using?
eg:
da.Dispose()
The MyRstEdit routine is this:
Public Function MyrstEdit(strSQL As String,
Optional strCon As String = "",
Optional ByRef oReader As SqlDataAdapter = Nothing) As DataTable
' can pass custom connection string - if not passed, then default
If strCon = "" Then
strCon = GetConstr() ' global func to get application con string
End If
Dim mycon As New SqlConnection(strCon)
oReader = New SqlDataAdapter(strSQL, mycon)
Dim rstData As New DataTable
Dim cmdBuilder = New SqlCommandBuilder(oReader)
Try
oReader.Fill(rstData)
oReader.AcceptChangesDuringUpdate = True
Catch
End Try
Return rstData
End Function
So, the return sqlDataAdaptor object must be holding the connection, since the scope in above routine is limited.
So do I have to dispose the connection object?
Can I do this with the sqlAdaptor?
I can't dispose/close the connection object in above sub, since my da.Update() in the calling code still needs that connection until I do the update.
How then can I ensure the connection object is disposed and closed correctly?
Don't worry about it; it's not your job. DataAdapter makes it, DataAdapter will clean it up
However, I still should clean up the connection object after above is done
No, for the aforementioned reason
Is disposing of the sql data adaptor object sufficient here to also dispose the connection object that the data adaptor is using?
Yes, unless you have good reason to believe that Microsoft's code has a critical flaw and their classes will benefit from your code micromanaging the resources they create..
You can also read the reference source if you want to know what they do internally
The MyRstEdit routine is this:
It's hard to understand why it exists in that form; you'd be better off just passing a datatable around and creating dataadapters as and when you need them. MyRstEdit isn't well named; it doesn't seem to edit anything, it always overwrites the passed in adapter with stuff that a any passed in adapter might already know (the connstr and sql) and then doesn't really do anything that couldn't just be put to
Using da As New SqlDataAdapter("SELECT * FROM tblHotels WHERE ID = 19", GetConStr())
Dim dt as New DataTable
da.Fill(dt)
Dim rst = dt(0)
rst("HotelName") = "My Cool"
rst("FirstName") = "Albert"
rst("City") = "Edmonton"
New SqlCommandBuilder(da)
da.Update(rst.Table)
End Using
About the most useful thing it does is apply a command builder, but that's only a single line and only needed for an update..
Perhaps you could create an extension method that applies to a DataAdapter, that gets the first row, so you could say:
Using da As SqlDataAdapter("SELECT * FROM tblHotels WHERE ID = 19", GetConStr())
Dim rst = da.FirstRow()
rst("HotelName") = "My Cool"
rst("FirstName") = "Albert"
rst("City") = "Edmonton"
New SqlCommandBuilder(da)
da.Update(rst.Table)
End Using
But it isn't saving much over "just using it how MS intended", and there is still that block of "data columns accessed by string and hence intellisense can't help" in the middle.
If you're looking to find ways of making your SqlDataAdapter life easier it might be an opportunity to skip it and have a look at Dapper instead; a set of extension methods on a SqlConnection that map classes to and from:
Using c as New SqlConnection(GetConStr())
Dim h = Await c.SingleAsync(Of Hotel)("SELECT * FROM tblHotels WHERE ID = #id", New With { .ID = 19 } )
h.HotelName = "My Cool"
h.FirstName = "Albert"
h.City = "Edmonton"
Await c.ExecuteAsync("UPDATE tblHotels SET HotelName = #HotelName, FirstName = #FirstName, City = #City WHERE ID = #ID", h)
End Using
You still have to write the queries, but it's a one time op, or you could look at something like Dapper Contrib.. The main use here is that instead of being some DataRow object you access by "string column names" you have a first class VB.NET object - Hotel, with named typed proeprties, and dapper can create them from SQL queries and put their values directly into SQL parameters (another thing that is absent from your existing code)

How I do a WCF service with two methods: one that waits and insert data into the DB and another to poll the DB?

This is the scenario:
I'm doing a web service in WCF and a client application in ASP.NET. The language is VB.NET.
I want a method than send "jobs" to the server. I'm trying to simulate this kind of jobs as long running processes that wait up to 20-30 seconds with thread sleeping and inserting a row into a JOBS table in the DB.
I want another method that polls the database every 5 seconds with client postbacks to get the jobs lists from the JOBS table (finished and currently running)
This is what i've got so far, both methods are working, except when a long running job is sleeping, then the client can't retrieve the list of jobs with another call. I've tried "PerCall" in the but it didn't work out. I suspect the thread is somewhat locking the service or maybe I have to use async calls. I'm using HttpBasicBinding at web.config ... I'm a little lost and I have not found any code doing something similar.
Here is the code in the service side:
iServiceJobs.vb
<ServiceContract()>
Public Interface IServiceJobs
<OperationContract(IsOneWay:=True)>
Sub SendJob(ByVal runTime As Integer, ByVal id As String)
<OperationContract>
Function GetJobsList() As List(Of ClassJob)
ServiceJobs.vb
Public Class ServiceJobs Implements IServiceJobs
Public SendJob(ByVal runTime As Integer, ByVal id As String) Implements IServiceJobs.SendJob
Dim connStr As String = "..."
Dim conn As New OracleConnection(connStr)
conn.Open()
Dim query As String = "INSERT INTO JOBS(...)"
Dim cmd As New OracleCommand(query, conn)
cmd.ExecuteNonQuery()
Threading.Thread.Sleep(runTime* 1000)
cmd = New OracleCommand(query, conn)
cmd.ExecuteNonQuery()
End Sub
Public Function GetJobList() As List(Of ClassJob) Implements IServiceJobs.GetJobsList
Try
Dim jobList As New List(Of ClassJobs)
Dim connStr As String = "..."
Dim conn As New OracleConnection(connStr)
conn.Open()
Dim query As String = "SELECT * FROM JOBS"
Dim cmd As New OracleCommand(query, conn)
Dim dr As OracleDataReader
Dim job As ClassJob
dr = cmd.ExecuteReader
While dr.Read
job = New ClassJob
job.id = dr(0)
job.lock = dr(3)
...
jobList.Add(job)
job = Nothing
End While
Return jobList
Catch ex As Exception
Return Nothing
End Try
End Function
End Class
The code in the client is very simple, just two buttons, one with a ws call for insert jobs and another to get the job list.
I'm open to any suggestion on how to do a better implementation of this scenario.
EDIT:
I've tried
<ServiceBehavior(ConcurrencyMode:=ConcurrencyMode.Multiple, InstanceContextMode:=InstanceContextMode.PerCall)>
But it doesn't work, GetJobList() returns Nothing while SendJob() is working on this thread.
SOLVED: Apparently, I was using an outdated version of Oracle libraries. When I used Oracle.ManagedDataAccess from NuGet everything went OK.
SOLVED: Apparently, I was using an outdated version of Oracle libraries. When I used Oracle.ManagedDataAccess from NuGet everything went OK.

Query about Oracle Connections

I seem to be getting intermittent problems with my Oracle connection indicating something about a problem with semaphores which suggest that Oracle is somehow holding onto the connections instead of removing them after they have been used.
Here's the code and connection string I use:
Connection string: user id=user;password=password;data source=dataSource; Validate Connection=true;Min Pool Size=10;Connection Lifetime=5;Connection Timeout=60;Incr Pool Size=5;
And the code I use is this:
Dim OracleConn As New OracleConnection()
Dim DataTable As DataTable
Dim queryOracle As OracleCommand
Dim OracleDataAdapter As OracleDataAdapter
Dim connStr As String = "user id=user;password=password;data source=dataSource; Validate Connection=true;Min Pool Size=10;Connection Lifetime=5;Connection Timeout=60;Incr Pool Size=5;"
OracleConn.ConnectionString = connStr
Try
OracleConn.Open()
queryOracle = OracleConn.CreateCommand()
queryOracle.CommandText = "select * from table1"
DataTable = New DataTable()
OracleDataAdapter = New OracleDataAdapter(queryOracle)
OracleDataAdapter.Fill(DataTable)
table1.DataSource = DataTable.DefaultView
table1.DataBind()
Catch OracleEx As OracleException
Throw
Catch ex As Exception
Throw
Finally
If Not OracleConn Is Nothing And OracleConn.State = ConnectionState.Open Then
OracleConn.Close()
End If
End Try
Now my questions are:
Is this the best way of doing this?
I only "Close" my connection do I need to "Dispose" of it also?
I'm using Oracle.DataAccess.Client by the way.
Any help will be much appreciated
Try to put everything between Using.
Using oracleConn as OracleConnection = new OracleConnection()
'Your stuff goes here
End Using
the same goes for Commands.
P.S. There's is no need to catch Exception if they are just thrown again.
I suggest you to use using block (Execute Dispose in the end)
Using connection As New OracleConnection()
....
End Using
2 An application can call Close more than one time. No exception is generated.
If you called Dispose method SqlConnection object state will be reset. If you try to call any method on disposed SqlConnection object, you will receive exception.

Reducing SQL connections to just 1 - ASP.net VB

I am currently working on an asp.net web page with a GridView displaying a table from a database. This GridView has 4 DropDownLists that will be used to filter the data shown on the GridView. When the page loads 4 Sub routines are run, each one connecting to the database with a select statement to fill the DropDownList with relevant filter headings.
Initially, I had one connection with a loop that populated all of the drop downs but these contained duplicates. I then split the filling of each DDL so that the select statements could contain DISTINCT.
I would like (and am sure there is a way here) to be able to populate all of the DDLs with data from one connection.
Code for one connection:
Protected Sub FillDepDDL()
Dim conn As New SqlConnection()
conn.ConnectionString = WebConfigurationManager.ConnectionStrings("TestDBConnectionString").ConnectionString
Dim connection As New SqlConnection(conn.ConnectionString)
connection.Open()
Const FillAllQS As String = "SELECT DISTINCT [Department] FROM [Employees]"
Dim command As New SqlCommand(FillAllQS, connection)
Dim reader As SqlDataReader = command.ExecuteReader()
Dim sel As New ListItem
sel.Text = "Please Select"
sel.Value = "*"
DDLDepartment.Items.Add(sel)
While reader.Read
Dim Deplist As New ListItem()
Deplist.Value = reader("Department")
Deplist.Text = reader("Department")
DDLDepartment.Items.Add(Deplist)
End While
reader.Close()
conn.Close()
End Sub
The other 3 column names: FirstName > DDLFN, LastName > DDLLN, Wage > DDLWag.
This is only a test DB and the princibles learned here will be applied to a larger live project.
I'm sure some guru will be able to work this out easily but I just can't get my head round it even after hours of searching.
Thanks in advance.
I'm adding this in as answer because I cannot format it in a comment, but this doesn't answer the original question of how to write the sql to return all three distinct result sets. Instead, it answers how to rewrite the code you have above so that connections are properly disposed of in case of an exception.
Protected Sub FillDepDDL()
Dim Deplist As ListItem
Dim sel As New ListItem
sel.Text = "Please Select"
sel.Value = "*"
DDLDepartment.Items.Add(sel)
Using conn As New SqlConnection(WebConfigurationManager.ConnecitonString("TestDBConnectionString").ConnectionString)
Using cmd As New SqlCommand("SELECT DISTINCT [Department] FROM [Employees]", conn)
conn.Open()
Using reader = cmd.ExecuteReader()
While reader.Read
Deplist = New ListItem()
Deplist.Value = reader("Department")
Deplist.Text = reader("Department")
DDLDepartment.Items.Add(Deplist)
End While
End Using
End Using
End Using
End Sub
I don't see any reason for you to try to return all three results in a single query. That will just make your code unnecessarily complicated just to save a millisecond or two. Connection pooling handles the creation of connections on the database server for you, so opening a new connection in your code is very fast.

.NET Framework Data Provider for Oracle multiple open connection

I have the below mentioned code in a seperate class file for establishing connection and carry out DB transactions. I have an issue where multiple connections being opened which sometime exceed the connection pool. When I stepped through the code I found that there are codes which call ConnectDB() in a loop without calling DisconnectDB(). But I expected that the condition OraConn.State = ConnectionState.Closed should handle the situation. Somehow the condition is always satisfied hence openning another set of connection. Can you suggest where am I going wrong and also what best practice can be adopted here?
Public Class Connection
Dim Str_conn As String = "Data Source=...; User=...; password=...; Min Pool Size=10; Max Pool Size=500;"
Public OraConn As OracleConnection
Dim cmd As OracleCommand
Dim dr As OracleDataReader
Dim data_adapt As OracleDataAdapter
Dim dt As DataTable
Dim ds As DataSet
Public Sub ConnectDB()
OraConn = New OracleConnection(Str_conn)
If OraConn.State = ConnectionState.Closed Then
OraConn.Open()
End If
End Sub
Public Sub DisconnectDB()
If OraConn.State = ConnectionState.Open Then
OraConn.Close()
End If
End Sub
Public Function get_dataset(ByVal query As String, ByRef ds As DataSet) As DataSet
data_adapt = New OracleDataAdapter(query, OraConn)
data_adapt.Fill(ds)
Return ds
End Function
Public Function get_datareader(ByVal query As String) As OracleDataReader
cmd = New OracleCommand(query, OraConn)
dr = cmd.ExecuteReader()
Return dr
End Function
Public Sub UpdateDB(ByVal query As String)
cmd = New OracleCommand(query, OraConn)
cmd.ExecuteNonQuery()
cmd.Dispose()
End Sub
The class is refered in other classes or directly in the aspx.vb pages like this.
Public Function InsertData(ByVal var1 As String, ByVal var2 As String) As Integer
conn.ConnectDB()
Dim qryInsert As String
qryInsert = " INSERT INTO TABLE VALUES ('" & var1 & "', "
qryInsert = qryInsert & var2 & "')"
Try
conn.UpdateDB(qryInsert)
Catch ex As OracleException
If ex.Code = 1 Then
updData(var1, var2)
ElseIf ex.Code = 2091 Then
msgprompt("Duplicate Unique Key!", "Warning")
End If
Finally
conn.DisconnectDB()
End Try
Return count
End Function
The connection is again opened in function updData(). While I understand that it has to be closed correctly but keeping tab on every developer is not possible. Hence I want to control it directly from the connection class by using the same connection but the condition If OraConn.State = ConnectionState.Closed is not helping.
UPDATE
I have put the code in UpdateDB under a Using block and removed call to ConnectDB and DisconnectDB from function like InsertData(...). It seems that the issue has been resolved. But I would like to know in case of exception will the connection remain open? and also OraConn is a public variable defined outside Using block so will it be disposed of by the GC?
Public Sub UpdateDB(ByVal query As String)
Using OraConn = New OracleConnection(Str_conn)
cmd = New OracleCommand(query, OraConn)
Try
OraConn.Open()
cmd.ExecuteNonQuery()
Catch ex As Exception
Throw
Finally
cmd.Dispose()
End Try
End Using
End Sub
You must close all the connections as soon as you are done with it, no matter what.
Suggestion:
The best practice for closing the connection is to do it in finally block. So that even if there is any error, catch it (log it if required) in catch block, and then connection will get close in finally block.
UPDATE
You can put one private static counter in your Connection class. When ever ConnectDB() is called you increment this counter and decrement it in every DisconnectDB(). Now in ConnectDB() you check the value of counter, if it exceeds a minimum threshold you throw error, by doing this way; you can come to know idle connection present in your code and refactor it. On production keep this threshold value high or ignore it in code.

Resources