I have an ASP.NET application importing data from a CSV file, and storing it to a (SQL Server) database table. Basically, the import process consists of:
Importing the raw CSV data into a corresponding SQL table (with the same columns)
"Merging" the data into the DB, with some sql clauses (INSERTS and UPDATE)
The whole import procedure is wrapped with a transaction.
using (SqlConnection c = new SqlConnection(cSqlHelper.GetConnectionString()))
{
c.Open();
SqlTransaction trans = c.BeginTransaction();
SqlCommand cmd = new SqlCommand("DELETE FROM T_TempCsvImport", c, trans);
cmd.ExecuteNonQuery();
// Other import SQL ...
trans.Commit();
}
Trying this import procedure from a virtual machine (everything is local), I got an error
[SqlException (0x80131904): Timeout. The timeout period elapsed prior to completion of the operation or the server is not responding.
Trying the same without the transaction, works fine.
Something I tried:
Executing the same queries from SQL Server Management Studio, all of them runs quite fast (500ms)
Executing from my development machine, works fine
Increasing the Command Timeout, I get the error anyhow. I also tried to set CommandTimeout to 0 (infinite), and the procedure seems to run "forever" (I get a server timeout, which I set to 10 minutes)
So, the final question is: why the SQL transaction is creating such problems? Why is it working without the transaction?
After several tests I did, I found out that the problem is ...Not Enough Memory!
What I found out is that my situation is exactly the same as this answer:
Help troubleshooting SqlException: Timeout expired on connection, in a non-load situation
I have both IIS and SQL server on my local machine, with the test running on a virtual machine. This virtual machine was using 2Gb of RAM, that is 50% of the total RAM of my PC. Reducing the RAM available to the virtual machine to 512Mb fixed the problem.
Furthermore, I noticed that using a transaction or not using it has exactly the same results, when the system is working, so my first guess was wrong, as well.
Related
I'm using SQLite.net in my WFC project to write files to the DB on the local disk.
I thought I was getting slow writing performance because there was a new connection open for every db operation.
I tried using connection pooling in the following manner:
string dbConnectionString = $"Data Source={dbConfigPath};Version=3;Pooling=True;Max Pool Size=1000;";
However I'm not seeing any improvement.
Am I using it wrong? Is there a way to make sure the pooling is working?
After some more debugging I realized the main bottleneck came from the WCF service that I was using to run the insert queries.
I have a asp.net application deployed to Azure websites connecting to Azure SQL Database. This has been working fine for the last year, but last weekend I have started getting errors connecting to the database giving the following stacktrace.
[Win32Exception (0x80004005): Access is denied]
[SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)]
This error comes and goes staying a few hours and then goes away for a few hours. The database is always accessible from my machine.
A few things that I have tried are:
Adding a "allow all" firewall rule (0.0.0.0-255.255.255.255) has no effect
Changing the connection string to use the database owner as credential has no effect.
what does have effect is to change the azure hosting level to something else. This resolves the issue temporarily and the website can access the database for a few hours more.
What could have have happened for this error to start showing up? The application hasn't been changed since August.
EDIT:
Azure support found that there was socket and port exhaustion on the instance. What the root cause for that is, is still unkown.
Craig is correct in that you need to implement SQLAzure Transient Fault Handling. You can find instructions here: https://msdn.microsoft.com/en-us/library/hh680899(v=pandp.50).aspx
From the article
You can instantiate a PolicyRetry object and wrap the calls that you
make to SQL Azure using the ExecuteAction method using the methods
show in the previous topics. However, the block also includes direct
support for working with SQL Azure through the ReliableSqlConnection
class.
The following code snippet shows an example of how to open a reliable
connection to SQL Azure.
using Microsoft.Practices.EnterpriseLibrary.WindowsAzure.TransientFaultHandling;
using Microsoft.Practices.EnterpriseLibrary.WindowsAzure.TransientFaultHandling.AzureStorage;
using Microsoft.Practices.EnterpriseLibrary.WindowsAzure.TransientFaultHandling.SqlAzure;
...
// Get an instance of the RetryManager class.
var retryManager = EnterpriseLibraryContainer.Current.GetInstance<RetryManager>();
// Create a retry policy that uses a default retry strategy from the
// configuration.
var retryPolicy = retryManager.GetDefaultSqlConnectionRetryPolicy();
using (ReliableSqlConnection conn =
new ReliableSqlConnection(connString, retryPolicy))
{
// Attempt to open a connection using the retry policy specified
// when the constructor is invoked.
conn.Open();
// ... execute SQL queries against this connection ...
}
The following code snippet shows an example of how to execute a SQL command with retries.
using Microsoft.Practices.EnterpriseLibrary.WindowsAzure.TransientFaultHandling;
using Microsoft.Practices.EnterpriseLibrary.WindowsAzure.TransientFaultHandling.AzureStorage;
using Microsoft.Practices.EnterpriseLibrary.WindowsAzure.TransientFaultHandling.SqlAzure;
using System.Data;
...
using (ReliableSqlConnection conn = new ReliableSqlConnection(connString, retryPolicy))
{
conn.Open();
IDbCommand selectCommand = conn.CreateCommand();
selectCommand.CommandText =
"UPDATE Application SET [DateUpdated] = getdate()";
// Execute the above query using a retry-aware ExecuteCommand method which
// will automatically retry if the query has failed (or connection was
// dropped).
int recordsAffected = conn.ExecuteCommand(selectCommand, retryPolicy);
}
The answer from Azure support indicated that the connection issues I was experiencing was due to port/socket exhaustion. This was probably caused by another website on the same hosting plan.
Some answers to why the symptoms were removed by changing hosting service level:
Changing the hosting plan helped for a while since this moved the virtual machine and closed all sockets.
Changing the hosting plan from level B to level S helped since azure limits the number of sockets on level B.
I have a SP which takes 20 seconds in SqlServer environment but sometimes in my ASP.NET page when I run the SP I get SqlServer timeout excaption.
I event set CommandTimeout and ConnectionTimeout to 60 but I still get the exception.
I would appreciate to help me.
Some other operation might be locking the table. set the timeout to a higher value and check.
while running the proc execute sp_lock and sp_who2 system procedure for any locking
You can try
cmd.CommandTimeout = 0;
if you are executing a query taking long time.
1) tried something like??
SqlCommand cmd = new SqlCommand("MyReport", conn);
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandTimeout = 3660; //// or set it zero (0)
2) and this???
3) Assuming your db server and point of execution are different, is your internet/intranet connectivity fine?
4) check for vpn connection (if used)
Execute the query from the SSMS and save the execution plan. Then run the application and have the SQL Profiler to capture the trace and then save the execution plan from profiler as mentioned in this link.
Compare the two execution plan to find out the actual difference in execution.
Check for parameter sniffing. If you still have the issue make sure the DB statistics are updated, sometimes this might be the issue after that drop and create the procedure.
I think the problem is the sending parameters from your application to store procedure.
try it again but this time use SQL Server Profiler to trace your query execution .
you can use TextData column value in SQL Server Profiler and run real executed query again to find the real problem.
We are experiencing an issue where our sql server where cpu usage jumps to and remains at 100% until the site is taken down and restarted. We have gone through the code and optimized everything we can, and this is still happening.
What we do with cache is run a query that loads an entire structure (~6000 rows) from sql server, store that in cache, and query that cache to perform the various operations we need to perform throughout the rest of the application (there are a lot of recursive operations that need to be performed on the data and it would be a huge hit to the sql server otherwise).
I describe the above because it seems that when sql cache dependency is used, we encounter the cpu spike on the sql server. If it is disabled, we no longer encounter the spike (on the sql server or the web server) even though we are still caching the same amount of data.
Does anyone have any idea what about sql cache dependency could cause behavior like this?
The sql server used is SQL Server 2008 R2. The web server is IIS 7.5 and we used ASP.NET 3.5. This web servers are set up as a cluster (x2), and they both point to the same sql server.
This is the code that loads/sets up the cache:
using (SqlConnection cn = new SqlConnection(ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString))
{
cn.Open();
string query =
#"SELECT
id,
parentid,
field1,
field2,
field3,
field4,
field5
FROM
dbo.theTableWithDataInIt";
SqlCommand cmd = new SqlCommand(query, cn);
cmd.Notification = null;
cmd.NotificationAutoEnlist = true;
SqlCacheDependencyAdmin.EnableNotifications(
ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString);
if (!SqlCacheDependencyAdmin.GetTablesEnabledForNotifications(
ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString)
.Contains("dbo.theTableWithDataInIt"))
{
SqlCacheDependencyAdmin.EnableTableForNotifications(
ConfigurationManager.ConnectionStrings["MyConnectionString"].
ConnectionString, "dbo.theTableWithDataInIt");
}
SqlCacheDependency dependency = new SqlCacheDependency(cmd);
cmd.ExecuteNonQuery();
//
// Get Cache Data is a function that returns a DataSet with the data to be added to cache
//
Cache.Insert("mycache", GetCacheData(), dependency);
}
The problem was resolved. It turns out the indexes somehow became corrupt or lost. Right clicking on the index in the table and selecting "Rebuild" solved the problem.
We recently launched a new web site... there are roughly ~150 users active during peak hours. During peak hours, we are experiencing an issue every few minutes, the exception text is listed below.
System.Web.HttpUnhandledException:
Exception of type 'System.Web.HttpUnhandledException' was thrown.
---> System.Data.SqlClient.SqlException: The client was unable to establish a connection because of an error during connection initialization process before login.
Possible causes include the following:
the client tried to connect to an unsupported version of SQL Server;
the server was too busy to accept new connections;
or there was a resource limitation (insufficient memory or maximum allowed connections) on the server. (provider: Named Pipes Provider, error: 0 - No process is on the other end of the pipe.)
Our data access layer calls various DataTableAdapters using the following syntax.
EDIT
Yes, da is the name assigned to the DataTableAdapter. There is no connection.Open() because the DataTableAdapter takes care of all that, right?
using(TheDataLayer.some.strongly.typedNameTableAdapters.suchAndSuchTableAdapter da = new TheDataLayer.some.strongly.typedNameTableAdapters.suchAndSuchTableAdapter())
{
StronglyTyped.DataTable dt = new StronglyTyped.DataTable();
da.FillByVariousArguments(dt, ..., ...);
//da.Dispose();
return something;
}
The connection string looks something like:
<add name="MyConnectionString"
connectionString="Data Source=myDBServerName;Initial Catalog=MyDB;User ID=MyUserName;Password=MyPassword"
providerName="System.Data.SqlClient" />
I'm trying to rule the problem being in Code. Is there anything "simple" that can be done to minimize this issue?
Thanks.
Have you tried "Connection Pooling" directly in connection string settings?
Example:
connectionString="....;Pooling=true;Min Pool Size=1;Max Pool Size=10;..."
You can read more info here: http://msdn.microsoft.com/en-us/library/8xx3tyca%28v=vs.71%29.aspx
Without seeing the code that actually opens and uses the connection, it's hard to say where the problem is.
Please update your question with what happens when you create that DataAdapter (I'm guessing that's what da means).
Also, if you're using the using statement, you shouldn't be disposing of the thing you created the using statement for.
We had similar issue which only happenes in our production environment and it was particularly associated with load. During busy time of day we would recieve several of the above mentioned exception.
We gone through a massive investigation around why this exception occurs and did a lot of changes to fix the issue. The defacto change we did which aleviated the problem was connection pool setting by setting min pool size to 1 and max pool size to 10. (It can vary based on your situation)
This issue will be more prevalent when you have several i.e. 1000's of Customer DB and use default connection string (i.e. database=DBName;server=ServerName). We were not explicitly setting min/max pool size hence it took default settings which set Min pool size to 0 and max pool size to 100.
Again, I dont have concrete proof but the theory is that during busy time of the day based on load it made several connection to DB server and DB server was bombarded with a lot of connection request at single point to several databases. Either Application server or DB server did have bandwidth to handle that many connection in a short period of time. Also, it was happening with server with most databases. Though we did not see a lot of connection at a time but Application server was not able to make connection to databases for a short duration when it had surge of requests going in.
After we set min pool size we aliveated this problem as there is atleast one connection to each database which is available all the time and if there is blast of request which required to make connection to several databases we already had atleast one connection to the database available before we request a new one.
Maybe unrelated to the actual problem you were facing, but this error is also thrown if you are trying to connect without specifying the correct port along with the database server name.