We are experiencing an issue where our sql server where cpu usage jumps to and remains at 100% until the site is taken down and restarted. We have gone through the code and optimized everything we can, and this is still happening.
What we do with cache is run a query that loads an entire structure (~6000 rows) from sql server, store that in cache, and query that cache to perform the various operations we need to perform throughout the rest of the application (there are a lot of recursive operations that need to be performed on the data and it would be a huge hit to the sql server otherwise).
I describe the above because it seems that when sql cache dependency is used, we encounter the cpu spike on the sql server. If it is disabled, we no longer encounter the spike (on the sql server or the web server) even though we are still caching the same amount of data.
Does anyone have any idea what about sql cache dependency could cause behavior like this?
The sql server used is SQL Server 2008 R2. The web server is IIS 7.5 and we used ASP.NET 3.5. This web servers are set up as a cluster (x2), and they both point to the same sql server.
This is the code that loads/sets up the cache:
using (SqlConnection cn = new SqlConnection(ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString))
{
cn.Open();
string query =
#"SELECT
id,
parentid,
field1,
field2,
field3,
field4,
field5
FROM
dbo.theTableWithDataInIt";
SqlCommand cmd = new SqlCommand(query, cn);
cmd.Notification = null;
cmd.NotificationAutoEnlist = true;
SqlCacheDependencyAdmin.EnableNotifications(
ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString);
if (!SqlCacheDependencyAdmin.GetTablesEnabledForNotifications(
ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString)
.Contains("dbo.theTableWithDataInIt"))
{
SqlCacheDependencyAdmin.EnableTableForNotifications(
ConfigurationManager.ConnectionStrings["MyConnectionString"].
ConnectionString, "dbo.theTableWithDataInIt");
}
SqlCacheDependency dependency = new SqlCacheDependency(cmd);
cmd.ExecuteNonQuery();
//
// Get Cache Data is a function that returns a DataSet with the data to be added to cache
//
Cache.Insert("mycache", GetCacheData(), dependency);
}
The problem was resolved. It turns out the indexes somehow became corrupt or lost. Right clicking on the index in the table and selecting "Rebuild" solved the problem.
Related
I have an ASP.NET application importing data from a CSV file, and storing it to a (SQL Server) database table. Basically, the import process consists of:
Importing the raw CSV data into a corresponding SQL table (with the same columns)
"Merging" the data into the DB, with some sql clauses (INSERTS and UPDATE)
The whole import procedure is wrapped with a transaction.
using (SqlConnection c = new SqlConnection(cSqlHelper.GetConnectionString()))
{
c.Open();
SqlTransaction trans = c.BeginTransaction();
SqlCommand cmd = new SqlCommand("DELETE FROM T_TempCsvImport", c, trans);
cmd.ExecuteNonQuery();
// Other import SQL ...
trans.Commit();
}
Trying this import procedure from a virtual machine (everything is local), I got an error
[SqlException (0x80131904): Timeout. The timeout period elapsed prior to completion of the operation or the server is not responding.
Trying the same without the transaction, works fine.
Something I tried:
Executing the same queries from SQL Server Management Studio, all of them runs quite fast (500ms)
Executing from my development machine, works fine
Increasing the Command Timeout, I get the error anyhow. I also tried to set CommandTimeout to 0 (infinite), and the procedure seems to run "forever" (I get a server timeout, which I set to 10 minutes)
So, the final question is: why the SQL transaction is creating such problems? Why is it working without the transaction?
After several tests I did, I found out that the problem is ...Not Enough Memory!
What I found out is that my situation is exactly the same as this answer:
Help troubleshooting SqlException: Timeout expired on connection, in a non-load situation
I have both IIS and SQL server on my local machine, with the test running on a virtual machine. This virtual machine was using 2Gb of RAM, that is 50% of the total RAM of my PC. Reducing the RAM available to the virtual machine to 512Mb fixed the problem.
Furthermore, I noticed that using a transaction or not using it has exactly the same results, when the system is working, so my first guess was wrong, as well.
I am trying to run a non query using a Oracle connection in ASP C# with CLR 4.5. Here is my code:
string connectionString = ConfigurationManager.ConnectionStrings["OracleConnectionString1"].ConnectionString;
OracleConnection conn = new OracleConnection(connectionString);
conn.Open();
OracleCommand cmd = new OracleCommand();
cmd.Connection = conn;
cmd.CommandText = "update SALES_ADVENTUREWORKS2012.SALESORDERDETAIL set UNITPRICEDISCOUNT=0 where ROWGUID='4A399178-C0A0-447E-9973-6AB903B4AECD'";
cmd.CommandType = CommandType.Text;
cmd.CommandTimeout = QUERY_TIMEOUT;
int row_affected = cmd.ExecuteNonQuery();
HttpContext.Current.Response.Write("Rows affected:" + row_affected + "<br/>");
conn.Close();
when I run the query in oracle development tool, it works fine.
when I use the asp code above, it freezes when performing the query. It freezes forever even though I used a 5 second timeout.
I've tried using the managed and unmanaged oracle libraries; both behave the same.
Note that using the fill or scalar query work perfectly fine so there is nothing wrong with my connection string. Also the fact that oracle development can perform this update query proves that this is not a permission problem.
Any ideas?
Most likely your query is waiting to get access to the record. You probably have modified that row in "oracle development tool" and have not committed or rolled back that transaction.
Just commit/rollback in your tool or close open session.
You can check for open transactions in v$transaction view.
More on automatic locks in Oracle:
http://docs.oracle.com/cd/E11882_01/server.112/e41084/ap_locks001.htm
Are you certain you are using the 4.5 library? The 3.5 documentation states that the CommandTimeout property has no effect.
The 4.5 documentation suggests it should work, but the Remarks section doesn't mention the change, which warrants suspicion.
Otherwise, the code you posted doesn't seem to show where you actually set the value of QUERY_TIMEOUT to 5 seconds. If QUERY_TIMEOUT has a value of zero, then any other provider (SQLCommand, for example) would wait indefinitely. As vav suggested, locks from other sources could cause an indefinite wait.
I got an asp.net gridview connected to my sql database. When Inserting a new record or updating a record im doing some serverside checks and then either update/insert a record or do nothing. right now i got 2 methods CheckArtistExists and CheckSongExists which are both using a SqlConnection Object e.g.
public bool CheckSongExists(string _title, int _artistId)
{
int cnt = -1;
using (SqlConnection con = new SqlConnection(CS))
{
//check if song already is exists in DB
SqlCommand cmd = new SqlCommand("Select Count(ID) from tblSong WHERE Title = #newTitle AND ArtistId = #newArtistId;", con);
cmd.Parameters.AddWithValue(#"newTitle", _title);
cmd.Parameters.AddWithValue(#"newArtistId", _artistId);
con.Open();
cnt = (int)cmd.ExecuteScalar();
// if cnt ==1 song exists in DB, of cnt == 0 song doesnt exist
if(cnt == 1)
{ return true; }
else
{ return false; }
}
}
So for the Update function in the gridview i need to establish 3 SqlConnections (at max) one to check for the artist(if artist doesnt exist i have to insert a record to tblArtist first)
then a check if the song exists(only if artist exists) and finally if song doesnt exist I have to insert a new record.
I know database connections are valuable resources thats why i put them in a using block. So im not quite sure if its good style to use 3 SqlConnection objects to update/insert. Can you please tell me if my code is ok or if i should rather use another approach for this problem.
thank you
ADO.NET internally manages the underlying Connections to the DBMS in the ADO-NET Connection-Pool:
In practice, most applications use only one or a few different
configurations for connections. This means that during application
execution, many identical connections will be repeatedly opened and
closed. To minimize the cost of opening connections, ADO.NET uses an
optimization technique called connection pooling.
Connection pooling reduces the number of times that new connections
must be opened. The pooler maintains ownership of the physical
connection. It manages connections by keeping alive a set of active
connections for each given connection configuration. Whenever a user
calls Open on a connection, the pooler looks for an available
connection in the pool. If a pooled connection is available, it
returns it to the caller instead of opening a new connection. When the
application calls Close on the connection, the pooler returns it to
the pooled set of active connections instead of closing it. Once the
connection is returned to the pool, it is ready to be reused on the
next Open call.
So obviously there's no reason to avoid creating,opening or closing connections since actually they aren't created, opened and closed at all. This is "only" a flag for the connection pool to know when a connection can be reused or not. But it's a very important flag, because if a connection is "in use"(the connection pool assumes), a new physical connection must be openend to the DBMS what is very expensive.
So you're gaining no performance improvement if you "reuse" connections but the opposite.
Create, open(in case of Connections), use, close and dispose them where you need them(f.e. in a method)
use the using-statement to dispose and close(in case of Connections) implicitely
So yes, it's absolutely fine to use one connection per method since you are not using a physical connection at all if connection-pooling is enabled (default).
Another question is if you could improve your approach. You could create a stored-procedure which checks existence and updates or inserts accordingly.
Solutions for INSERT OR UPDATE on SQL Server
I have a SP which takes 20 seconds in SqlServer environment but sometimes in my ASP.NET page when I run the SP I get SqlServer timeout excaption.
I event set CommandTimeout and ConnectionTimeout to 60 but I still get the exception.
I would appreciate to help me.
Some other operation might be locking the table. set the timeout to a higher value and check.
while running the proc execute sp_lock and sp_who2 system procedure for any locking
You can try
cmd.CommandTimeout = 0;
if you are executing a query taking long time.
1) tried something like??
SqlCommand cmd = new SqlCommand("MyReport", conn);
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandTimeout = 3660; //// or set it zero (0)
2) and this???
3) Assuming your db server and point of execution are different, is your internet/intranet connectivity fine?
4) check for vpn connection (if used)
Execute the query from the SSMS and save the execution plan. Then run the application and have the SQL Profiler to capture the trace and then save the execution plan from profiler as mentioned in this link.
Compare the two execution plan to find out the actual difference in execution.
Check for parameter sniffing. If you still have the issue make sure the DB statistics are updated, sometimes this might be the issue after that drop and create the procedure.
I think the problem is the sending parameters from your application to store procedure.
try it again but this time use SQL Server Profiler to trace your query execution .
you can use TextData column value in SQL Server Profiler and run real executed query again to find the real problem.
I'm having difficulty with an SQL query against Server 2008 from IIS7. I have a VB.NET class library which runs an update statement. The underlying code used to create the connection hasn't changed, but suddenly the query is failing in our testing and development environments. It does, however, still work against the same server/database using the slightly older code in our production environment.
I've tried setting the connection timeout in the web.config and I'm at a loss to explain the cause.
The basic structure of the query is:
Dim conn = New SqlConnection()
conn.ConnectionString = "Data Source=someserver\sqlexpress2008;Initial Catalog=DatabaseName;User ID=sa;Password=pass"
conn.Open()
Using cmd As SqlCommand = conn.CreateCommand()
cmd.CommandText = "UPDATE ..."
cmd.Parameters.AddWithValue("#UName", user.name)
cmd.ExecuteNonQuery() 'fails with error
End Using
The error is:
A transport-level error has occurred when sending the request to the
server. (provider: TCP Provider, error: 0 - An existing connection was
forcibly closed by the remote host.)
I've tried restarting IIS and the SQL server and I'm totally out of ideas. I just need a fix
You need to open the connection before calling SqlCommand.ExecuteNonQuery(). You do this by calling SqlConnection.Open().
Dim conn = New SqlConnection()
conn.ConnectionString = "Data Source=someserver\sqlexpress2008;Initial Catalog=DatabaseName;User ID=sa;Password=pass"
Using cmd As SqlCommand = conn.CreateCommand()
cmd.CommandText = "UPDATE ..."
cmd.Parameters.AddWithValue("#UName", user.name)
conn.Open()
cmd.ExecuteNonQuery() 'fails with error
conn.Close()
End Using
Also, ensure you database isn't in single user mode.
This helped another person who was stuck recently. You could examine the problem from the database server by setting up a SQL Server Profiler.
You can find lots of info about SQL Profiler by just googling around. Here's a site with a video that might help you get started. For starters, you would be able to see if the request is even reaching the database server.
This was a nightmare to track down. It turned out to be cause by a horrible quirk in VB.NET. Nullable datetimes seem to be coerced to DateTime.MinValue, which resulted in a DateTime.MinValue being inserted into an sql datetime. The fix was to check for either !property.HasValue && property.Value != DateTime.MinValue when setting the parameters for the command.
This is a network-level error. The database server is killing the connection for some reason. In order to troubleshoot this, I would open a connection using SSMS to the DEV and TEST servers and make sure that I can run simple queries w/o problems. It's unlikely that the issue is your library since you would be getting timeout or some other kind of errors.
as Lcarus, said, database server is killing the connection for unknown reason.
you can check the logs, to verfiy. Log path will be C:\Program Files\Microsoft SQL Server\<your instance>\MSSQL\LOG
from MSDN Blog MSDN Blog
this will occur when A connection is taken from the connection pool,
the application does not know that the physical connection is gone, an
attempt to use it is done under the assumption that the physical
connection is still there.