I am trying to maximize the benefits from an experience.
Also I usually use Enterprise library logging block, I log errors and a portion of statistical information into the database, because it is centralized place to track errors, if database logging failed, Normally it goes to Event Log.
Tracing messages should go into file:
Which choice you believe we should go
1- Only Some tracing messages can be left in code if there is a complex algorithm or unstable module.
OR
2- We should not keep any tracing messages in code, clean it up as soon as bug is resolved.
For database.
I think that Errors raised from SP and functions should be logged into another table in the database, and that exactly what is done by AdventureWorksLT2008 database.
Is it a bad idea to log database events directly to Enterprise library Log table without raising this errors to next tier. I think it is more fixable, because I can put more custom information in the message. of course some errors will not be handled and will reach the next tier.
Any ideas, or comments, something else you do. something you want to clarify.
Thanks
Are you talking about catching errors and logging directly in T-SQL and not then doing RAISERROR to get it to the caller?
I think that's a viable strategy for certain kinds of issues - for instance, if an SP wants to find a problem and correct it silently and simply issue a warning.
But the kind of issues it would apply to might not be terribly frequent.
The kind of things I would think about are things like unusual cases where unexpected UPDATEs are done instead of INSERTs? Or where data already exists so is not generated. Or in a deployment or build script which skips an existing table, etc.
What if your database has performance issues and SP/functions start timing out - logging the error to the database may not work?
Related
Our site has been running for a few weeks in Azure without getting this error:
SqlException: Database 'database' on server 'server' is not currently
available. Please retry the connection later. If the problem
persists, contact customer support, and provide them the session
tracing ID of 'guid'.
It finally got that one day when there were a little over 2K of active (concurrent) users. This is the closest question that I can find in SO. We are not using EF though but rather we're using Dapper. I'm out of ideas how to debug our application to find out what caused the issue, and it's even harder now that the issue has not come up for the past 2 days. I definitely need to be on the lookout and I need you guys, any tip, on where I should be looking into, what I need to do to determine the cause of the issue, and possibly fix it.
It sounds like you need to handle transient failures via some sort of transient fault handling mechanism. Here is post asking a similar question:
SQL Azure Database retry logic David's answer is similar to the approach we took do deal with the issue.
Here is another link to some code that is similar to the David's and our solution to get your head around it. http://www.getcodesamples.com/src/4A7E4E66/41D6FAD
We had similar issues when we first moved to SQL Azure but by implementing back-off retry logic for the transient connection issues the majority of the time it recovers after a few seconds.
We went down the path of handling transient errors with the Azure Transient Fault Block, but this caused bigger issues - namely, if you reach the SQL connection limit (easy to do), having retry logic in place only makes things worse.
If it only happens once a month, I'd leave it be, and just handle it gracefully higher up the stack. An alternative is to create a custom retry policy to avoid retrying on certain errors, but it may still do more harm than good.
Having deployed a new build of an ASP.NET site in a production environment, I am logging dozens of data errors every second, almost always with the error "Cannot find table 0." We use datasets and frequently refer to Table[0], and while I understand the defensive coding practice of checking the dataset for tables before accessing Table[0], it's never been a problem in the past. A certain page will load fine one second, and then be missing one of its data-driven components the next. Just seeing if this rings a bell for anyone.
More detail: I used a different build server this time, and while I imagine the compiler settings are the same on both, I have a hard time thinking that there's a switch that makes 50% of my database calls come back with no tables. I also switched the project to VS 2008, but then reverted all of those changes when I switched back to VS 2005. I notice that the built assembly has a new MyLibrary.XmlSerializers.dll, where it didn't used to, but I also can't imagine that that's causing all the trouble. (It also doesn't fall down on calls to MyLibrary, or at least no more than any other time.)
Updated to add: I've discovered that the troublesome build is a "Release" build, where the working build was compiled as "Debug". Could that explain it?
Rolling back to the build before these changes fixed it. (Rebooting the SQL Server, the step we tried before that, did not.)
The trouble also seems to be load-based - this cruised through our integration and QA environments without a problem, and even our smoke test environment - the one that points to production data - is fine under light load.
Does this have the distinguishing characteristics of anything you might have seen in the past?
Bumping this old question because we have encountered the same issue and perhaps our solution would give more insight in what causes this.
Essentially this problem occurs in a production environment that is under very heavy load in a Windows service that uses multiple threads to process several jobs simultaneously (100 users use the same DB via ASP.NET web app and there are about 60 transactions/second on older hardware with SQL Server 2000).
No variables are shared, that is connections are opened anew, transaction is started, operations executed, transaction committed and connection closes.
Under heavy load sometimes one of the following exceptions occurs:
NullReferenceException: Object reference not set to an instance of an
object.
at System.Data.SqlClient.SqlInternalConnectionTds.get_IsLockedForBulkCopy()
or
System.Data.SqlClient.SqlException:
The server failed to resume the transaction. Desc:3400000178
or
New request is not allowed to start because it should come with valid transaction descriptor
or
This SqlTransaction has completed; it is no longer usable
It seems somehow the connection that is within the pool becomes corrupted and remains associated with previously used transactions. Furthermore, if such connection is retrieved from pool then sqlAdapter.Fill(dataset) results in an empty dataset, causing "Cannot find table 0". Because our service would retry the operation (reading job list) on failure and it would always get the same corrupt connection from the pool it would fail with this error until restarted.
We removed the issue by using SqlConnection.ClearPool(connection) on exception to make sure this connection is discarded from the pool and restructuring the application so less threads access the same resources simultaneously.
I have no clue who exactly caused this issue so I am not sure we have really fixed that, maybe just made it so rare it had not occurred again yet.
I've fought precisely this error message before. The key is that an underlying data method is swallowing a timeout exception.
You're probably doing something like this:
var table = GetEmployeeDataSet().Tables[0];
GetEmployeeDataSet is swallowing an exception, probably a timeout exception, which is why it only happens sporadically - it happens under load. You need to do the following to fix it:
Modify the underlying code to not swallow the exception, but rather let it bubble up to the next level so you can identify it properly.
Identify the query(s) causing the problem, and then rewrite, reindex, denormalize or throw hardware at the problem. See this for more info: System.Data.SqlClient.SqlException: Timeout expired
I've seen something similar. I believe our problem had to do with failed sessions being re-used (once the session object failed it went into a poor state and could not recover.) We fixed it by increasing the memory for the session pool and increasing the frequency of the web application recycling.
It also was "caused" by a new version that at first blush did not seem to have any change to cause such an effect. However, eventually it became clear that the logic of the program was opening and closing a lot more connections (maybe 20% more) than it used to. This small change pushed the limit of our prior configuration.
You might check the SQL Server logs for errors. Or, the Web server event log. It sounds like your connection pool could be out of open connections or your db could be out.
Which database calls changed between versions?
The error is obviously telling you one of your database calls isn't returning any data on occasion; I can't think of any cases where a code/assembly issue would cause it.
I have seen something like this when doing something with nHibernate Sessions in a non-thread-safe manner. That would explain why you only see it under load. Would need to see your code to guess at what isn't thread-safe though.
I inherited an application with a lot of stored procedures, and many of them have exception handling code that inserts a row in an error table and sends a DBMail. We have ELMAH on the ASP.NET side, so I'm wondering if exception management in the stored procs is necessary. But before I rip it out, I want to ensure that I'm not making a grave mistake because of ignorance about a best practice.
Only one application uses the stored procedures.
When would one prefer using exception management in a SQL Server 2005 stored procedure over handling the exception on the ASP.NET side?
If there are other applications utilizing these stored procedures then it might make sense to retain the error handling in the stored procedures. In your edit you indicate that this is not the case so removing the exception handling is probably not a bad idea.
In the MSDN article Exception Handling it is outlined when to catch exceptions and when to let them bubble up the stack. It can be argued that it makes sense to handle and log database exceptions that are recoverable from in the stored procedure.
There is a principle sometimes referred to as "First Failure Data Capture" - ie. it's the responsibility of the first "chunk of code" that indentifies an error to immediately capture it for future diagnosis. In multi-tier architectures this leads to some interesting questions about who "first" actually is.
I believe that it's quite reasonable for the stored procedure to log something to a db (sending an email sounds somewhet overkill for all but the most critical of errors but that's another issue). It cannot assume taht higher layers will be well behaved, you may only have one client now, but you can't predict the future.
The stored procedure can still throw an exception as well as logging. And sometimes in difficult situations being able to correlate errors in the different layers is actually very handy.
I would take a lot of persuading to remove that error logging.
I believe that logging to a table only works for simpler systems where everything is done within a single stored procedure call.
Once the system is complex enough that you implement transactions across database calls, logging to the database within the stored procedure becomes much more of a problem.
The rollbacks undo the logging to the table.
Logic that allows rollbacks and logging, in my opinion, creates too much potential for defects.
I have a Flex 3 app which I want to instrument to report errors generated by the app to a server via simple HTTPService call.
My idea is to wrap all the methods in try ... catch blocks which then pass the Error object to the reportError() function (which then fires off the HTTP request and pops up a dialog) but is there a better way?
I have implemented a system such as the one you suggest, wrapping all of my methods in try/catch and sending the stack trace to a service that emails me the errors. I created a basic format for the error that logs which method the error occurred in. I noticed that sometimes I end up getting null from the stack traces, so I wanted to log that information for these situations.
It GREATLY improved my application. I tracked down a (large) handful of errors and released a much cleaner build to my users. Now I don't ever get the emails.
The better way IMO is something like this.
I've no idea how good is this particular project (aside from this spooky GPL license), but I don't see why logging in action script should be any different from J2EE, C++, or say Python. Yes, it has some sand box security issues, but I think if this solved, you could log into some centralized log server..
Unfortunately, there really isn't -- errors don't bubble up in such a way as to be trappable at a global level, so the only real way you have to catch errors is to try and catch them all manually. (The community's been pretty vocal in asking for a global exception-handling feature for a while, but it's not there yet.)
For Asp.net web applications, is it best to:
trap errors within sql stored procedures and test for a return value in the code or
just let the error occur in sql (dont handle it) and rely on ado.net raising the errors within the code.
What are the best practises here?
A general rule that applies here is to catch the error as near the source as possible. SQL Server now has "try ... catch ..." error trapping syntax. So use it. The overhead of the little bit of extra code is insignificant, and if you have multiple statements in your SP, you can adapt the string in RAISERROR to help localize the problem.
In the interface, it shouldn't be difficult to trap the SP error event and handle it the same way you handle other error trapping in your procedural code.
This is one of the more neglected "best practices" in stored procedures, and it's even more important than in "regular" code because it's trickier to use a step=through debugger.
One useful pattern is to handle this in your SP the same way you it expect it to be handled in any other opaque SDK library.
According this article, this is a type of "boneheaded" exception - i.e. if an error occurs in the SP, this means that there's something wrong with the SP or the data itself.
My advise would be to trap the error in aps.net, as there you have much more possibilities to log the error, as well as all the parameters passed to the SP in order to investigate the problem.
I use try/catch on all potentially error generating calls to libraries or other servers including database queries. I'm primarily a developer not a DBA so I handle the error in the language that I'm most proficient in, C# not SQL. I'm sure that'll vary for every programmer. Not handling the error is never fine in my book.
I prefer both -
the stored procedure calls RAISEERROR, which manifests as an exception in ADO.NET
the stored procedure returns an error code, in case one sproc is calling another
in general, I always make db calls inside a try-catch
The most likely failure mode in the data layer is bad user input vales such as requesting a record that doesn't exist. Most other errors are caused by defective code and usually are caught right away when testing. I like to catch the db error and and always throw a custom error with more information about the data and context of the request. Then the error bubbles back up the call stack and if it was a user correctable error I can alert them to that fact. Otherwise the central error page handler for the application will be called when the error reaches the top of the stack. The worst thing to do is to catch an error and not let it bubble back up. Return codes are outdated constructs unless talking to com components or foreign systems.
Ideally, your web application isn't aware of the back-end or data access. So it shouldn't be handling sql-related errors - those should be handled by the data access layer. But the web application needs to handle failure gracefully, by providing the end-user with a friendly message.
I would say it depends on what you are doing in your stored proc and what you want to do with certain errors that occur. Sometimes I handle it in the sp but other times i let it raise up to the data layer code.
Bear in mind, sometimes you can miss errors when using sql server 2005 try/catch, see my post on this. Whereas, in code (C# in my case) you can access all the errors in the SqlErrorCollection object.
Just be absolutely certain that your error handling in the stored procedure is well thought out and any uncertainties are left to be passed up to the data layer code where you must log everything.
The answer is that you need to do both. Some errors are actually triggered downline from the Database engine. Their source varies, of course, according to the actual way you connect to the database (OLBC, OLEDB etc). You've got to find a way of dealing with these. Some errors such as Deadlock errors out to be handled at the application level too.
As well as errors, it is a good idea to receive and deal with messages from the SQL Server Database Engine. These are very similar to errors and can give the application a lot of useful diagnostic information. If you’re using System.Data.SQLClient, you’ll need to create a SqlInfoMessageEventHandler delegate, identifying the method that handles the event, to listen for the InfoMessage event on the SqlConnection class. You’ll find that message-context information such as severity and state are passed as arguments to the callback, because from the system perspective, these messages are just like errors.
I hope this helps!