Scenario:
I have a stored procedure for student seat allotment and there are more than 8000 students being allotted using this stored procedure, and it is working fine when I am executing it from SQL Server 2014.
But when I am calling the same stored procedure from my .net program, it is terminating in the middle of execution. My stored procedure contains more than 500 of lines so I'm not putting its code here. But the .net code is as follows:
protected void btnAllotment_Click(object sender, EventArgs e)
{
try
{
DataClassesAdmission2015DataContext dc = new DataClassesAdmission2015DataContext();
dc.Allotment(ChkReshuffle.Checked, ddlCriteria.SelectedItem.Text, ddlRound.SelectedItem.Text,null);
Response.Write("<script>alert('Allotment is done Successfully')</script>");
dc.Connection.Close();
}
catch (Exception ex)
{
Response.Write("<script>alert('Error at Alloment <br>" + ex.ToString() + "')</script>");
}
}
Note: I'm not getting any error here but store procedure terminating in background. So If any one have any suggestion or solution than please let me know.
I Solved it using CommandTimeout property. Actually When I was calling my stored procedure from dc object than it was taking around 15 minutes to complete its task. That's way I was unable to trace it because default CommandTimeout is 30 seconds as MSDN.
using(DataClassesAdmission2015DataContext dc = new DataClassesAdmission2015DataContext())
{
dc.CommandTimeout = 60*60;
dc.Allotment(ChkReshuffle.Checked, ddlCriteria.SelectedItem.Text, ddlRound.SelectedItem.Text, null);
}
Now it is working for me.
Related
I'm using Lucene in asp-core project. But every 2-4 days my index is broken. So i logged the exceptions and got the following stacktrace:
2017-09-06 10:13:50.8338|An unhandled exception has occurred: Lock
obtain timed out:
SimpleFSLock#e:\inetpub\Static_Data\GKHUB\lucene_index\write.lock:
System.IO.IOException: lockFile
'e:\inetpub\Static_Data\GKHUB\lucene_index\write.lock' alredy
exists.EXCEPTION OCCURRED:Lucene.Net.Store.LockObtainFailedException
After some researches i found out, that
This exception is thrown when the write.lock could not be acquired. This happens when a writer tries to open an index that another writer already has open.[ src ]
So the second Update-Request couldn't update the index. But why is the whole index broken afterwards?
My Code doesn't do anything special:
public void UpdateIndex(Document doc, int idToUpadate)
{
var indexwriterConfig = new IndexWriterConfig(LuceneVersion.LUCENE_48, Analyzer);
indexwriterConfig.WriteLockTimeout = 5000; // doesn't fix the problem
using (var writer = new IndexWriter(GetLuceneDirectory, indexwriterConfig)) {
try {
writer.UpdateDocument(new Term("Id", idToUpadate.ToString()), doc);
writer.Commit();
}
catch (Exception e) {
Debug.WriteLine(e);
writer.Rollback();
}
}
}
As you already mention in the question:
LockObtainFailedException is thrown when the write.lock could not be
acquired. This happens when a writer tries to open an index that
another writer already has open.
That's happening if you have multiple updates, in you code, you're creating multiple instances of the IndexWriter that are trying to obtain the lock on the index. You should try to re-use a single writer instead of closing and opening/creating a new one. This should solve your problem.
Also, do not forget, that
IndexWriter instances are completely thread safe, meaning multiple
threads can call any of its methods, concurrently
I am doing a project in wicket
How to solve the problem.
I came across such a message:
WicketMessage: Can't instantiate page using constructor public itucs.blg361.g03.HomePage()
Root cause:
java.lang.UnsupportedOperationException: [SQLITE_BUSY] The database file is locked (database is locked)
at itucs.blg361.g03.CategoryEvents.CategoryEventCollection.getCategoryEvents(CategoryEventCollection.java:41)
public List<CategoryEvent> getCategoryEvents() {
List<CategoryEvent> categoryEvents = new
LinkedList<CategoryEvent>();
try {
String query = "SELECT id, name, group_id"
+ " FROM event_category";
Statement statement = this.db.createStatement();
ResultSet result = statement.executeQuery(query);
while (result.next()) {
int id = result.getInt("id");
String name = result.getString("name");
int group_id = result.getInt("group_id");
categoryEvents.add(new CategoryEvent(id, name, group_id));
}
} catch (SQLException ex) {
throw new UnsupportedOperationException(ex.getMessage());
}
return categoryEvents;
}
at itucs.blg361.g03.HomePage.(HomePage.java:71)
categories = categoryCollection.getCategoryEvents();
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
Sqlite allows only one writer to the whole database at a time and, unless you selected "WAL" journal mode, no reader while writing. Moreover unless you explicitly ask it to wait, it simply returns the SQLITE_BUSY status for any attempt to access the database while conflicting operation is running.
You can tell sqlite to wait for the database to become available for a specified amount of time. The C-level API is sqlite3_busy_timeout; I never used sqlite from Java though, so I don't know where to find it there.
(...) tell sqlite to wait for the database to become available for specified amount of time.
In order to do it from Java, run the following statement just like a simple SQL statement:
pragma busy_timeout=30000; -- Busy timeout set to 30000 milliseconds
I'm having a problem closing my database before an attempt to delete the file. The code is just
myconnection.Close();
File.Delete(filename);
And the Delete throws an exception that the file is still in use. I've re-tried the Delete() in the debugger after a few minutes, so it's not a timing issue.
I have transaction code but it doesn't run at all before the Close() call. So I'm fairly sure it's not an open transaction. The sql commands between open and close are just selects.
ProcMon shows my program and my antivirus looking at the database file. It does not show my program releasing the db file after the close().
Visual Studio 2010, C#, System.Data.SQLite version 1.0.77.0, Win7
I saw a two year old bug just like this but the changelog says it's fixed.
Is there anything else I can check? Is there a way to get a list of any open commands or transactions?
New, working code:
db.Close();
GC.Collect(); // yes, really release the db
bool worked = false;
int tries = 1;
while ((tries < 4) && (!worked))
{
try
{
Thread.Sleep(tries * 100);
File.Delete(filename);
worked = true;
}
catch (IOException e) // delete only throws this on locking
{
tries++;
}
}
if (!worked)
throw new IOException("Unable to close file" + filename);
Encountered the same problem a while ago while writing a DB abstraction layer for C# and I never actually got around to finding out what the issue was. I just ended up throwing an exception when you attempted to delete a SQLite DB using my library.
Anyway, this afternoon I was looking through it all again and figured I would try and find out why it was doing that once and for all, so here is what I've found so far.
What happens when you call SQLiteConnection.Close() is that (along with a number of checks and other things) the SQLiteConnectionHandle that points to the SQLite database instance is disposed. This is done through a call to SQLiteConnectionHandle.Dispose(), however this doesn't actually release the pointer until the CLR's Garbage Collector performs some garbage collection. Since SQLiteConnectionHandle overrides the CriticalHandle.ReleaseHandle() function to call sqlite3_close_interop() (through another function) this does not close the database.
From my point of view this is a very bad way to do things since the programmer is not actually certain when the database gets closed, but that is the way it has been done so I guess we have to live with it for now, or commit a few changes to System.Data.SQLite. Any volunteers are welcome to do so, unfortunately I am out of time to do so before next year.
TL;DR
The solution is to force a GC after your call to SQLiteConnection.Close() and before your call to File.Delete().
Here is the sample code:
string filename = "testFile.db";
SQLiteConnection connection = new SQLiteConnection("Data Source=" + filename + ";Version=3;");
connection.Close();
GC.Collect();
GC.WaitForPendingFinalizers();
File.Delete(filename);
Good luck with it, and I hope it helps
Just GC.Collect() didn't work for me.
I had to add GC.WaitForPendingFinalizers() after GC.Collect() in order to proceed with the file deletion.
Had a similar issue, though the garbage collector solution didn't fix it.
Found disposing of SQLiteCommand and SQLiteDataReader objects after use saved me using the garbage collector at all.
SQLiteCommand command = new SQLiteCommand(sql, db);
command.ExecuteNonQuery();
command.Dispose();
The following worked for me:
MySQLiteConnection.Close();
SQLite.SQLiteConnection.ClearAllPools()
More info:
Connections are pooled by SQLite in order to improve performance.It means when you call Close method on a connection object, connection to database may still be alive (in the background) so that next Open method become faster.When you known that you don't want a new connection anymore, calling ClearAllPools closes all the connections which are alive in the background and file handle(s?) to the db file get released.Then db file may get removed, deleted or used by another process.
In my case I was creating SQLiteCommand objects without explicitly disposing them.
var command = connection.CreateCommand();
command.CommandText = commandText;
value = command.ExecuteScalar();
I wrapped my command in a using statement and it fixed my issue.
static public class SqliteExtensions
{
public static object ExecuteScalar(this SQLiteConnection connection, string commandText)
{
using (var command = connection.CreateCommand())
{
command.CommandText = commandText;
return command.ExecuteScalar();
}
}
}
The using statement ensures that Dispose is called even if an exception occurs.
Then it's a lot easier to execute commands as well.
value = connection.ExecuteScalar(commandText)
// Command object created and disposed
I was having a similar problem, I've tried the solution with GC.Collect but, as noted, it can take a long time before the file becomes not locked.
I've found an alternative solution that involves the disposal of the underlying SQLiteCommands in the TableAdapters, see this answer for additional information.
I've been having the same problem with EF and System.Data.Sqlite.
For me I found SQLiteConnection.ClearAllPools() and GC.Collect() would reduce how often the file locking would happen but it would still occasionally happen (Around 1% of the time).
I've been investigating and it seems to be that some SQLiteCommands that EF creates aren't disposed and still have their Connection property set to the closed connection. I tried disposing these but Entity Framework would then throw an exception during the next DbContext read - it seems EF sometimes still uses them after connection closed.
My solution was to ensure the Connection property is set to Null when the connection closes on these SQLiteCommands. This seems to be enough to release the file lock. I've been testing the below code and not seen any file lock issues after a few thousand tests:
public static class ClearSQLiteCommandConnectionHelper
{
private static readonly List<SQLiteCommand> OpenCommands = new List<SQLiteCommand>();
public static void Initialise()
{
SQLiteConnection.Changed += SqLiteConnectionOnChanged;
}
private static void SqLiteConnectionOnChanged(object sender, ConnectionEventArgs connectionEventArgs)
{
if (connectionEventArgs.EventType == SQLiteConnectionEventType.NewCommand && connectionEventArgs.Command is SQLiteCommand)
{
OpenCommands.Add((SQLiteCommand)connectionEventArgs.Command);
}
else if (connectionEventArgs.EventType == SQLiteConnectionEventType.DisposingCommand && connectionEventArgs.Command is SQLiteCommand)
{
OpenCommands.Remove((SQLiteCommand)connectionEventArgs.Command);
}
if (connectionEventArgs.EventType == SQLiteConnectionEventType.Closed)
{
var commands = OpenCommands.ToList();
foreach (var cmd in commands)
{
if (cmd.Connection == null)
{
OpenCommands.Remove(cmd);
}
else if (cmd.Connection.State == ConnectionState.Closed)
{
cmd.Connection = null;
OpenCommands.Remove(cmd);
}
}
}
}
}
To use just call ClearSQLiteCommandConnectionHelper.Initialise(); at the start of application load.
This will then keep a list of active commands and will set their Connection to Null when they point to a connection that is closed.
Try this... this one tries all the above codes... worked for me
Reader.Close()
connection.Close()
GC.Collect()
GC.WaitForPendingFinalizers()
command.Dispose()
SQLite.SQLiteConnection.ClearAllPools()
Hope that helps
Use GC.WaitForPendingFinalizers()
Example:
Con.Close();
GC.Collect();`
GC.WaitForPendingFinalizers();
File.Delete(Environment.CurrentDirectory + "\\DATABASENAME.DB");
I believe the call to SQLite.SQLiteConnection.ClearAllPools() is the cleanest solution. As far as I know it is not proper to manually call GC.Collect() in the WPF environment. Although, I did not notice the problem until I have upgraded to System.Data.SQLite 1.0.99.0 in 3/2016
Had a similar problem. Calling Garbage Collector didn't help me. LAter I found a way to solve the problem
Author also wrote that he did SELECT queries to that database before trying to delete it. I have the same situation.
I have the following code:
SQLiteConnection bc;
string sql;
var cmd = new SQLiteCommand(sql, bc);
SQLiteDataReader reader = cmd.ExecuteReader();
reader.Read();
reader.Close(); // when I added that string, the problem became solved.
Also, I don't need to close database connection and to call Garbage Collector. All I had to do is to close reader which was created while executing SELECT query
Best answer that worked for me.
dbConnection.Close();
System.Data.SQLite.SQLiteConnection.ClearAllPools();
GC.Collect();
GC.WaitForPendingFinalizers();
File.Delete(Environment.CurrentDirectory + "\\DATABASENAME.DB");
The reason for this seems to be a feature called "Pooling".
Appending "Pooling=false" to the connection string causes the DB-File to be released with "connection.Close()".
See the FAQ on connection pooling here:
https://www.devart.com/dotconnect/sqlite/docs/FAQ.html#q54
I was struggling with the similar problem. Shame on me... I finally realized that Reader was not closed. For some reason I was thinking that the Reader will be closed when corresponding connection is closed. Obviously, GC.Collect() didn't work for me.
Wrapping the Reader with "using: statement is also a good idea. Here is a quick test code.
static void Main(string[] args)
{
try
{
var dbPath = "myTestDb.db";
ExecuteTestCommand(dbPath);
File.Delete(dbPath);
Console.WriteLine("DB removed");
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
Console.Read();
}
private static void ExecuteTestCommand(string dbPath)
{
using (var connection = new SQLiteConnection("Data Source=" + dbPath + ";"))
{
using (var command = connection.CreateCommand())
{
command.CommandText = "PRAGMA integrity_check";
connection.Open();
var reader = command.ExecuteReader();
if (reader.Read())
Console.WriteLine(reader.GetString(0));
//without next line database file will remain locked
reader.Close();
}
}
}
Maybe you don't need to deal with GC at all. Please, check if all sqlite3_prepare is finalized.
For each sqlite3_prepare, you need a correspondent sqlite3_finalize.
If you don't finalize correctly, sqlite3_close will not close the connection.
This works for me but i noticed sometimes journal files -wal -shm are not deleted when the process is closed. If you want SQLite to remove -wal -shm files when all connection are close the last connection closed MUST BE non-readonly. Hope this will help someone.
I was using SQLite 1.0.101.0 with EF6 and having trouble with the file being locked after all connections and entities disposed.
This got worse with updates from the EF keeping the database locked after they had completed.
GC.Collect() was the only workaround that helped and I was beginning to despair.
In desperation, I tried Oliver Wickenden's ClearSQLiteCommandConnectionHelper (see his answer of 8 July). Fantastic. All locking problems gone!
Thanks Oliver.
Waiting for Garbage Collector may not release the database all time and that happened to me. When some type of Exception occurs in SQLite database for example trying to insert a row with existing value for PrimaryKey it will hold the database file until you dispose it. Following code catches SQLite exception and cancels problematic command.
SQLiteCommand insertCommand = connection.CreateCommand();
try {
// some insert parameters
insertCommand.ExecuteNonQuery();
} catch (SQLiteException exception) {
insertCommand.Cancel();
insertCommand.Dispose();
}
If you not handle problematic commands' exceptions than Garbage Collector cannot do anything about them because there are some unhandled exceptions about these commands so they are not garbage. This handling method worked well for me with waiting for garbage collector.
We are running a reporting web application that allows the user to select a few fields and a crystal report is generated based off of the fields selected. The SQL that is generated for the most complex report will return the data in < 5 seconds, however it takes the report and average of 3 minutes to run, sometimes longer causing a time out. We are running VS2010. The reports are basically set up out of the box with no real manipulations or computations being done, just displaying the data in a nice format. Is there anything we can try to speed it up, pre-loading a dummy report to load the dlls, some hack to make crystal run faster, anything?
EDIT: Code Added to show the databinding
protected void Page_Load(object sender, EventArgs e)
{
if (!Page.IsPostBack)
{
string strFile = Server.MapPath(#"AwardStatus.rpt");
CrystalReportSource1.Report.FileName = strFile;
DataTable main = Main();
CrystalReportSource1.ReportDocument.SetDataSource(main);
CrystalReportViewer1.HasCrystalLogo = false;
CrystalReportSource1.ReportDocument.ExportToHttpResponse(CrystalDecisions.Shared.ExportFormatType.PortableDocFormat, Response, false, "pmperformance");
}
}
private DataTable Main()
{
Guid guidOffice = Office;
CMS.Model.ReportsTableAdapters.ViewACTableAdapter rptAdapter = new CMS.Model.ReportsTableAdapters.ViewACTableAdapter();
Reports.ViewAwardedContractsDataTable main = new Reports.ViewAwardedContractsDataTable();
if (Office == new Guid())
{
IEnumerable<DataRow> data = rptAdapter.GetData().Where(d => UserPermissions.HasAccessToOrg(d.guidFromId, AuthenticatedUser.PersonID)).Select(d => d);
foreach (var row in data)
{
main.ImportRow(row);
}
}
else if (guidOffice != new Guid())
{
main = rptAdapter.GetDataByOffice(guidOffice);
}
else
{
main = new Reports.ViewACDataTable();
}
return main;
}
private Guid Office
{
get
{
string strOffice = Request.QueryString["Office"];
Guid guidOffice = BaseControl.ParseGuid(strOffice);
if (!UserPermissions.HasAccessToOrg(guidOffice, AuthenticatedUser.PersonID))
{
return Guid.Empty;
}
else
{
return guidOffice;
}
}
}
protected void CrystalReportSource1_DataBinding(object sender, EventArgs e)
{
//TODO
}
This may be a bit flippant, but possibly consider not using crystal reports... We had a fair bit of trouble with them recently (out of memory errors being one), and we've moved off to other options and are quite happy...
Here's what I would do:
Put clocks from the time you get the field choices from the user, all the way to when you display the report. See where your processing time is going up.
When you look at the clocks, there can be various situations:
If Crystal Reports is taking time to fill the report, check how you're filling it. If you're linking the report fields directly to your data table, CR is probably taking time looking up the data. I suggest creating a new table (t_rpt) with dynamic columns (Field1, Field2,..FieldN) and pointing your report template to that table. I don't know if you're already doing this.
If it's taking time for you to lookup the data itself, I suggest creating a view of your table. Even though a memory hog, this will make your lookup quick and you can delete the view once you're done.
If it's none of the above, let us know what your clocks show.
In terms of loading any large amount of data, you'll always want to use a stored procedure.
Outside of that, you WILL see a delay in the report running the first time the Crystal DLLs load. Yes, you can preload them as you mentioned and that will help some.
Can you guys let me know the way of handling transactions in asp.net?
i.e. I have a few queries (Present in different functions and called under various situations) that have to be executed as a whole. So, how should I go about it?
Not sure of the syntax and the method/practice for writing the statements in .net (commit, rollback etc).
Kindly let me know. Also, plz point me to some good articles if possible. Thanks!!!
I recommend using TransactionScope, because you can use it no mater what DB you are using. You can even do distributed transactions (operations against multiple databases within the same transaction) with it.
You can refer to a link for a code example, but in general, you do this:
try
{
using (TransactionScope scope = new TransactionScope())
{
using (MySqlConnection connection1 = new MySqlConnection (connectionString))
{
// Opening the connection automatically enlists it in the
// TransactionScope as a lightweight transaction.
connection1.Open();
// create the DB commands and perform the DB operations
.
.
.
// The Complete method commits the transaction. If an exception has been thrown,
// Complete is not called and the transaction is rolled back.
scope.Complete();
}
}
}
catch (Exception e)
{
// something went wrong, handle the exception accordingly. Note
// that since we did not call TransactionScope.Complete, nothing
// gets committed to the DB.
}
Here's another starter for TransactionScope: Implementing an Implicit Transaction using Transaction Scope
Don't know much about TransactionScope, but I just use the normal IDbTransaction like this:
IDbConnection conn = null;
IDbCommand cmd = null;
IDbTransaction tran = null;
try
{
conn = DatabaseUtil.GetConnection(); //Get the connection somehow
cmd = conn.CreateCommand();
tran = conn.BeginTransaction();
cmd.Transaction = tran;
//Do your DB Work
tran.Commit();
}
catch (SystemException ex)
{
tran.Rollback();
}
finally
{
if (conn != null) conn.Close();
}
With the IDb classes you are DB independent too to a certain degree.
If its a local transaction you can also use ado.net's transaction object. TransactionScope will handle distributed transactions if needed but requires MSDTC to be configured if a transaction is promoted to a distributed transaction.
http://msdn.microsoft.com/en-us/library/2k2hy99x.aspx
Both are in the System.Transactions Namespace http://msdn.microsoft.com/en-us/library/system.transactions.aspx