SQLite.NET PCL Busy Exception - sqlite

We are using the SQLite.NET PCL in a Xamarin application.
When putting the database under pressure by doing inserts into multiple tables we are seeing BUSY exceptions being thrown.
Can anyone explain what the difference is between BUSY and LOCKED? And what causes the database to be BUSY?
Our code uses a single connection to the database created using the following code:
var connectionString = new SQLiteConnectionString(GetDefaultConnectionString(),
_databaseConfiguration.StoreTimeAsTicks);
var connectionWithLock = new SQLiteConnectionWithLock(new SQLitePlatformAndroid(), connectionString);
return new SQLiteAsyncConnection (() => { return connectionWithLock; });

So our problem turned out to be that although we had ensured within the class we'd written that it only created a single connection to the database we hadn't ensured that this class was a singleton, therefore we were still creating multiple connections to the database. Once we ensured it was a singleton then the busy errors stopped
What I've take from this is:
Locked means you have multiple threads trying to access the database, the code is inherently not thread safe.
Busy means you have a thread waiting on another thread to complete, your code is thread safe but you are seeing contention in using the database.

...current operation cannot proceed because the required resources are locked...
I am assuming that you are using async-style inserts and are on different threads and thus an insert is timing out waiting for the lock of a different insert to complete. You can use synchronous inserts to avoid this condition. I personally avoid this, when needed, by creating a FIFO queue and consuming that queue synchronously on a dedicated thread. You could also handle the condition by retrying your transaction X number of times before letting the Exception ripple up.
SQLiteBusyException is a special exception that is thrown whenever SQLite returns SQLITE_BUSY or SQLITE_IOERR_BLOCKED error code. These codes mean that the current operation cannot proceed because the required resources are locked.
When a timeout is set via SQLiteConnection.setBusyTimeout(long), SQLite will attempt to get the lock during the specified timeout before returning this error.
Ref: http://www.sqlite.org/lockingv3.html
Ref: http://sqlite.org/capi3ref.html#sqlite3_busy_timeout

I have applied the following solution which works in my case(mobile app).
Use sqlitepclraw.bundle_green nugget package with SqlitePCL.
Try to use the single connection throughout the app.
After creating the SQLiteConnection.
Apply busytime out using following call.
var connection = new SQLiteConnection(databasePath: path);
SQLite3.BusyTimeout(connection.Handle, 5000); // 5000 millisecond.

Related

The configured execution strategy 'SqlRetryingExecutionStrategy' does not support user-initiated transactions

I have ASP.Net 4.7.2 window service which is processing NServiceBus messages. Currently it is deployed to On-Premise server. It has retry mechanism as well and working fine. Now I am going to containerizing it. While running into docker window container, it is doing SQL operation using Entity framework and giving exception as mentioned below:
The configured execution strategy 'SqlRetryingExecutionStrategy' does not support user-initiated transactions. Use the execution strategy returned by 'DbContext.Database.CreateExecutionStrategy()' to execute all the operations in the transaction as a retriable unit.
While running locally by installing manually or on On-Premise server, it is working fine but in container it is throwing exception.
Can any one help me what can be the root cause?
It sounds like the piece of code does manual transaction management and is not wrapped within an execution strategy execute.
if your code initiates a transaction using BeginTransaction() you are defining your own group of operations that need to be treated as a unit, and everything inside the transaction would need to be played back shall a failure occur.
The solution is to manually invoke the execution strategy with a delegate representing everything that needs to be executed. If a transient failure occurs, the execution strategy will invoke the delegate again.
https://learn.microsoft.com/en-us/ef/core/miscellaneous/connection-resiliency#execution-strategies-and-transactions
using var db = new SomeContext();
var strategy = db.Database.CreateExecutionStrategy();
strategy.Execute(
() =>
{
using var context = new SomeContext();
using var transaction = context.Database.BeginTransaction();
context.SaveChanges();
transaction.Commit();
});
``

Cosmo ChangeFeed -Errors,exceptions and Service fail scenario's

All,
I am using Change Feed Processor Library.Want to know the best way to handle service failure along with the exceptions/errors scenario's in ProcessChangesAsync method. Below are the events am referring to.
1) Service failure - Service having the processor library crashed in the middle of some operation. How to start the process from the same document(doc on failure instance)? is there any inbuilt mechanism where change feed will start with the last failed documents? E.g. Let assume,in current batch we have 10 docs.5 processed successfully and then service breaks because of network failure or by some other reasons.Will my process starts with 6th document once service is re-started? How to achieve this?
2) Exception and Errors- Any errors in ProcessChangesAsync method can be handle using try catch at the global level but how to persist those failure records and make them available for the next batch? Again,looking for any available inbuilt mechanism in change feed process.
1) The Processor Library, by default, checkpoints after a successful run of ProcessChangesAsync. In the latest library version, you can customize the Checkpointer to do manual checkpoints in case you need it. If for some reason the processor shuts down before checkpointing, then it will start processing next from the the last successful checkpoint stored in the Leases collection. In your case, it will start with the first document again, so you will never lose a change but you could experience double processing (this is an "at least once" model).
2) There is no built-in mechanism that you can leverage, handling exceptions within the ProcessChangesAsync is your responsibility. You could not only add a global try/catch but, in the case you are looping over the documents, add a try/catch inside the loop, to handle a failing document (maybe send it to queue for later analysis/post-process) without losing the batch. If you require logging for those errors (I'm assuming that's what you mean by persisting errors?), then the latest version is compatible with LibLog, so plugging your own custom logging is as simple as:
using Microsoft.Azure.Documents.ChangeFeedProcessor.Logging;
var hostName = "SampleHost";
var tracelogProvider = new TraceLogProvider(); //You can use any provider supported by LibLog
using (tracelogProvider.OpenNestedContext(hostName))
{
LogProvider.SetCurrentLogProvider(tracelogProvider);
// After this, create IChangeFeedProcessor instance and start/stop it.
}
Source
Extra info for the comments
To avoid exceptions halting the batch or causing a batch to be reprocessed, you can have handling like this:
public async Task ProcessChangesAsync(IChangeFeedObserverContext context, IReadOnlyList<Document> documents, CancellationToken cancellationToken)
{
try
{
foreach(var document in documents)
{
try
{
// Do your work for the document
}
catch(Exception ex)
{
// Something happened with the current document, handle it, send it to a queue / another storage to analyze, log it. This catch will make the loop continue with the next.
}
}
}
catch(Exception ex)
{
// Something unhandled happened, log it and avoid throwing it again so the next batch is processed
}
}

Workflow hosted inside WorkflowApplication, aborting on persistence

I have been trying to resolve a somewhat intermittent issue working with a long running state machine running inside a WorkflowApplication. I can step through the workflow and this behaves as expected, transitioning through the states as expected, then a bookmark is reached which then persists the workflow. However, the workflow is then aborted and I get the following message:
The execution of an InstancePersistenceCommand was interrupted because the instance owner registration for owner ID 'ba26f4e9-f38b-4179-aa09-31ab9f8af337' has become invalid. This error indicates that the in-memory copy of all instances locked by this owner have become stale and should be discarded, along with the InstanceHandles. Typically, this error is best handled by restarting the host.
The Sql Instance store is initialised as follows:
SqlStore = new SqlWorkflowInstanceStore(ConfigurationManager.ConnectionStrings["SqlInstanceStore"].ConnectionString);
SqlStore.HostLockRenewalPeriod = TimeSpan.FromSeconds(15);
SqlStore.InstanceCompletionAction = InstanceCompletionAction.DeleteAll;
handle = SqlStore.CreateInstanceHandle();
InstanceView sqlView = SqlStore.Execute(handle, new CreateWorkflowOwnerCommand(), TimeSpan.FromSeconds(30));
SqlStore.DefaultInstanceOwner = sqlView.InstanceOwner;
WorkflowHost = new WorkflowApplication(WorkflowDefinition, inputs);
WorkflowHost.Run();
To create the bookmark:
context.CreateBookmark(bkmk, OnResume);
The exception doesn't really provide enough information to help troubleshooting this issue. Any help would be appreciated.
I managed to resolve this by using an overload of the instance store where I am now passing a new GUID. Need to understand this a bit more.
handle = SqlStore.CreateInstanceHandle(Guid.NewGuid());

How to deal with TransactionScope during debugging?

I have WebService that is hosted by ASP.NET web site. Inside the TransactionScope object is used to handle transactions:
using (TransactionScope scope = new TransactionScope())
{
...
scope.Complete();
}
The problem is that during debugging, when I am going through each line in step-by-step mode,
transaction timeout is occurred and any attempt to access DB crashed with '' error, and as a result: further debugging is prohibited.
How could I handle that without deleting mentioned lines of code?
P.S. I've tried to find, how to increase a time-out of created transaction, but didn't find something helpful.
Any thoughts are welcome.
Thanks.
You can specify an infinite timeout for the Transaction by passing in a zero length TimeSpan as part of the constructor:
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.Required, new TimeSpan(0)))
The TransactionScopeOption of Required is what is used as default with your parameterless constructor.
See http://msdn.microsoft.com/en-us/library/ms172152(VS.90).aspx for more information.

Error #3119: Database file is currently locked

I have developed to applications in flex. The one application constantly retrieves data from the internet, while the other can be opened and closed when you want, both apps use the same database. The problem is that at random I get an Error #3119: Database file is currently locked. Is it not possible to have two stable connections in a Adobe AIR environment? Anyone has any solutions?
I think not. Not at once.
I know this is a really old question, but I ran into this issue myself and found a solution for it for those who may come across this. I hope this helps someone, because I know for me, all I could find on this topic was false information, like that given by Konrad. You can, in fact have multiple open database connections. Actually, in my application, I have an asynchronous connection used for writing data to the database (INSERT, UPDATE, DELETE), and a synchronous, read-only connection for reading from the database. On the asynchronous connection, for every execute, I always get an immediate lock by putting all statements in a transaction using
conn.begin(SQLTransactionLockType.IMMEDIATE);
This will allow you to read from the database while writing to it with another connection. Where I ran into a problem is when trying to read from the database from one connection after committing this async statement and before it actually finished writing the data. So, even though the documentation for SQLTransactionLockType.IMMEDIATE states you can still do reads while it is locked, you actually cannot while another statement is actively in the process of writing data.
I got around this by writing my own execute for the synchronous connection. It simply tries to execute, and if it fails due to Error #3119, try again until you succeed. Between each function call, the data will continue to be written to the database and eventually will no longer be busy. Here is the code for that function:
public static function execute(stmt:SQLStatement):void {
try {
stmt.execute();
} catch (e:SQLError) {
if(e.errorID == 3119) {
execute(stmt);
} else {
trace(e.details + "\n" + e.getStackTrace());
if(stmt.sqlConnection != null && stmt.sqlConnection.inTransaction) {
stmt.sqlConnection.rollback();
}
}
}
}
Another gotcha to watch out for with this error (if you're an idiot like me anyway) is to check if you've got the SQLite db file open in a db browser, which can lock the database, and cause this error (and hours of googling and irritation).

Resources