I'm using SQLite.net in my WFC project to write files to the DB on the local disk.
I thought I was getting slow writing performance because there was a new connection open for every db operation.
I tried using connection pooling in the following manner:
string dbConnectionString = $"Data Source={dbConfigPath};Version=3;Pooling=True;Max Pool Size=1000;";
However I'm not seeing any improvement.
Am I using it wrong? Is there a way to make sure the pooling is working?
After some more debugging I realized the main bottleneck came from the WCF service that I was using to run the insert queries.
Related
I am experiencing database connection errors with an ASP.NET application written in VB, running on three IIS servers. The underlying database is MS Access, which is on a shared network device. It uses Entity Framework, code first implementation and JetEntityFrameworkProvider.
The application is running stable. But, approximately 1 out of 1000 attempts to open the database connection fails with either one of the following two errors:
06:33:50 DbContext "Failed to open connection at 2/12/2020 6:33:50 AM +00:00 with error:
Cannot open database ''. It may not be a database that your application recognizes, or the file may be corrupt.
Or
14:04:39 DbContext "Failed to open connection at 2/13/2020 2:04:39 PM +00:00 with error:
Could not use ''; file already in use.
One second later, with refreshing (F5), the error is gone and it works again.
Details about the environment and used code.
Connection String
<add name="DbContext" connectionString="Provider=Microsoft.Jet.OLEDB.4.0;Data Source=x:\thedatabase.mdb;Jet OLEDB:Database Password=xx;OLE DB Services=-4;" providerName="JetEntityFrameworkProvider" />
DbContext management
The application uses public property to access DbContext. DbContext is kept in the HttpContext.Current.Items collection for the lifetime of the request, and is disposed at it’s end.
Public Shared ReadOnly Property Instance() As DbContext
Get
SyncLock obj
If Not HttpContext.Current.Items.Contains("DbContext") Then
HttpContext.Current.Items.Item("DbContext") = New DbContext()
End If
Return HttpContext.Current.Items.Item("DbContext")
End SyncLock
End Get
End Property
BasePage inits and disposes the DbContext.
Protected Overrides Sub OnInit(e As EventArgs)
MyBase.OnInit(e)
DbContext = Data.DbContext.Instance
...
End Sub
Protected Overrides Sub OnUnload(e As EventArgs)
MyBase.OnUnload(e)
If DbContext IsNot Nothing Then DbContext.Dispose()
End Sub
What I have tried
Many of the questions on SO which address above error messages, deal with generally not being able to establish a connection to the database – they can’t connect at all. That’s different with this case. Connection works 99,99% of the time.
Besides that, I have checked:
Permissions: Full access is granted for share where .mdb (database) and .ldb (locking file) resides.
Network connection: there are no connection issues to the shared device; it’s a Gigabit LAN connection
Maximum number of 255 concurrent connections is not reached
Maximum size of database not exceeded (db has only 5 MB)
Changed the compile option from “Any CPU” to “x86” as suggested in this MS Dev-Net post
Quote: I was getting the same "Cannot open database ''" error, but completely randomly (it seemed). The MDB file was less than 1Mb, so no issue with a 2Gb limit as mentioned a lot with this error.
It worked 100% on 32 bit versions of windows, but I discovered that the issues were on 64 bit installations.
The app was being compiled as "Any CPU".
I changed the compile option from "Any CPU" to "x86" and the problem has disappeared.
Nothing helped so far.
To gather more information, I attached an Nlog logger to the DbContext which writes all database actions and queries to a log file.
Shared Log As Logger = LogManager.GetLogger("DbContext")
Me.Database.Log = Sub(s) Log.Debug(s)
Investigating the logs I figured out that when one of the above errors occured on one server, another one of the servers (3 in total) has closed the db connection at exactly the same time.
Here two examples which correspond to the above errors:
06:33:50 DbContext "Closed connection at 2/12/2020 6:33:50 AM +00:00
14:04:39 DbContext "Closed connection at 2/13/2020 2:04:39 PM +00:00
Assumption
When all connections of a DbContext have been closed, the according record is removed from the .ldb lock file. When a connection to the db is being opened, a record will be added to the lock file. When these two events occur at the exact same time, from two different servers, there is a write conflict to the .ldb lock file, which results in on of the errors from above.
Question
Can anyone confirm or prove this wrong? Has anyone experienced this behaviour? Maybe I am missing something else. I’d appreciate your input and experience on this.
If my assumption is true, a solution could be to use a helper class for accessing db, which catches and handles this error, waiting for a minimal time period and trying again.
But this feels kind of wrong. So I am also open to suggestions for a “proper” solution.
EDIT: The "proper" solution would be using a DBMS Server (as stated in the comments below). I'm aware of this. For now, I have to deal with this design mistake without being responsible for it. Also, I can't change it in the short run.
I write this as an aswer because of space but this is not really an answer.
It's for sure an OleDb provider issue.
I think that is a sharing issue.
You could do some tries:
use a newer OleDb provider instead of Microsoft.Jet.OLEDB.4.0. (if you have try 64 bits you could already have try another provider because Jet.OLEDB.4.0 is 32 bits only)
Implement a retry mechanism on the new DbContext()
Reading your tests this is probaly not your case. I THINK that Dispose does not always work properly on Jet.OLEDB.4.0 connections. I noted it on tests and I solved it using a different testing engine. Before giving up I used this piece of code
GC.Collect(GC.MaxGeneration, GCCollectionMode.Forced, true);
GC.WaitForPendingFinalizers();
GC.Collect(GC.MaxGeneration, GCCollectionMode.Forced, true);
As you can understand reading this code, they are tries and the latest solution was changing the testing engine.
If your app is not too busy you could try to lock the db using a different mechanism (for example using a lock file). This is not really different from new DbContext() retries.
In late '90s I remember I had an issue related to disk sharing OS (I were using Novel Netware). Actually I have not experience in using mdb files on a network share. You could try to move the mdb on a folder shared with Windows
Actually I use Access databases only for tests. If you really need to use a single file database you could try other solutions: SQL Lite (you need a library, also this written by me, to apply code first https://www.nuget.org/packages/System.Data.SQLite.EF6.Migrations/ ) or SQL Server CE
Use a DBMS Server. This is for sure the best solution. As the writer of JetEntityFrameworkProvider I think that single file databases are great for single user apps (for this apps I suggest SQL Lite), for tests (I think that for tests JetEntityFrameworkProvider is great), for transfering data or, also, for readonly applications. In other cases use a DBMS Server. As you know, with EF, you can change from JetEntityFrameworkProvider to SQL Server or to MySql without effort.
You went wrong at the design stage: The MS Access database engine is unfit for ASP.Net sites, and this is explicitly stated on multiple places, e.g. the official download page under details.
The Access Database Engine 2016 Redistributable is not intended .... To be used by ... a program called from server-side web application such as ASP.NET
If you really have to work with an Access database, you can run a helper class that retries in case of common errors. But I don't recommend it.
The proper solution here is using a different RDBMS which exhibits stateless behavior. I recommend SQL Server Express, which has limitations, but if you exceed those you will be far beyond what Access supports, and wont cause errors like this.
I am unsing entity framework in my web application. Earlier everything was working fine. But then I deleted database from Sql Server. After that whenever I run the application, I get the following exception :
The underlying provider failed on open entity framework
Isn't it true that if we delete the database from Sql Server and again run the application, database is again created?
Can you show some details about connection string. I basically want to check the authentication you are using.
Demo for setting a typical authentication : http://msdn.microsoft.com/en-us/library/ff649314.aspx
Another possibility can be with connection pooling. Try explicitly closing the connection(if there are any open connections)
db.Database.Connection.Close();
Entity framework will recreate your database on application start assuming the following:
You have permission to create databases in SQL
A connection string is specified correctly (if it was working before this should be fine)
You are using CodeFirst or ModelFirst strategies for databases (ie if you generated an EDMX off an existing database it wont recreate it for you)
I had a similar error and it was due to the connection problem.
Usually you need to open your *.EDPS file under your entity (EDML) and check your connection string and make sure it has got a correct setting especially your Default Oracle home.
Apparently, using AttachDbFilename and user instance in your connection string is a bad way to connect to a DB. I'm using SQL server express on my local machine and it all seems to work fine. But what's the proper way to connect to SQL server then?
Thanks for your explanation.
Using User Instance means that SQL Server is creating a special copy of that database file for use by your program. If you have two different programs using that same connection string, they get two entirely different copies of the database. This leads to a lot of confusion, as people will test updating data with their program, then connect to a different copy of their database in Management Studio, and complain that their update isn't working. This sends them through a flawed series of wild goose chase steps trying to troubleshoot the wrong problem.
This article goes into more depth about how to use this feature, but heed the very first note: the User Instance feature has been deprecated. In SQL Server 2012, the preferred alternatives are (in this order, IMHO):
Create or attach your database to a real instance of SQL Server. Your connection string will then just need to specify the instance name, the database name, and credentials. There will be no mixup as Management Studio, Visual Studio and your program(s) will all be connecting to a single copy of the database.
Use a container for local development. Here's a great starter video by Anna Hoffman and Anthony Nocentino, and I have some other resources here, here, and here. If you're on an M1 Mac, you won't be able to use a full-blown SQL Server instance, but you can use Azure SQL Edge if you can get by with most SQL Server functionality (the omissions are enumerated here).
Use SqlLocalDb for local development. I believe I pointed you to this article yesterday: "Getting Started with SQL Server 2012 Express LocalDB."
Use SQL Server Compact. I like this option the least because the functionality and syntax is not the same - so it's not necessarily going to provide you with all the functionality you're ultimately going to want to deploy. Compact Edition is also deprecated, so there's that.
Of course if you are using a version < SQL Server 2012, SqlLocalDb is not an option - so you should be creating a real database and using that consistently. I only mention the Compact option for completeness - I think that can be almost as bad an idea as using AttachDbFileName.
EDIT: I've blogged about this here:
Bad Habits : Using AttachDBFileName
In case someone had the problem.
When attaching the database with a connection string containing AttachDBFile
with SQLEXPRESS, I noticed this connection was exclusive to the ASP.NET application that was using the database. The connection did block the access to all other processes on the file level when made with System.Data.SqlClient as provider.
In order to assure the connection to be shareable with other processes
instead use DataBase to specify the database name in your connection string
Example or connection string :
Data Source=.\SQLEXPRESS;DataBase=PlaCliGen;User ID=XXX;password=ZZZ; Connect Timeout=30
,where PlaCliGen is the name (or logical name) by which SQLEXPRESS server knows the database.
By connecting to the data base with AttachDBFile giving the path to the .mdf file
(namely : replacing DataBase = PlacliGen by AttachDBFile = c:\vs\placligen\app_data\placligen.mdf) the File was connected exclusively and no other process could connect to the database.
I have a Sql Server 2008 Express database file that's currently being used by an ASP.NET application, and I'm not sure how to query the database without taking the website down.
I'm unable to copy the database files (.mdf and .ldf files) to another directory, since they're in use by the web server. Also, if I attach the databases to an instance of the sql server (using the 'Create Database [DB name] on (filename = '[DB filename.mdf]') for attach;' command at the sqlcmd prompt), then the application pool user becomes unable to access the database (i.e. the webpages start producing http 500 errors. I think this might have to do with the username for the application pool becoming somehow divorced from the login credentials in the sql server database).
Any suggestions? I realize this is probably a newbie question, since it seems like a rather fundamental task. However, due to my inexperience, I really don't know what the answer is, and I'm pretty stumped at this point, since I've tried a couple of different things.
Thanks!
Andrew
if I attach the databases to an instance of the sql server (using the 'Create Database [DB name] on (filename = '[DB filename.mdf]') for attach;' command at the sqlcmd prompt),
Don't do this to a live database - it's attempting to be setup an MDF to be written to by two different databases...
Use Backup/Restore
As you've found, Attach/ReAttach requires the database to be offline - use the Backup/Restore functionality:
MSDN: Using SSMS to Backup the Database
MSDN: Using SSMs to Restore the Backup
Be aware that the backup/restore doesn't maintain logins (& jobs if you have any associated with the database) - you'll have to recreate & sync if using an account other than those with uber access.
Maybe Linked Server would work?
Another alternative would be to setup another SQL Server Express/etc instance on a different box, and use the Linked Server functionality to create a connection to the live/prod data. Use a different account than the one used for the ASP application...
Am trying to run a C# application which fetches data from PostgreSQL database(8.4) and am using the PostgreSQL OLE DB Provider
This is my connection string
string connString =
#"Provider=PostgreSQL OLE DB
Provider;Data
Source=localhost;location=;User
ID=;password=;timeout=1000";
The error am getting is
The 'PostgreSQL OLE DB Provider'
provider is not registered on the
local machine
Can anyone point out me where should I correct so as to get the connect to the database. By the way am using PGOLEDBLib.
I have seen a few different "pgoledb" libraries. None appear to actually be named "pgoledblib", so this info may or may not apply to you.
The most recently active PGOLEDB has documented their connection string attributes here; it appears they use a different Provider value -- "PGNP.1".
Another provider, which seems to have been languishing since 2006, has connection strings here; these are in line with the earlier suggestions from OrbMan.
Of course, you may find that paying nothing for such a provider leads to paying much in installation, configuration, and utilization headaches. Towards easing such headaches, I offer a link to my employer's options supporting ODBC, JDBC, and ADO.NET clients on many platforms.
Any particular reason you're using the OLE DB provider? I believe the general idea is that you get better performance, and I think also functionality, if you use a native .net provider - e.g. http://npgsql.projects.postgresql.org/
Try using "PostgreSQL" or "PostgreSQL.1" for the Provider attribute.. If that does not work, you will probably need to re-install the driver.
I found the solution.
Step 1: Down load & install
https://www.pgoledb.com/index.php/component/filecabinet/?task=download&cid[0]=15
Step 2: Restart PC.
Step 3: Set the connection string as below
Provider=PGNP.1;Data Source=localhost;Persist Security Info=True;Initial Catalog=myDatabase;User ID=yyy;password=xxx
It should work as your expected