I am using the pool package to connect a Shiny application to a fairly large SQLite database (3Gb, 70M rows).
I create a pool using:
pool <- dbPool(
drv = RSQLite::SQLite(),
dbname = "mydb.db")
Everything works perfectly locally, but when I put it on my server (I am using a DigitalOcean droplet running Shiny server), the pool expires very quickly after ~15 seconds, whether I am active or not on the app.
In the logs I see
Error in pool$fetch: This pool is no longer valid. Cannot fetch new objects.
I have tried changing the idleTimeout and minSize parameters when creating the pool, to no avail.
How can I prevent this? Is there a way I can check whether the pool is still valid and if not reconnect to the DB?
Also, it would be good if someone could give some insights on why this could be happening.
Just in case someone else bumps into this problem, the way I have solved the issue is to check the valid attribute of the pool connection. So I can prepend queries with something like:
if (!conn$valid) # pool has expired
connect_to_db() # This re-connects to the DB
<do query>
This seems to have solved the issue
Related
I am in a corporate environment.
I can easily establish a connection between R and a SQL database thanks to DBI and odbc packages this way:
con <- DBI::dbConnect(
odbc::odbc(),
driver = SQL.driver,
server = SQL.server,
database = SQL.database,
encoding = SQL.encoding
)
This implicitely uses the "Windows Authentication" method. However, the server that will have to run the above R code will not be using a Windows Account that has actually access to the MS SQL DB.
Therefore, I would like to know how can I explicitely specify another Windows Account prior to or within the code above?
Any help would be greatly appreciated.
In other words, I would like to reproduce following behavior in R the following manual process (on a Windows session that uses credentials not allowed to connect to the SQL DB):
SHIFT+right click on MSSSM,
"Run as a different user",
Enter Windows Account #2 (the one that actually has access to the DB),
Connect to SQL DB using Windows Authentication (the #2 one, not the Windows session one which does not)
Hope the context is clear.
For obscure corporate reasons, creating a local SQL account is not possible. Otherwise, thanks to UID/PWD of DBI/odbc it would be theoritically straighforward.
I am facing a problem with my live site. The Issue is a SQLTimeout error.
I have followed the below scenario to solve the issue. But I can't do it .
Steps taken:
Increased the SqlCommand Timeout = 0 and 240
Increased the SqlCommand Connection Timeout = 0
I have applied raw SQL in code to fetch the data from SQL Server
Kindly share with me if you have any suggestions about this issue.
Thanks
It is really hard to address your issue with so little info you provided.
Generally I would recommend to execute your query in SQL Server Management Studio and see what happens.
It could be either really long query or locking issue in database.
Also be aware, if you host you site on IIS, that apart from SQL Server timeout, the IIS request timeout would apply.
Example :
Table 1,
Table 2 ,
Table 3
Issue Scenario:
You have request to get those three table values .But the second request used to store some value in any one table at the same time existing request is not completed .
you will get the timeout error here , because existing request is not completed ,
Solutions :
Please use isolation concept to avoid this error while writing and read and updated time
Thanks.
In my ASP.NET application, I am getting the following error:
Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
But I can successfully connect to the database server using 'SQL Server Mangement Studio' and I can also correctly PING the host where SQL Server is hosted.
What can be wrong here.
Check your connection string in web.config. The connection string you are using via SQL Management Studio is different to the one in the web.config.
You can increase timeout in web.config too which is better or say best approach.
Also whenever you get this type of error and everything is good in confirable, then first debug the code,
if the issue code side, then you manage simply by changing logic.
If it is sqlserver side, then get the parameter value and sp or query. Run in to SSMS which gives you better idea.
Increasing Command timeout to 120 fixed my problem for me.
adapter.SelectCommandCommandTimeout=120;
We recently launched a new web site... there are roughly ~150 users active during peak hours. During peak hours, we are experiencing an issue every few minutes, the exception text is listed below.
System.Web.HttpUnhandledException:
Exception of type 'System.Web.HttpUnhandledException' was thrown.
---> System.Data.SqlClient.SqlException: The client was unable to establish a connection because of an error during connection initialization process before login.
Possible causes include the following:
the client tried to connect to an unsupported version of SQL Server;
the server was too busy to accept new connections;
or there was a resource limitation (insufficient memory or maximum allowed connections) on the server. (provider: Named Pipes Provider, error: 0 - No process is on the other end of the pipe.)
Our data access layer calls various DataTableAdapters using the following syntax.
EDIT
Yes, da is the name assigned to the DataTableAdapter. There is no connection.Open() because the DataTableAdapter takes care of all that, right?
using(TheDataLayer.some.strongly.typedNameTableAdapters.suchAndSuchTableAdapter da = new TheDataLayer.some.strongly.typedNameTableAdapters.suchAndSuchTableAdapter())
{
StronglyTyped.DataTable dt = new StronglyTyped.DataTable();
da.FillByVariousArguments(dt, ..., ...);
//da.Dispose();
return something;
}
The connection string looks something like:
<add name="MyConnectionString"
connectionString="Data Source=myDBServerName;Initial Catalog=MyDB;User ID=MyUserName;Password=MyPassword"
providerName="System.Data.SqlClient" />
I'm trying to rule the problem being in Code. Is there anything "simple" that can be done to minimize this issue?
Thanks.
Have you tried "Connection Pooling" directly in connection string settings?
Example:
connectionString="....;Pooling=true;Min Pool Size=1;Max Pool Size=10;..."
You can read more info here: http://msdn.microsoft.com/en-us/library/8xx3tyca%28v=vs.71%29.aspx
Without seeing the code that actually opens and uses the connection, it's hard to say where the problem is.
Please update your question with what happens when you create that DataAdapter (I'm guessing that's what da means).
Also, if you're using the using statement, you shouldn't be disposing of the thing you created the using statement for.
We had similar issue which only happenes in our production environment and it was particularly associated with load. During busy time of day we would recieve several of the above mentioned exception.
We gone through a massive investigation around why this exception occurs and did a lot of changes to fix the issue. The defacto change we did which aleviated the problem was connection pool setting by setting min pool size to 1 and max pool size to 10. (It can vary based on your situation)
This issue will be more prevalent when you have several i.e. 1000's of Customer DB and use default connection string (i.e. database=DBName;server=ServerName). We were not explicitly setting min/max pool size hence it took default settings which set Min pool size to 0 and max pool size to 100.
Again, I dont have concrete proof but the theory is that during busy time of the day based on load it made several connection to DB server and DB server was bombarded with a lot of connection request at single point to several databases. Either Application server or DB server did have bandwidth to handle that many connection in a short period of time. Also, it was happening with server with most databases. Though we did not see a lot of connection at a time but Application server was not able to make connection to databases for a short duration when it had surge of requests going in.
After we set min pool size we aliveated this problem as there is atleast one connection to each database which is available all the time and if there is blast of request which required to make connection to several databases we already had atleast one connection to the database available before we request a new one.
Maybe unrelated to the actual problem you were facing, but this error is also thrown if you are trying to connect without specifying the correct port along with the database server name.
I'm getting the following error when using the Database Publishing wizard to script a SQL Server Express database for deployment. I have googled for hours unsuccessfully. Anyone had this isssue or know how to solve it?
Timeout expired. The timeout period
elapsed prior to obtaining a
connection from the pool. This may
have occurred because all pooled
connections were in use and max pool
size was reached.
Is the database held remotely? - Have you checked firewall settings...?
The error is what it says it is...it's timed out trying to connect.
Make sure you can connect via other means (SQL Management Studio, the app itself). Check the connection string, even try copying the database locally and scripting it that way.
So I downloaded the latest Database Publishing Wizard and it seems to work :-)