We have a Utility Class that opens and disconnects the Derby DB Connection
The database is stored in a folder on C drive of a Windows 7 computer
The application is being written with JavaFX 8
We are NOT using transactions
Before using the Utility Class we would Open and Disconnect the connection with each CRUD function rs.close() con.close()
Our question has two parts
1. Do we really need to open and close the connection with each CRUD function?
2. Why is the Utility Class not closing the rs and stmnt?
The conn.close() fires when
We will post the code for the Utility Class and the Delete function
We are also using the code below in the Main Class to shutdown the Derby DB when the application is closed
private void handle(WindowEvent e) throws SQLException {
//Proper CLOSE of connection to DB keeps proper incrementing by 1 as set when the table is created
JDBCUtil.closeConnection(conn);
String conURL = "jdbc:derby:;shutdown=true";
try{
DriverManager.getConnection(conURL);
}catch (SQLException se){
if(!(se.getErrorCode() == 50000) && (se.getSQLState().equals("XJ015")))
System.err.println(se);
}
System.exit(0);
Platform.exit();
}
Utility Class
public class JDBCUtil {
public static Connection conn;
public static Connection getConnection() throws SQLException {
// The URL is specific to the JDBC driver and the database you want to connect
String dbName="InfoDB";
String dbURL = "jdbc:derby:C:/A_DerbyDataBase/DBName/" + dbName + ";create=true";
//String dbURL = "jdbc:derby:DATABASE_NAME;create=true";
// Set the user id and password
//String userId = "app";
//String password = "app";
// Get a connection
conn = DriverManager.getConnection(dbURL);
// Set the auto-commit to false ONLY if you use Transactions
/*conn.setAutoCommit(true);*/
System.out.println("111111111111111111111111111 Get Connection ");
return conn;
}
public static void closeConnection(Connection conn) throws SQLException {
if (conn != null) {
System.out.println("222222222222222222222 conn.close ");
conn.close();
}
}
public static void closeStatement(Statement stmnt) throws SQLException{
if (stmnt != null) {
System.out.println("3333333333333333333333 stmnt.close ");
stmnt.close();
}
}
public static void closeResultSet(ResultSet rs) throws SQLException {
if (rs != null) {
System.out.println("44444444444444444444444 rs.close ");
rs.close();
}
}
/*public static void commit(Connection conn) throws SQLException {
if (conn != null) {
conn.commit();
}
}
public static void rollback(Connection conn) throws SQLException {
if (conn != null) {
conn.rollback();
}
}*/
public static void main(String[] args) throws SQLException {
//conn = JDBCUtil.getConnection();
JDBCUtil.closeConnection(conn);
}
And the Delete Function
#FXML
private void onDelete(ActionEvent e) throws SQLException, IOException{
conn = JDBCUtil.getConnection();
String sql = "DELETE FROM infodata WHERE ID = ?";
pstmt = conn.prepareStatement(sql);
int ID = Integer.valueOf(txfID.getText());
pstmt.setInt(1, ID);
pstmt.executeUpdate();
pstmt.close();
JDBCUtil.closeConnection(conn);
ReadFromDB();
btnEdit.setVisible(false);
btnDelete.setVisible(false);
btnCancel.setVisible(false);
btnAdd.setVisible(true);
txfInfo.setText("Record Deleted");
}
Database Access with a Connection Pool
Rather than writing your own connection logic, I advise using a connection pool, for example see the Baeldung Hikari Tutorial. You can use it like this:
Initialize the connection pool in your application's init method.
When you want to use a connection, use the Java try-with-resources construct, which will auto-close the connection (returning it to the connection pool) when you are done with it.
In your application's stop method, close the connection pool.
Sample wrapper class for DataSource connections (copied from Baeldung Hikari link above):
public class DataSource {
private static HikariConfig config = new HikariConfig();
private static HikariDataSource ds;
static {
config.setJdbcUrl( "jdbc_url" );
config.setUsername( "database_username" );
config.setPassword( "database_password" );
config.addDataSourceProperty( "cachePrepStmts" , "true" );
config.addDataSourceProperty( "prepStmtCacheSize" , "250" );
config.addDataSourceProperty( "prepStmtCacheSqlLimit" , "2048" );
ds = new HikariDataSource( config );
}
private DataSource() {}
public static Connection getConnection() throws SQLException {
return ds.getConnection();
}
}
Sample database call using the Hikari connection pool with autoclose of connections using Java's try with resources construct (copied from Baeldung Hikari link above):
public static List<Employee> fetchData() throws SQLException {
String SQL_QUERY = "select * from emp";
List<Employee> employees = null;
try (Connection con = DataSource.getConnection();
PreparedStatement pst = con.prepareStatement( SQL_QUERY );
ResultSet rs = pst.executeQuery();) {
employees = new ArrayList<>();
Employee employee;
while ( rs.next() ) {
employee = new Employee();
employee.setEmpNo( rs.getInt( "empno" ) );
employee.setEname( rs.getString( "ename" ) );
employee.setJob( rs.getString( "job" ) );
employee.setMgr( rs.getInt( "mgr" ) );
employee.setHiredate( rs.getDate( "hiredate" ) );
employee.setSal( rs.getInt( "sal" ) );
employee.setComm( rs.getInt( "comm" ) );
employee.setDeptno( rs.getInt( "deptno" ) );
employees.add( employee );
}
}
return employees;
}
For more info read the Baeldung tutorial and the Hikari site documentation:
Link to HikariCP project.
Database Access without a Connection Pool
Now, you don't need to use a connection pool to do this, you can open and close a connection for each database call, however I would recommend using a connection pool for performance reasons and so that you don't end up trying to re-invent the wheel and ending up with something square rather than round.
I didn't try debugging or examining the connection utility class you have in your question, but, if you are going to replace it with a connection pool anyway, there is no reason to do that.
Sample code for accessing a database from JavaFX without a connection pool:
JavaFX MySQL connection example please
Non-Task based database access.
JavaFX - Background Thread for SQL Query which uses a Task based sample for db access.
Some of the sample code is written to be minimal to demonstrate particular purposes, such as access to a database and feedback of data from a given table to a UI, and not as a general purpose database utility. For a robust implementation for an application that uses many database queries, a connection pool or a dedicated database connection manager class (such as you have in your question), is preferred.
Related
In the code below, I was expecting the in-memory database to be deleted when the connection closes as described at https://www.sqlite.org/inmemorydb.html
public static void main(String[] args) throws SQLException {
var connection = DriverManager.getConnection("jdbc:sqlite::memory");
var statement = connection.createStatement();
statement.execute("CREATE TABLE IF NOT EXISTS tab (name);");
statement.execute("INSERT INTO tab (name) VALUES ('foo');");
var result = statement.executeQuery("SELECT * from tab;");
while (result.next()) {
System.out.println(result.getString("name"));
}
connection.close();
}
In fact what I see is that each time I run this code there is an extra row in the table implying that the database is persisting between calls. Why is that happening?
Turned out it was a simple error. The connection string should have been:
var connection = DriverManager.getConnection("jdbc:sqlite::memory:");
The missing final : meant that it was interpreting it as as a database called "memory".
Setup
•Visual Studio 2010
•IIS 8.5
•.NET Framework 4.6
•Microsoft SQL Server 2014
•AppPool Account on IIS is domain\web
I have a web page that monitors changes in a database table. I am using dependency_OnChange to monitor the database and pass the data to the user via signalR. I set a breakpoint in the dependency_OnChange method and it is only getting hit a few times out of thousands of database updates.
In web.config... I am using Integrated Security=True.
My user is a sysadmin on the sql box. (This is just for proof of concept)
In Global.asax... specifying a queuename and stopping and starting sqldependency
void Application_Start(object sender, EventArgs e)
{
var queuename = "Q_Name";
var sConn = ConfigurationManager.ConnectionStrings["singalR_ConnString"].ConnectionString;
SqlDependency.Stop(sConn, queuename);
SqlDependency.Start(sConn, queuename);
}
void Application_End(object sender, EventArgs e)
{
var queuename = "Q_Name";
var sConn = ConfigurationManager.ConnectionStrings["singalR_ConnString"].ConnectionString;
SqlDependency.Stop(sConn, queuename);
}
In code behind...
public void SendNotifications()
{
//Identify Current User and Row No
string CurrentUser = GetNTName();
string message = string.Empty;
string conStr = ConfigurationManager.ConnectionStrings["singalR_ConnString"].ConnectionString;
using (SqlConnection connection = new SqlConnection(conStr))
{
string query = "SELECT [RowNo] FROM [dbo].[Test] WHERE [User] = #User";
string SERVICE_NAME = "Serv_Name";
using (SqlCommand command = new SqlCommand(query, connection))
{
// Add parameters and set values.
command.Parameters.Add("#User", SqlDbType.VarChar).Value = CurrentUser;
//Need to clear notification object
command.Notification = null;
//Create new instance of sql dependency eventlistener (re-register for change events)
SqlDependency dependency = new SqlDependency(command, "Service=" + SERVICE_NAME + ";", 0);
//SqlDependency dependency = new SqlDependency(command);
//Attach the change event handler which is responsible for calling the same SendNotifications() method once a change occurs.
dependency.OnChange += new OnChangeEventHandler(dependency_OnChange);
connection.Open();
SqlDataReader reader = command.ExecuteReader();
if (reader.HasRows)
{
reader.Read();
message = reader[0].ToString();
}
}
}
//If query returns rows, read the first result and pass that to hub method - NotifyAllClients.
NotificationsHub nHub = new NotificationsHub();
nHub.NotifyAllClients(message);
}
private void dependency_OnChange(object sender, SqlNotificationEventArgs e)
{
//Check type to make sure a data change is occurring
if (e.Type == SqlNotificationType.Change)
{
// Re-register for query notification SqlDependency Change events.
SendNotifications();
}
}
NotificationsHub.cs page...
//Create the Hub
//To create a Hub, create a class that derives from Microsoft.Aspnet.Signalr.Hub.
//Alias that can call class from javascript. - i.e. var hub = con.createHubProxy('DisplayMessage');
[HubName("DisplayMessage")]
public class NotificationsHub : Hub //Adding [:Hub] let c# know that this is a Hub
{
//In this example, a connected client can call the NotifyAllClients method, and when it does, the data received is broadcasted to all connected clients.
//Create NotifyAllClients Method
//public means accessible to other classes
//void means its not returning any data
public void NotifyAllClients(string msg)
{
IHubContext context = GlobalHost.ConnectionManager.GetHubContext<NotificationsHub>();
//When this method gets called, every single client has a function displayNotification() that is going to be executed
//msg is the data that is going to be displayed to all clients.
context.Clients.All.displayNotification(msg);
}
}
The first thing I would do here is refactor the Sql Dependency setup out to a stand alone method and call it from your send notification. (SoC and DRY) because if you are creating other SqlDependencies in other places they are going to trip each other up. Secondly your are creating a new NotificationsHub, You should be getting the currently active hub.
DefaultHubManager hubManager = new DefaultHubManager();
hub = hubManager.ResolveHub("NotificationsHub");
hub.NotifyAllClients(message);
There is also an older way to get the hub but I am not sure it will still work
GlobalHost.ConnectionManager.GetHubContext<NotificationsHub>()
I also have an example of a simpler version in this answer.
Polling for database changes: SqlDependency, SignalR is Good
Let me know if you have any questions.
I have a .net mvc / sql server website for a high-traffic application. I'm using async database connections.
Prior to launch I ran a load test, and it blew up pretty quickly under load. The connection pool quickly ran out of connections.
Running sp_who at the database level shows a pile of connections sleeping awaiting command. If I switch to non-async, this does not happen.
So, it appears that new database calls are not using connections sitting in the connection pool, instead they are opening their own new connection. This quickly exhausts the connection pool limit, and begins timing out queries.
The following is the helper I'm using to execute an async datareader. Does anyone see any issue here?
private async Task<List<T>> ExecuteReaderAsync<T>(SqlCommand command, Func<SqlDataReader, T> rowConverter)
{
List<T> ret = new List<T>();
using (SqlConnection connection = new SqlConnection(this.ConnectionString))
{
command.Connection = connection;
await connection.OpenAsync();
using (SqlDataReader reader = await command.ExecuteReaderAsync())
{
while (await reader.ReadAsync())
{
ret.Add(rowConverter(reader));
}
reader.Close();
}
connection.Close();
}
return ret;
}
I'm calling the code like the datareader helper like the following:
public async Task<Content> FindContentAsync(int id)
{
Content content = null;
using (SqlCommand command = CreateProcedure("dbo.FindContent"))
{
AddParam(command, "Id", SqlDbType.Int, id);
List<Content> items = await ExecuteReaderAsync<Content>(command, x => BindContent(x));
if (items.Count > 0)
{
content = items[0];
}
}
return content;
}
And calling that from a helper:
public async Task<Content> FindAsync(int id)
{
var db = new DataAccess();
var content = await db.FindContentAsync(id);
return content;
}
I have a class that gets tables from Sql Server. the class is static, but the variables are not. I want to know if it is OK in Asp net, because I had read not to use static at database in Asp net.
My Class: (There are more functions in the class, I put here one for example)
public static class DataBase
{
public static bool TableChange(string sqlCreate)
{
using (SqlConnection connection = new SqlConnection(Global.ConnectionString))
{
using (var cmd = new SqlCommand(sqlCreate, connection))
{
try
{
connection.Open();
cmd.ExecuteNonQuery();
}
catch (Exception ex)
{
Log.WriteLog(ex.Message + "\n" + sqlCreate, ex, HttpContext.Current.Request);
return false;
}
}
}
return true;
}
}
Thanks in advance
What you have read is most probably something to do with this approach:
public static EntityContext Database = new EntityContext();
// or
public static SqlConnection Database = new SqlConnection("...");
Here you store the database connection in a static variable and thus all parallel requests would want to use the same connection which is a very bad approach if it even works at all (it will probably work sort of fine until the page is under load).
You do not have this problem, because in your case only the methods are static, not the variables. Your code follows the recommended path - open connection (retrieve it from the pool), execute query, close the connection (return it to the pool).
Since I'm using Postgresql and can't use LINQ to SQL, I wrote my own wrapper classes.
This is a part of the Student class:
public class Student : User
{
private static NpgsqlConnection connection = null;
private const string TABLE_NAME = "students";
public int Id { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string Password { get; set; }
/// <summary>
/// Reads data from the data reader and moves it to the Student class.
/// </summary>
private static void ReadFields(Student student, NpgsqlDataReader dr)
{
student.Id = Int32.Parse(dr["id"].ToString());
student.FirstName = dr["first_name"].ToString();
student.LastName = dr["last_name"].ToString();
student.Password = dr["password"].ToString();
}
/// <summary>
/// Updates the student
/// </summary>
public void Update()
{
Connect();
Run(String.Format("UPDATE " + TABLE_NAME + " SET first_name='{0}', last_name='{1}', password='{2}' WHERE id={3}", FirstName, LastName, Password, Id));
connection.Dispose();
}
/// <summary>
/// Inserts a new student
/// </summary>
public void Insert()
{
Connect();
Run(String.Format("INSERT INTO " + TABLE_NAME + " (first_name, last_name, password) VALUES ('{0}', '{1}', '{2}')",FirstName, LastName, Password));
connection.Dispose();
}
private static void Run(string queryString)
{
NpgsqlCommand cmd = new NpgsqlCommand(queryString, connection);
cmd.ExecuteScalar();
cmd.Dispose();
}
private static void Connect()
{
connection = new NpgsqlConnection(String.Format("Server=localhost;Database=db;Uid=uid;Password=pass;pooling=false"));
connection.Open();
}
//....
So as you see with every INSERT, DELETE, UPDATE request I'm using Connect() method which connects to the database. I didn't realize how stupid it was before I had to wait for 10 minutes to have 500 rows inserted, as there were 500 connections to the database.
So I decided to move Connection property to a static DB class.
public static class DB
{
private static NpgsqlConnection connection = null;
public static NpgsqlConnection Connection
{
get
{
if (connection == null)
{
connection = new NpgsqlConnection(String.Format("Server=localhost;Database=db;Uid=uid;Password=pass;pooling=false"));
connection.Open();
}
return connection;
}
}
public static void Run(string queryString)
{
NpgsqlCommand cmd = new NpgsqlCommand(queryString, connection);
cmd.ExecuteScalar();
cmd.Dispose();
}
}
It works now! I replaces all Run methods in the Student class with DB.Run
But I want to know if it will work fine with a lot of people online, not me only. I'm not sure how static things work with ASP.NET, maybe it'll eat a lot of memory?..
It is better not to store the connection in a static field. Create the connection object on demand and let the connection pooling manage your connections.
You can enable connection pooling for PostgreSQL and let the pooler manage connections for you. Then you can use either piece of code without worry. Even when you issue multiple open/close commands the pooler will optimize them.
This provides you more flexibility and less worry about a custom management solution, also less code and edge cases. It will depend on the database provider you're using. Something in the connection string like:
Pooling: True or False. Controls
whether connection pooling is used.
Default = True
If you need a database provider that uses connection pooling for Postgres one option is npgsql: Npgsql is a .Net data provider for Postgresql.
It supports connection pooling as described in the docs.
Static classes are singletons. The danger here is what they reference. Because they always stay alive, everything they keep a reference to will not be garbage collected.
To see if this is the case, profile your web servers memory. If it always grows and never shrinks, you may be constantly adding references in a static class which never get collected.
In all honesty though, I'd create it only as needed, and completely avoid all of this.
EDIT:
I mean don't worry about sharing one single connection object across your data access layer. If the provider you're using supports connection pooling, then it will handle the actual connections made to the database. Just use and dispose of your connection objects as needed at any point in your data access layer.
using (var connection = new NpgsqlConnection("your connection string"))
{
//your data access stuff.
}
I know code like this is rather big, bulky, and repetitive, but that's ADO.NET. As long as you isolate these calls in their own classes by creating a data access library/layer, it's very manageable. Hiding the ADO.NET objects in a static class is dangerous, because you'll inevitably forget to close a connection or call Dispose() somewhere. Plus you run the risk of building a large object graph that will never get garbage collected.
public class Dataconnect
{
public static string connstring = ConfigurationSettings.AppSettings["SQLConnection"].ToString();
SqlConnection objcon = new SqlConnection(connstring);
SqlCommand objcmd = new SqlCommand();
public bool Opencon()
{
try {
if (objcon.State == ConnectionState.Closed)
{
objcon.Open();
}
objcmd.Connection = objcon;
return true;
}
catch (Exception ex) { throw new Exception("Error: In Open connesction"); return false; }
}
public bool Closecon()
{
try
{
if (objcon.State == ConnectionState.Open)
{
objcon.Close();
}
objcmd.Dispose();
return true;
}
catch (Exception ex) { throw new Exception("Error: In Close connesction"); return false; }
}
public static int ExecuteQuery(SqlCommand sqlcmd)
{
try
{
Dataconnect objdc = new Dataconnect();
int affectedrecord = 0;
if (objdc.Opencon() == true)
{
sqlcmd.Connection = objdc.objcon;
affectedrecord = sqlcmd.ExecuteNonQuery();
objdc.Closecon();
objdc = null;
return affectedrecord;
}
else { return affectedrecord; }
}
catch (Exception ex) { throw ex;/* new Exception("Error: In ExecuteNonquery");*/ }
}
public static DataTable Generatedatatable(SqlCommand sqlcmd)
{
try { Dataconnect objdc = new Dataconnect();
if (objdc.Opencon() == true)
{
sqlcmd.Connection = objdc.objcon;
SqlDataReader dr;
DataTable objdt = new DataTable();
dr = sqlcmd.ExecuteReader();
objdt.Load(dr);
objdc.Closecon();
objdc = null;
return objdt;
}
else { return null; }
}
catch (Exception Exception) { throw Exception /*new Exception("Error: In Generatedatatable")*/; }
}