`I am facing memory leak on each connection creation with database in libodbc++ library (0.2.5). I have copied valgrind log here. Used the below code to create and delete the connection. Any suggestion to fix the issue. Is libodbc++ stable to use ?
Sample Code:
`while(1)
{
Connection* con = DriverManager::getConnection(ConnectionString);
if(con != NULL)
{
delete con;
con = NULL;
}
}`
Valgrind Log:
==7534== 283,173,984 (1,074,640 direct, 282,099,344 indirect) bytes in 9,595 blocks are definitely lost in loss record 7,279 of 7,279 ==7534== at 0x4CE9B0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==7534== by 0x6FEB3B7: __gconv_open (gconv_open.c:74) ==7534== by 0x6FEB070: iconv_open (iconv_open.c:40) ==7534== by 0x3172BEB: unicode_setup (__info.c:601) ==7534== by 0x3136BD7: __SQLAllocHandle (SQLAllocHandle.c:492) ==7534== by 0x310DC4F: odbc::DriverManager::_createConnection() (drivermanager.cpp:140) ==7534== by 0x310DDBC: odbc::DriverManager::getConnection(std::__cxx11::basic_string<char, std::char_traits\<char="">, std::allocator\<char> > const&, std::__cxx11::basic_string<char, std::char_traits\<char="">, std::allocator\<char> > const&, std::__cxx11::basic_string<char, std::char_traits\<char="">, std::allocator\<char> > const&) (drivermanager.cpp:183) ==7534== by 0x</char></char,></char></char,></char></char,>
I have tried debugging libodbc++ library. SQLAllocHandle and SQLFreeHandle called properly for each connections and SQLFreeHandle returns SQL_SUCCESS. Memory not released by SQLFreeHandle. I didnt face connection leak. `
Related
I am using Apache Spark 1.5.1 and trying to connect to a local SQLite database named clinton.db. Creating a data frame from a table of the database works fine but when I do some operations on the created object, I get the error below which says "SQL error or missing database (Connection is closed)". Funny thing is that I get the result of the operation nevertheless. Any idea what I can do to solve the problem, i.e., avoid the error?
Start command for spark-shell:
../spark/bin/spark-shell --master local[8] --jars ../libraries/sqlite-jdbc-3.8.11.1.jar --classpath ../libraries/sqlite-jdbc-3.8.11.1.jar
Reading from the database:
val emails = sqlContext.read.format("jdbc").options(Map("url" -> "jdbc:sqlite:../data/clinton.sqlite", "dbtable" -> "Emails")).load()
Simple count (fails):
emails.count
Error:
15/09/30 09:06:39 WARN JDBCRDD: Exception closing statement
java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (Connection is closed)
at org.sqlite.core.DB.newSQLException(DB.java:890)
at org.sqlite.core.CoreStatement.internalClose(CoreStatement.java:109)
at org.sqlite.jdbc3.JDBC3Statement.close(JDBC3Statement.java:35)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$$anon$1.org$apache$spark$sql$execution$datasources$jdbc$JDBCRDD$$anon$$close(JDBCRDD.scala:454)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$$anon$1$$anonfun$8.apply(JDBCRDD.scala:358)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$$anon$1$$anonfun$8.apply(JDBCRDD.scala:358)
at org.apache.spark.TaskContextImpl$$anon$1.onTaskCompletion(TaskContextImpl.scala:60)
at org.apache.spark.TaskContextImpl$$anonfun$markTaskCompleted$1.apply(TaskContextImpl.scala:79)
at org.apache.spark.TaskContextImpl$$anonfun$markTaskCompleted$1.apply(TaskContextImpl.scala:77)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.TaskContextImpl.markTaskCompleted(TaskContextImpl.scala:77)
at org.apache.spark.scheduler.Task.run(Task.scala:90)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
res1: Long = 7945
I got the same error today, and the important line is just before the exception:
15/11/30 12:13:02 INFO jdbc.JDBCRDD: closed connection
15/11/30 12:13:02 WARN jdbc.JDBCRDD: Exception closing statement
java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (Connection is closed)
at org.sqlite.core.DB.newSQLException(DB.java:890)
at org.sqlite.core.CoreStatement.internalClose(CoreStatement.java:109)
at org.sqlite.jdbc3.JDBC3Statement.close(JDBC3Statement.java:35)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$$anon$1.org$apache$spark$sql$execution$datasources$jdbc$JDBCRDD$$anon$$close(JDBCRDD.scala:454)
So Spark succeeded to close the JDBC connection, and then it fails to close the JDBC statement
Looking at the source, close() is called twice:
Line 358 (org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD, Spark 1.5.1)
context.addTaskCompletionListener{ context => close() }
Line 469
override def hasNext: Boolean = {
if (!finished) {
if (!gotNext) {
nextValue = getNext()
if (finished) {
close()
}
gotNext = true
}
}
!finished
}
If you look at the close() method (line 443)
def close() {
if (closed) return
you can see that it checks the variable closed, but that value is never set to true.
If I see it correctly, this bug is still in the master. I have filed a bug report.
Source: JDBCRDD.scala (lines numbers differ slightly)
I am not able to login in GGS_OWNER from within GGSCI. I am able to login from sqlplus but when I try to login thorugh GGSCI it throws an error:
GG Version: 12.1.2
Database version: 12c
Type of replication: Integrated Capture
Replication setup: Unidirectional DML replication
Source Context :
SourceModule : [ggapp.util.pcs]
SourceID : [/scratch/aime1/adestore/views/aime1_adc4150327/oggcore/OpenSys/src/gglib/ggapp/pcsutl.c]
SourceFunction : [AbendHandler]
SourceLine : [1005]
ThreadBacktrace : [21] elements
: [/ggsadmin/app/ggate/12.1.2/libgglog.so(CMessageContext::AddThreadContext()+0x1e) [0x7f79191a286e]]
: [/ggsadmin/app/ggate/12.1.2/libgglog.so(CMessageFactory::CreateMessage(CSourceContext*, unsigned int, ...)+0x340) [0x7f791919d580]]
: [/ggsadmin/app/ggate/12.1.2/libgglog.so(_MSG_ERR_SIGNAL_RECEIVED(CSourceContext*, int, char const*, CMessageFactory::MessageDisposition)+0x3b) [0x7f791918078b]]
: [ggsci [0x50f653]]
: [/lib64/libpthread.so.0 [0x36b940eca0]]
: [/lib64/libpthread.so.0(pthread_mutex_lock+0) [0x36b9408dd0]]
: [ggsci(ggs::gglib::MultiThreading::Mutex::Lock()+0x9) [0x5449c9]]
: [ggsci(CContextItem::operator char const*() const+0x15) [0x4e2b3f]]
: [ggsci(DBOCI_init_connection_logon(ggs::gglib::ggapp::CLoginName const&, ggs::gglib::ggapp::CDBObjName<(DBObjType)12> const&, char const*, int, int, int, char*)+0x1e5) [0x553b85]]
: [ggsci [0x566ba4]]
: [ggsci(gl_db_login(char const*, char const*, ggs::gglib::ggapp::CDBObjName<(DBObjType)11>&, ggs::gglib::ggapp::CDBObjName<(DBObjType)12>&, short, char (&) [2048])+0x5f) [0x566edf]]
: [ggsci [0x4a4d12]]
: [ggsci(GGSCIDB_get_command(char const*, char const*, char const*, short, short, char (&) [2048])+0x11b) [0x4a525b]]
: [ggsci(do_cmd(char*, unsigned long, char*, unsigned long)+0xb14) [0x4dd654]]
: [ggsci [0x4e145b]]
: [ggsci(ggs::gglib::MultiThreading::MainThread::ExecMain()+0x4f) [0x54241f]]
: [ggsci(ggs::gglib::MultiThreading::Thread::RunThread(ggs::gglib::MultiThreading::Thread::ThreadArgs*)+0x104) [0x542624]]
: [ggsci(ggs::gglib::MultiThreading::MainThread::Run(int, char**)+0x8b) [0x54273b]]
: [ggsci(main+0x3f) [0x4bc94f]]
: [/lib64/libc.so.6(__libc_start_main+0xf4) [0x36b841d9f4]]
: [ggsci [0x49c379]]
2015-09-24 08:35:18 ERROR OGG-01117 Received signal: Segmentation violation (11).
2015-09-24 08:35:18 ERROR OGG-01668 PROCESS ABENDING.
Segmentation fault
You may be experiencing an issue with you path. Take a look at the LD_LIBRARY_PATH. Make sure it's pointing to your ORACLE_HOME/lib:/lib:/usr/lib directory.
Just a suggestion that you should always login to GGSCI from OGG_HOME path only. Because, when you login to GGSCI, GGSCI tries to read the params from the current path (and not the absolute path). This may lead to a few params not getting loaded when you login to GGSCI from any location other than OGG_HOME.
i.e.
cd $OGG_HOME
./ggsci
BUT NOT AS :
cd $OGG_HOME/dirprm (for example)
../ggsci
I hope by now you would have figured out the way to fix this issue; would be better if you could share the DBLOGIN command you ran which led to error.
Below is the command format:
DBLOGIN SOURCEDB AAA USERID XXX, PASSWORD YYY, SESSIONCHARSET latin1
These links may be of use too:
SOURCEDB : https://docs.oracle.com/goldengate/1212/gg-winux/GWURF/gg_parameters149.htm#GWURF649
DBLOGIN : https://docs.oracle.com/goldengate/1212/gg-winux/GWURF/ggsci_commands059.htm#GWURF251
sqlite3_prepare function fails if DB is opened with SQLITE_OPEN_READONLY.
Error message: error #10: disk I/O error
Sqlite extended error code: SQLITE_IOERR_LOCK (3850)
Errno: EBADF 9 /* Bad file number */
Everything works fine if DB is opened with SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE |
SQLITE_OPEN_FULLMUTEX;
Any ideas on what may cause this problem?
rc=sqlite3_open_v2("example.db",&db,SQLITE_OPEN_READONLY,0);
sqlite3_busy_timeout(db,1000);
selectQuery = "select * from test;";
rc = sqlite3_prepare(db, selectQuery, strlen(selectQuery), &stmt, 0);
if(rc!=SQLITE_OK)
{
printf("sql error #%d: %s", rc, sqlite3_errmsg(db));
printf( "SQL ext error: %d\n", sqlite3_extended_errcode(db));
printf( "errno: %d\n", errno );
}
The problem is in OS lock implementation. Fcntl() doesn't allow to lock file if it is opened with read only permissions. As a result - SQLite can't do anything in read only mode.
I am rather new to Qt, maybe that's why I cant fully understand the child-parent concept. I need to perform some sql query. I set the QSqlQuery, perform the "prepare and bind" operation and exec it. Next I pass it to the model and display the data. The problem occurs when closing the window - I get a memory violation error. The error occurs only, when I create the model with a parent. Here's the code:
QSqlQuery query;
query.prepare(QString("SELECT \
%1 as nazwa \
, kontrahentid \
FROM kontrahent WHERE %2 ilike ?"
).arg(showWhat, searchBy) //handled above, no need to escape
);
query.addBindValue(searchString); //user input data - so bind it
if (!query.exec()) {
qDebug() << query.lastError();
QApplication::restoreOverrideCursor();
return;
}
if (model == NULL)
// model = new QSqlQueryModel; // app closes the window correctly
model = new QSqlQueryModel(this); // app crashes when closing the window
model->setQuery(query);
if (model->lastError().isValid()) {
qDebug() << model->lastError();
QApplication::restoreOverrideCursor();
return;
}
model->setHeaderData(0, Qt::Horizontal, "ID");
ui.kontrahenciList->setModel(model);
//ui.kontrahenciList->setModelColumn(1);
ui.kontrahenciList->show();
Here's the error I'm getting:
Unhandled exception at 0x0fe29f9a (qsqlpsqld.dll) in HurBudClientGUI.exe: 0xC0000005: Access violation reading location 0x00000004.
and the call stack:
qsqlpsqld.dll!QScopedPointer<QObjectData,QScopedPointerDeleter<QObjectData> >::data() Line 143 + 0x3 bytes C++
qsqlpsqld.dll!qGetPtrHelper<QScopedPointer<QObjectData,QScopedPointerDeleter<QObjectData> > >(const QScopedPointer<QObjectData,QScopedPointerDeleter<QObjectData> > & p) Line 919 + 0xb bytes C++
qsqlpsqld.dll!QPSQLDriver::d_func() Line 106 + 0x13 bytes C++
qsqlpsqld.dll!QPSQLResultPrivate::privDriver() Line 212 C++
qsqlpsqld.dll!QPSQLResultPrivate::deallocatePreparedStmt() Line 306 + 0xc bytes C++
qsqlpsqld.dll!QPSQLResult::~QPSQLResult() Line 328 C++
qsqlpsqld.dll!QPSQLResult::`scalar deleting destructor'() + 0xf bytes C++
Qt5Sqld.dll!QSqlQueryPrivate::~QSqlQueryPrivate() Line 94 + 0x23 bytes C++
Qt5Sqld.dll!QSqlQueryPrivate::`scalar deleting destructor'() + 0xf bytes C++
Qt5Sqld.dll!QSqlQuery::~QSqlQuery() Line 245 + 0x1e bytes C++
Qt5Sqld.dll!QSqlQueryModelPrivate::~QSqlQueryModelPrivate() Line 90 + 0x3d bytes C++
Qt5Sqld.dll!QSqlQueryModelPrivate::`scalar deleting destructor'() + 0xf bytes C++
Qt5Cored.dll!672cbf06()
[Frames below may be incorrect and/or missing, no symbols loaded for Qt5Cored.dll]
Qt5Cored.dll!672cb92a()
Qt5Cored.dll!672c03f4()
Qt5Cored.dll!67200dc4()
Qt5Cored.dll!67203608()
Qt5Sqld.dll!QSqlQueryModel::~QSqlQueryModel() Line 175 + 0x9 bytes C++
As I mentioned above: the error doesn't happen when (one of below):
I create QSqlQueryModel without the parent (model = new QSqlQueryModel;)
I pass "static" query to QSqlQueryModel (regardless of having the parent).
eg:
model->setQuery(
QSqlQuery(
QString("SELECT \
%1 as nazwa \
, kontrahentid \
FROM kontrahent"
).arg(showWhat)
)
);
What am I doing wrong?
And the real question is: what is the purpose for QSqlQueryModel having a parent? If I delete it manually in the window's destructor - is there any diffrence?
I guess this is a bug - I reported it on qt bugtracker:
https://bugreports.qt.io/browse/QTBUG-43889
This is the key part:
if (model == NULL)
// model = new QSqlQueryModel; // app closes the window correctly
model = new QSqlQueryModel(this); // app crashes when closing the window
The parent-child concept in Qt provides many features, with automatic child destruction being one of them.
If any QObject has other QObject set as a parent, then when parent QObject is deleted, the child object will also be deleted.
Now, you mentioned you do explicitly delete the model in destructor of the window. You should not. If you define this as a parent, then the model will be automatically deleted together with this, but it was already deleted by you in the destructor, so model is deleted twice, thus your error appears.
So I removed the "delete operation" code in the destructor. Error still occurs, the only difference is diffrent call stack.
error message:
Unhandled exception at 0x0f249f9a (qsqlpsqld.dll) in HurBudClientGUI.exe: 0xC0000005: Access violation reading location 0x00000004.
call stack:
qsqlpsqld.dll!QScopedPointer<QObjectData,QScopedPointerDeleter<QObjectData> >::data() Line 143 + 0x3 bytes C++
qsqlpsqld.dll!qGetPtrHelper<QScopedPointer<QObjectData,QScopedPointerDeleter<QObjectData> > >(const QScopedPointer<QObjectData,QScopedPointerDeleter<QObjectData> > & p) Line 919 + 0xb bytes C++
qsqlpsqld.dll!QPSQLDriver::d_func() Line 106 + 0x13 bytes C++
qsqlpsqld.dll!QPSQLResultPrivate::privDriver() Line 212 C++
qsqlpsqld.dll!QPSQLResultPrivate::deallocatePreparedStmt() Line 306 + 0xc bytes C++
qsqlpsqld.dll!QPSQLResult::~QPSQLResult() Line 328 C++
qsqlpsqld.dll!QPSQLResult::`scalar deleting destructor'() + 0xf bytes C++
Qt5Sqld.dll!QSqlQueryPrivate::~QSqlQueryPrivate() Line 94 + 0x23 bytes C++
Qt5Sqld.dll!QSqlQueryPrivate::`scalar deleting destructor'() + 0xf bytes C++
Qt5Sqld.dll!QSqlQuery::~QSqlQuery() Line 245 + 0x1e bytes C++
Qt5Sqld.dll!QSqlQueryModelPrivate::~QSqlQueryModelPrivate() Line 90 + 0x3d bytes C++
Qt5Sqld.dll!QSqlQueryModelPrivate::`scalar deleting destructor'() + 0xf bytes C++
Qt5Cored.dll!QScopedPointerDeleter<QObjectData>::cleanup(QObjectData * pointer) Line 62 + 0x20 bytes C++
Qt5Cored.dll!QScopedPointer<QObjectData,QScopedPointerDeleter<QObjectData> >::~QScopedPointer<QObjectData,QScopedPointerDeleter<QObjectData> >() Line 109 + 0x9 bytes C++
Qt5Cored.dll!QObject::~QObject() Line 940 + 0x15 bytes C++
Qt5Cored.dll!QAbstractItemModel::~QAbstractItemModel() Line 1454 + 0xf bytes C++
Qt5Cored.dll!QAbstractTableModel::~QAbstractTableModel() Line 3299 + 0x8 bytes C++
Qt5Sqld.dll!QSqlQueryModel::~QSqlQueryModel() Line 175 + 0x9 bytes C++
HurBudClientGUI.exe!QSqlQueryModel::`scalar deleting destructor'() + 0x10 bytes C++
Qt5Cored.dll!QObjectPrivate::deleteChildren() Line 1841 + 0x24 bytes C++
Qt5Widgetsd.dll!QWidget::~QWidget() Line 1488 C++
Qt5Widgetsd.dll!QDockWidget::~QDockWidget() Line 1172 + 0x22 bytes C++
HurBudClientGUI.exe!searchDock::~searchDock() Line 28 + 0x1c bytes C++
searchDock is the class I'm working with.
I guess I solved it. When the model is initialized as view's child:
model = new QSqlQueryModel(this);
The problem is order of operations. When I close the connection and remove the database in the destructor of my class - the database gets disconnected and no further operations are possible. But after my class' destructor, destructors of its base classes get into action - one by another, as ordered by inheritance. When QObject::~QObject() is executed, it gets to the
QObjectPrivate::deleteChildren()
method. And then, it finally gets to delete the model. The model wants to free the resources - which means QSqlResult (QPSQLResult to be specific in that case). This it how it looks:
QPSQLResult::~QPSQLResult()
{
Q_D(QPSQLResult);
cleanup();
if (d->preparedQueriesEnabled && !d->preparedStmtId.isNull())
d->deallocatePreparedStmt();
};
So here Qt tries to deallocate preparedStatement - regardless of the fact, that the connection no longer exists:
void QPSQLResultPrivate::deallocatePreparedStmt()
{
const QString stmt = QLatin1String("DEALLOCATE ") + preparedStmtId;
PGresult *result = privDriver()->exec(stmt);
if (PQresultStatus(result) != PGRES_COMMAND_OK)
qWarning("Unable to free statement: %s", PQerrorMessage(privDriver()->connection));
PQclear(result);
preparedStmtId.clear();
};
So - to make it work properly, I wolud have to call
QSqlQueryModel::~QSqlQueryModel()
or
QSqlQueryModel::clear()
BEFORE closing the connection with the DB. I still think it's a bug.
I have worked little bit in OpenCL now but recently "clBuildProgram" failed in one of my program. My code excerpt is below:
cl_program program;
program = clCreateProgramWithSource(context, 1, (const char**) &kernel_string, NULL, &err);
if(err != CL_SUCCESS)
{
cout<<"Unable to create Program Object. Error code = "<<err<<endl;
exit(1);
}
if(clBuildProgram(program, 0, NULL, NULL, NULL, NULL) != CL_SUCCESS)
{
cout<<"Program Build failed\n";
size_t length;
char buffer[2048];
clGetProgramBuildInfo(program, device_id[0], CL_PROGRAM_BUILD_LOG, sizeof(buffer), buffer, &length);
cout<<"--- Build log ---\n "<<buffer<<endl;
exit(1);
}
Normally earlier I got syntax or other errors inside kernel file here with the help of "clGetProgramBuildInfo()" function whenever "clBuildProgram" Failed but when this program runs, on console it only prints:
Program Build failed
--- Build log ---
And when I tried to print the error code returned by "clBuildProgram"; it is "-11"......
What can be the problem with my kernel file that I dont get any build fail information ?
You can learn the meaning of OpenCL error codes by searching in cl.h. In this case, -11 is just what you'd expect, CL_BUILD_PROGRAM_FAILURE. It's certainly curious that the build log is empty. Two questions:
1.) What is the return value from clGetProgramBuildInfo?
2.) What platform are you on? If you are using Apple's OpenCL implementation, you could try setting CL_LOG_ERRORS=stdout in your environment. For example, from Terminal:
$ CL_LOG_ERRORS=stdout ./myprog
It's also pretty easy to set this in Xcode (Edit Scheme -> Arguments -> Environment Variables).
If you are using the C instead of C++:
err = clBuildProgram(program, 0, NULL, NULL, NULL, NULL);
////////////////Add the following lines to see the log file///////////
if (err != CL_SUCCESS) {
char *buff_erro;
cl_int errcode;
size_t build_log_len;
errcode = clGetProgramBuildInfo(program, devices[0], CL_PROGRAM_BUILD_LOG, 0, NULL, &build_log_len);
if (errcode) {
printf("clGetProgramBuildInfo failed at line %d\n", __LINE__);
exit(-1);
}
buff_erro = malloc(build_log_len);
if (!buff_erro) {
printf("malloc failed at line %d\n", __LINE__);
exit(-2);
}
errcode = clGetProgramBuildInfo(program, devices[0], CL_PROGRAM_BUILD_LOG, build_log_len, buff_erro, NULL);
if (errcode) {
printf("clGetProgramBuildInfo failed at line %d\n", __LINE__);
exit(-3);
}
fprintf(stderr,"Build log: \n%s\n", buff_erro); //Be careful with the fprint
free(buff_erro);
fprintf(stderr,"clBuildProgram failed\n");
exit(EXIT_FAILURE);
}
I encountered the same problem with an empty log file. I was testing my ocl kernel on a different computer. It had 2 platforms instead of one. One Intel GPU and one AMD GPU. I only had AMD OCL SDK installed. Installing the Intel OCL SDK fixed the problem. Also selecting the AMD platform instead of the Intel GPU platform fixed it.
I've seen this happen on OSX 10.14.6 when the OpenCL kernel source is missing the _kernel attribute tag. If both the _kernel tag and return type are missing it seems to crash the system OpenCL compiler daemon, which then takes a few seconds to restart before new kernels will compile again.