Copying in-memory database to disk crashes on db.close() - sqlite

Question

Using Qt 5.15.2 on Windows 10 x64
My sqlite3.h says
#define SQLITE_VERSION "3.34.0"
#define SQLITE_VERSION_NUMBER 3034000
#define SQLITE_SOURCE_ID "2020-12-01 16:14:00 a26b6597e3ae272231b96f9982c3bcc17ddec2f2b6eb4df06a224b91089fed5b"
I'm copying a temp database to disk (:memory: if release, tmp file if debug) according to the sqlite3 documentation and other stuff I've found on this site.
How to backup/store between sqlite memory database and file database in Qt?
How to access sqlite3 directly from Qt without linking sqlite3.dll a second time
The problem is if the database is over a given size, the db.close() line crashes the program, showing me this.
The database is created like this. If I use 100 instead of 1000 it seems to work fine, no crash.
void SymbolLibDocument::init() {
if (m_activeState) {
return;
} else {
// Creates temp database to prime save, save as, etc.
// Does not create anything with full filename
QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE", QString::fromStdString(m_connName));
#if defined(QT_DEBUG)
db.setDatabaseName("tmp_" + QString::fromStdString(m_name));
#elif defined(QT_NO_DEBUG)
db.setDatabaseName(":memory:");
#endif
if(!db.open()) {
qDebug() << "Can't create database";
}
QSqlQuery query(db);
const
QStringList qsl = {"DROP TABLE IF EXISTS hello;",
"CREATE TABLE hello (ID INTEGER PRIMARY KEY AUTOINCREMENT, \n"
" name TEXT CHECK(length(name) > 0)); ",
"INSERT INTO hello (name) VALUES ('giraffe');"};
dbutils::executeList(query, qsl, "Could not init", __LINE__);
query.exec("BEGIN");
QString s = QString("INSERT INTO hello (name) VALUES (:v);");
query.prepare(s);
for (int i = 0; i < 10000; i++) {
query.bindValue(":v", QVariant(i));
query.exec();
}
query.exec("COMMIT");
}
m_activeState = true;
}
void dbSaveFromTo(const std::string & connFrom, const std::string & fileTo) {
// Uses sqlite3 backup mechanism to write database connFrom to fileTo
//auto secs = std::chrono::milliseconds(50);
QString qsConnFrom = QString::fromStdString(connFrom);
// Need to clone db so it can be used from this function, which is callable
// as another thread.
// Also this needs to go in another scope so db object is destroyed
// at exit, prior to close and removal
{
QSqlDatabase db = QSqlDatabase::cloneDatabase(qsConnFrom, "CloneDb");
db.open();
QVariant qvhandle = db.driver()->handle();
if (qvhandle.isValid() && qstrcmp(qvhandle.typeName(), "sqlite3*") == 0) {
sqlite3 *pFrom = *static_cast<sqlite3 **>(qvhandle.data());
sqlite3 *pTo;
sqlite3_open(fileTo.c_str(), &pTo);
sqlite3_backup *pBackup = sqlite3_backup_init(pTo, "main", pFrom, "main");
if (pBackup) {
int pagesPerCycle = 1;
int pageCount = 0;
int pagesCopied = 0;
int pagesRemaining = 0;
do {
(void) sqlite3_backup_step(pBackup, pagesPerCycle);
if (sqlite3_errcode(pFrom) != SQLITE_OK) {
qDebug() << sqlite3_errmsg(pFrom);
}
if (sqlite3_errcode(pTo) != SQLITE_OK) {
qDebug() << sqlite3_errmsg(pTo);
}
pageCount = sqlite3_backup_pagecount(pBackup);
pagesRemaining = sqlite3_backup_remaining(pBackup);
pagesCopied = pageCount - pagesRemaining;
emit intEmitter.emitInt(pagesCopied, pageCount);
qDebug() << "emitting" << pagesCopied << "/" << pageCount;
//std::this_thread::sleep_for(secs);
} while(pagesRemaining > 0);
(void) sqlite3_backup_finish(pBackup);
} else {
throw std::logic_error("sqlite3_backup_init(...) failed");
}
// causes error 21 bad parameter or other API misuse,
// But this occurs even if open and close are called back-to-back
// with no operations in between
sqlite3_close(pTo);
if (sqlite3_errcode(pFrom) != SQLITE_OK) {
qDebug() << sqlite3_errmsg(pFrom);
}
if (sqlite3_errcode(pTo) != SQLITE_OK) {
qDebug() << sqlite3_errmsg(pTo);
}
} else {
throw std::logic_error("invalid driver handle");
}
db.close();
}
QSqlDatabase::removeDatabase("CloneDb");
}
I'm concerned this is caused because I'm using the built-in Qt Sql functionality (QT += sql in .pro file) for db.open, db.close, QSqlDatabase::cloneDatabase, etc.., but I'm using a separate sqlite3 precompiled .dll to access the sqlite3_backup_* functions.
Why does this crash when I do db.close()?

Answer 1

The problem was indeed some weird interaction between the Qt sqlite code used to access the db connection and the sqlite3.dll code used to init and step through the backup.
The solution was to use the Qt Sqlite functionality only to get the filename from the connection name, then use the raw sqlite api to open the source file and do the backup.