What is the defined behavior of SQLite when interleaving statements that affect each other? - sqlite

Question

In SQLite if I prepare a SELECT statement and begin stepping through it, then before the last row of the results is reached I execute another statement that has an effect on the SELECT statement that I am stepping through, what is the expected result?
I can't find anything in the SQLite documentation about what is supposed to happen but it seems like an extremely common case when programming in a multi-threaded environment.
Below is a c++ file that can be compiled and run on Windows to demonstrate the situation.
#include "stdafx.h"
#include "sqlite3.h"
#include <Windows.h>
#include <iostream>
#include <Knownfolders.h>
#include <Shlobj.h>
#include <wchar.h>
#include <comdef.h>
using namespace std;
int exec_sql(sqlite3 *db, const char* sql)
{
char *errmsg;
int result = sqlite3_exec(db, sql, NULL, NULL, &errmsg);
if (result != SQLITE_OK) {
cout << errmsg << endl;
return -1;
}
return 0;
}
int _tmain(int argc, _TCHAR* argv[])
{
cout << "Running jsqltst with SQLite version: ";
cout << sqlite3_libversion();
cout << endl;
PWSTR userhome;
if (!SUCCEEDED(SHGetKnownFolderPath(FOLDERID_Profile, NULL, NULL, &userhome))) {
cout << "Failed getting user home dir\n";
return -1;
}
wcout << "User home: " << userhome << endl;
wchar_t *ws1 = userhome, *ws2 = L"\\test.sqlite";
wstring dbpath_str(ws1);
dbpath_str += wstring(ws2);
_bstr_t dbpath(dbpath_str.c_str());
cout << "DB path: " << dbpath << endl;
sqlite3 *db;
int result = sqlite3_open_v2(dbpath, &db, SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE, NULL);
if (result != SQLITE_OK) {
cout << sqlite3_errmsg(db) << endl;
return -1;
}
const char * create_stmt = "CREATE TABLE IF NOT EXISTS atable (id INTEGER PRIMARY KEY, name TEXT, number INTEGER);";
if (exec_sql(db, create_stmt) != 0) {
return -1;
}
const char * delete_stmt = "DELETE FROM atable;";
if (exec_sql(db, delete_stmt) != 0) {
return -1;
}
const char * insert_stmt = "INSERT INTO atable (name,number) VALUES ('Beta',77),('Alpha',99);";
if (exec_sql(db, insert_stmt) != 0) {
return -1;
}
sqlite3_stmt* select_ss;
const char * select_stmt = "SELECT * FROM atable;";
result = sqlite3_prepare_v2(db, select_stmt, -1, &select_ss, NULL);
if (result != SQLITE_OK) {
cout << sqlite3_errmsg(db) << endl;
return -1;
}
int i = 0;
boolean gotrow;
do {
result = sqlite3_step(select_ss);
gotrow = result == SQLITE_ROW;
if (gotrow) {
i++;
cout << "I got a row!" << endl;
if (i == 1) {
if (exec_sql(db, insert_stmt) != 0) {
return -1;
}
}
}
} while (gotrow);
cout << "Last result: " << result << ", errstr: " << sqlite3_errstr(result) << endl;
result = sqlite3_finalize(select_ss);
if (result != SQLITE_OK) {
cout << sqlite3_errmsg(db) << endl;
return -1;
}
return 0;
}

Answer 1

SQLite's behaviour for concurrent statements in the same transaction is neither documented nor defined.
As you have seen, newly inserted records might be seen when a SELECT's cursor has not yet reached that part of the table.
However, if SQLite needed to create a temporary result table for sorting or grouping, later changes in the table will not appear in that result.
Whether you have a temporary table or not might depend on decisions made by the query optimizer, so this is often not predictable.
If multiple threads access the same connection, SQLite will lock the DB around each sqlite3_step call.
This prevent data corruption, but you will still have the problem that automatic transaction end when their last active statement ends, and that explicit transaction will fail the COMMIT if there is some other active statement.
Multi-threaded programs are better off using (at least) one connection per thread.