How to debug SQLite queries inside a Qt application? - sqlite

Question

Qt's QSqlDatabase can use SQLite as its backend. I'm trying to find out how to properly debug and optimize my SQLite queries (esp. their execution time) when they run inside a Qt application. This is esp. relevant when there is a difference between their behaviour in a usual SQLite client (such as the sqlite3 command-line client) and inside a Qt application.
Usual debugging and diagnostic techniques no longer work in this context; for example I can't use the dot commands to get information about indexes, track execution time etc. because these commands are specific to the sqlite3 CLI software.

Answer 1

Here are the techniques I found to be the most useful:
Use the CLI. The most obvious is to connect with the sqlite3 command-line client to the same database you are using from your application, and to look up information about indexes etc. there. It also helps to paste a query there that did not work when executed via Qt in order to get more detailed error messages. Obviously this technique won't work when the error is due to a difference of the SQLite3 library used inside Qt vs. the system's library.
Use DB Browser for SQLite. The open source DB Browser for SQLite (sqlitebrowser) is great for debugging issues with SQLite queries that happen in a Qt desktop application but not in the sqlite3 CLI tool. Because this tool is made with Qt itself and can be installed from the Ubuntu repos (sudo apt install sqlitebrowser), so it uses the same Qt as you would use when developing a Qt application for desktop use on that system. That way, it also uses the ame SQLite library that Qt itself uses, whether that is the system's SQLite library or Qt's bundled SQLite library (which depends on how Qt was compiled).
Note that with the default settings queries will take ~4 times longer to execute in sqlitebrowser than in the sqlite3 CLI or in your Qt application. This is approximately proportional for all queries, so it can still be used to improve query efficiency. It can probably be changed on the "PRAGMA" settings page of the software.
Use pragma functions. A difficulty when running diagnostic SQLite queries inside your Qt application is how to obtain their output. There will be no debug messages on the console from the SQLite library. But fortunately most PRAGMA queries that produce output (i.e. diagnostic ones) are also available as PRAGMA functions for use inside SELECT queries. This way, you can return information about indexes etc. inside the tabled results of your SELECT queries.
Use EXPLAIN queries. The SQLite EXPLAIN QUERY PLAN query (and to a lesser degree also the EXPLAIN query) are very useful to analyze why a query is slow. When run inside the sqlite3 CLI software, a EXPLAIN QUERY PLAN query produces a tree-like diagram drawn to the screen. So it does not seem to be a query one could use from inside Qt's SQL implementation, but that actually works. Because this represents actual tabular data, as can be seen from running such a query in any SQLite client that is not the sqlite3 CLI. Example output:
EXPLAIN QUERY PLAN SELECT [...];
id parent notused detail
6 0 0 SEARCH TABLE products USING COVERING INDEX idx_code (code=?)
10 0 0 SEARCH TABLE product_categories USING PRIMARY KEY (product_id=?)
20 0 0 SEARCH TABLE categories USING INTEGER PRIMARY KEY (rowid=?)
Recompile SQLite for debugging. The most extreme measure is to recompile Qt's SQLite3 library with debugging enabled and then to switch on debugging output with PRAGMA vdbe_debug=1;. This should print debug output to stdout while queries run. (I did not test this and it may still be that Qt intercepts this output. But probably not.)