Is it possible to run sqlite dot commands, e.g. .backup / .restore from Java (Android) with for example SqliteOpenHelper or something else?
Currently I have to run the .backup command using shell using sqlite3 binary and I don't want to do that for efficiency reasons. Also it is cumbersome.
Otherwise, any other way to backup and restore from inside Java / Android?
Related
I am trying to start out using Notepad++ to run SQLite commands. I have tried following two brief YouTube tutorials to get me going. I can run the initial .bat file, but still cannot run the .sql file.
I have a Windows system environment Path variable set to the folder containing sqlite3.exe
"C:\Users\Adam\sqlite\"
I have saved the following file RunSQLite.bat in the folder containing sqlite3.exe
sqlite3.exe testDB.db
I have created a second file queries.sql
SELECT 34;
When I try to run queries.sql from Notepad++, using the RUN command:
C:\Users\Adam\sqlite\RunSQLite.bat "$(FULL_CURRENT_PATH)"
the only file that appears to run is RunSQLite.bat, giving the output:
SQLite version 3.36.0 2021-06-18 18:36:39
Enter ".help" for usage hints.
sqlite>
Can anyone tell where I have gone wrong?
Thanks in advance.
aphopk
This C:\Users\Adam\sqlite\RunSQLite.bat "$(FULL_CURRENT_PATH)" will do exactly the same thing if run at the shell. RunSQLite.bat does not take any arguments so the Run command in npp is working as expected.
sqlite3 takes input from an external file with the .read command.
Path issues notwithstanding a bat file something like this should accomplish the task:
sqlite3.exe testDB.db ".read %1"
Notepad++ is a text editor, so you can now use it to edit your SQL file. After selecting the Language > SQL, Notepad++ will highlight SQL syntax as you type. Try typing some SQL, like
SELECT "Hi";
SELECT * FROM mydatabase WHERE id LIKE 'ID%';
You will see color, bold, and other possible formatting applied to the text you type. If you save the file as something.sql, and then load something.sql in your SQL client, the client will run the SQL commands from that file. If you have an existing somethingElse.sql file, you can open it in Notepad++, which will auto-recognize that it’s SQL and apply the syntax highlighting, allowing you to edit it and save it.
By using the Run > Run dialog, you can run an arbitrary command. For example, if your SQL client has a command-line mode accessed thru sqlclient.exe, you could type
c:\path\to\sqlclient.exe $(FILE_NAME)
If you just run it, that probably won’t show you any results… but if you ran
cmd /k c:\path\to\sqlclient.exe $(FILE_NAME)
It will open a new cmd.exe Windows command prompt, and show the output from that file.
If instead of running, you hit “SAVE”, you can give it a name (which will end up later in the Run menu), and/or a keyboard shortcut, so that you can easily re-use that many times.
If you want to do something more fancy, use the NppExec plugin, which includes a better batch/scripting language. Once again, you can save the NppExec script, and make it show up in the Macro menu.
If Python is a programming language you know or could learn (or if, like me, you know enough other programming languages that you can fake the Python), then the Python Script plugin will allow you to do even fancier stuff. (Python is a complete programming language, and has many libraries written, which could act as an interface between your SQL source file and your database engine; PythonScript has access to a full python2.7 interpreter. For example, you could write a script in Python which executes the commands from your SQL, grabs the results from your database engine, and displays them in Notepad++, either inline with your original SQL code, or in a new text document. You are really limited only by your imagination and knowledge of Python.)
Qt's QSqlDatabase can use SQLite as its backend. I'm trying to find out how to properly debug and optimize my SQLite queries (esp. their execution time) when they run inside a Qt application. This is esp. relevant when there is a difference between their behaviour in a usual SQLite client (such as the sqlite3 command-line client) and inside a Qt application.
Usual debugging and diagnostic techniques no longer work in this context; for example I can't use the dot commands to get information about indexes, track execution time etc. because these commands are specific to the sqlite3 CLI software.
Here are the techniques I found to be the most useful:
Use the CLI. The most obvious is to connect with the sqlite3 command-line client to the same database you are using from your application, and to look up information about indexes etc. there. It also helps to paste a query there that did not work when executed via Qt in order to get more detailed error messages. Obviously this technique won't work when the error is due to a difference of the SQLite3 library used inside Qt vs. the system's library.
Use DB Browser for SQLite. The open source DB Browser for SQLite (sqlitebrowser) is great for debugging issues with SQLite queries that happen in a Qt desktop application but not in the sqlite3 CLI tool. Because this tool is made with Qt itself and can be installed from the Ubuntu repos (sudo apt install sqlitebrowser), so it uses the same Qt as you would use when developing a Qt application for desktop use on that system. That way, it also uses the ame SQLite library that Qt itself uses, whether that is the system's SQLite library or Qt's bundled SQLite library (which depends on how Qt was compiled).
Note that with the default settings queries will take ~4 times longer to execute in sqlitebrowser than in the sqlite3 CLI or in your Qt application. This is approximately proportional for all queries, so it can still be used to improve query efficiency. It can probably be changed on the "PRAGMA" settings page of the software.
Use pragma functions. A difficulty when running diagnostic SQLite queries inside your Qt application is how to obtain their output. There will be no debug messages on the console from the SQLite library. But fortunately most PRAGMA queries that produce output (i.e. diagnostic ones) are also available as PRAGMA functions for use inside SELECT queries. This way, you can return information about indexes etc. inside the tabled results of your SELECT queries.
Use EXPLAIN queries. The SQLite EXPLAIN QUERY PLAN query (and to a lesser degree also the EXPLAIN query) are very useful to analyze why a query is slow. When run inside the sqlite3 CLI software, a EXPLAIN QUERY PLAN query produces a tree-like diagram drawn to the screen. So it does not seem to be a query one could use from inside Qt's SQL implementation, but that actually works. Because this represents actual tabular data, as can be seen from running such a query in any SQLite client that is not the sqlite3 CLI. Example output:
EXPLAIN QUERY PLAN SELECT [...];
id parent notused detail
6 0 0 SEARCH TABLE products USING COVERING INDEX idx_code (code=?)
10 0 0 SEARCH TABLE product_categories USING PRIMARY KEY (product_id=?)
20 0 0 SEARCH TABLE categories USING INTEGER PRIMARY KEY (rowid=?)
Recompile SQLite for debugging. The most extreme measure is to recompile Qt's SQLite3 library with debugging enabled and then to switch on debugging output with PRAGMA vdbe_debug=1;. This should print debug output to stdout while queries run. (I did not test this and it may still be that Qt intercepts this output. But probably not.)
I'd like to access my previous inputs on the sqlite CLI tool. In bash it works with the arrow up-key but in after running sqlite3 and pressing that key I'm just getting ^[[A. How can I fix this? Which key lets me use my previous typed commands?
Thank you in advance.
Your sqlite3 binary was not built with support for readline.
Either get one that does. Or you can wrap it with readline via rlwrap.
rlwrap sqlite3
I am writing a installer for windows using NSIS. The installer gets few properties during installation and it needs to update one of the table in sqlite database that is bundled with the installer. Is it possible to update sqlite database file using NSIS?
It does not seem like there are any SQLite plugins.
Your options are:
Write your own plugin (included for completeness, but almost certainly not a real option)
Use nsExec to run SQLite commands via the command line interface. See discussion on NSIS forums
Write a small app to include with your installer that makes the required changes
Decision probably depends on how well you know the command line interface for SQLite vs. complexity of writing a small app to do what you want.
For #3, it would be similar to what you would do with a third party installer:
ReserveFile "myexe.exe"
...
SetOutPath $TEMP
File "myexe.exe"
ExecWait '"$TEMP\myexe.exe" /parameters"
alt option: http://sourceforge.net/projects/nsissqliteplug/
nsisSqlplugin::executeQuery "sqliteDatabase" "sql_query"
Limitations:
Currently the plugin executes only insert and update queries.
I have been developing locally for some time and am now pushing everything to production. Of course I was also adding data to the development server without thinking that I hadn't reconfigured it to be Postgres.
Now I have a SQLite DB who's information I need to be on a remote VPS on a Postgres DB there.
I have tried dumping to a .sql file but am getting a lot of syntax complaints from Postgres. What's the best way to do this?
For pretty much any conversion between two databases the options are:
Do a schema-only dump from the source database. Hand-convert it and load it into the target database. Then do a data only dump from the source DB in the most compatible form of SQL dump it offers. Try loading that into the target DB. When you hit problems, script transformations to the dump using sed/awk/perl/whatever and try again. Repeat until it loads and the results match.
Like (1), hand-convert the schema. Then write a script in your preferred language that connects to both databases, SELECTs from one, and INSERTs into the other, possibly with some transformations of data types and representations.
Use an ETL tool like Talend or Pentaho to connect to both databases and convert between them. ETL tools are like a "somebody else already wrote it" version of (2), but they can take some learning.
Hope that you can find a pre-written conversion too. Heroku one called sequel that will work for SQLite -> PostgreSQL; is it available without Heroku and able to function without all the other Heroku infrastructure and code?
After any of those, some post-transfer steps like using setval() to initialize sequences is typically required.
Heroku's database conversion tool is called sequel. Here are the ruby gems you need:
gem install sequel
gem install sqlite3
gem install pg
Then this worked for me for a sqlite database file named 'tweets.db' in the current working directory:
sequel -C sqlite://tweets.db postgres://pgusername:pgpassword#localhost/pgdatabasename
PostgreSQL supports "foreign data wrappers", which allow you to directly access any data source through the DB, including sqlite. Even up to automatically importing the schema. You can then use create table localtbl as (select * from remotetbl) to get your data into the actual PG storage.
https://wiki.postgresql.org/wiki/Foreign_data_wrappers
https://github.com/pgspider/sqlite_fdw