I have a custom collation sequence that I named NOCASE_UTF8 that I'm using in a couple sqlite dbs for an iPhone app I made. I would love to use a sqlite gui on my Mac to be able to manage my dbs rather than doing it all through my app's C code. I would also like to not have to roll my own gui tool JUST to handle this one problem.
I've tried a couple sqlite guis and neither one allows you to interact with collation unless I made a horrible oversight. I've tried using SQLite Manager and Base to insert rows into my tables that use the custom collation. Both just blow up with the error message "no such collation sequence: NOCASE_UTF8". I saw this answer for a list of guis but nothing I've read of the guis talks about custom collation support.
Am I just SOL or is there a sqlite gui out there with custom collation support?
I haven't tested it, but this solution is open source so if it doesn't support custom collation, you could add it yourself... if you dare.
http://code.google.com/p/phpliteadmin/
There are very few options for Mac OS in this category. I've always used the Firefox addon, but I've never used weird collations.
Related
Qt's QSqlDatabase can use SQLite as its backend. I'm trying to find out how to properly debug and optimize my SQLite queries (esp. their execution time) when they run inside a Qt application. This is esp. relevant when there is a difference between their behaviour in a usual SQLite client (such as the sqlite3 command-line client) and inside a Qt application.
Usual debugging and diagnostic techniques no longer work in this context; for example I can't use the dot commands to get information about indexes, track execution time etc. because these commands are specific to the sqlite3 CLI software.
Here are the techniques I found to be the most useful:
Use the CLI. The most obvious is to connect with the sqlite3 command-line client to the same database you are using from your application, and to look up information about indexes etc. there. It also helps to paste a query there that did not work when executed via Qt in order to get more detailed error messages. Obviously this technique won't work when the error is due to a difference of the SQLite3 library used inside Qt vs. the system's library.
Use DB Browser for SQLite. The open source DB Browser for SQLite (sqlitebrowser) is great for debugging issues with SQLite queries that happen in a Qt desktop application but not in the sqlite3 CLI tool. Because this tool is made with Qt itself and can be installed from the Ubuntu repos (sudo apt install sqlitebrowser), so it uses the same Qt as you would use when developing a Qt application for desktop use on that system. That way, it also uses the ame SQLite library that Qt itself uses, whether that is the system's SQLite library or Qt's bundled SQLite library (which depends on how Qt was compiled).
Note that with the default settings queries will take ~4 times longer to execute in sqlitebrowser than in the sqlite3 CLI or in your Qt application. This is approximately proportional for all queries, so it can still be used to improve query efficiency. It can probably be changed on the "PRAGMA" settings page of the software.
Use pragma functions. A difficulty when running diagnostic SQLite queries inside your Qt application is how to obtain their output. There will be no debug messages on the console from the SQLite library. But fortunately most PRAGMA queries that produce output (i.e. diagnostic ones) are also available as PRAGMA functions for use inside SELECT queries. This way, you can return information about indexes etc. inside the tabled results of your SELECT queries.
Use EXPLAIN queries. The SQLite EXPLAIN QUERY PLAN query (and to a lesser degree also the EXPLAIN query) are very useful to analyze why a query is slow. When run inside the sqlite3 CLI software, a EXPLAIN QUERY PLAN query produces a tree-like diagram drawn to the screen. So it does not seem to be a query one could use from inside Qt's SQL implementation, but that actually works. Because this represents actual tabular data, as can be seen from running such a query in any SQLite client that is not the sqlite3 CLI. Example output:
EXPLAIN QUERY PLAN SELECT [...];
id parent notused detail
6 0 0 SEARCH TABLE products USING COVERING INDEX idx_code (code=?)
10 0 0 SEARCH TABLE product_categories USING PRIMARY KEY (product_id=?)
20 0 0 SEARCH TABLE categories USING INTEGER PRIMARY KEY (rowid=?)
Recompile SQLite for debugging. The most extreme measure is to recompile Qt's SQLite3 library with debugging enabled and then to switch on debugging output with PRAGMA vdbe_debug=1;. This should print debug output to stdout while queries run. (I did not test this and it may still be that Qt intercepts this output. But probably not.)
I'm looking at switching from MySQL Workbench to Navicat because we're using MariaDB and the incompatibilities are starting to annoy me.
I'm working through the issues of getting Navicat to run on Centos under WINE but assume I will succeed (edit: this failed. The "linux" version requires WINE. Navicat will sort of run with a bit of hacking, but critical features rely on MS-Windows/WINE)
How do I get Navicat to work with git (or any other source code control)? Workbench is sufficiently primitive that file changes either get picked up automatically or completely ignored (almost always a dialog "file on disk has changed, reload?")
Specific problems:
when adding new query files Navicat only seems to rescan the folder when I add a new query. Is there a smart way to do that? (edit: no. You can manually refresh one file at a time by right clicking)
model and query files are buried deep in the WINE tree. Can I relocate them or or symlinks work? I'd rather keep all the DB-related code in one repo, rather than having a special Navicat repo. (edit: yes, but the explanation of how to do so is lengthy)
is there a way to merge a model file if more than one person has changed it? Workbench can't do this but I'd really like the feature. (edit: no, never. Merge the schema SQL files instead)
Also, bonus question: can we make multiple edits using Navicat other than repeated use of the GUI? If I want to change (say) a bunch of columns from VARCHAR(255) to CHAR(20) I'd normally script that in SQL but Navicat models don't do reverse engineering, only "delete the table from the model then re-import it" so there doesn't seem to be a non-tedious way to do that. (edit: no, but they might look at it in the future)
Final edit: I used the Navicat forums and the team were very helpful, but fundamentally Navicat is Windows software and the 64-bit purists behind Centos will never support WINE. For most Linux users this is not a problem, but I work with Centos enthusiasts and have long since lost the argument about which distro to use.
To the 1st question, you can sync it in different ways with a remote database/folder, when you are managing the database with Navicat, just right-click in your current connection and press "refresh", so you will be updated with the server changes. You also can do it with a programmed task.
Another matter is, why would you want to run navicat from wine when it has a native linux version? (I hope that answers the 2nd question)
For the 3rd question note that Navicat has an internal utility to sync data between servers, so you don't need git at all, or at most, you can automate the structure exportation and then sync it with a git repository (in form of a .sql file)
IMHO you need to review your concepts about mariadb and navicat, both are quite flexible and offer several ways to do such things you propose, like sync the data and they also allow to insert git in the workflow, just review your strategy and try to apply some new perspective with the available features.
I am writing a installer for windows using NSIS. The installer gets few properties during installation and it needs to update one of the table in sqlite database that is bundled with the installer. Is it possible to update sqlite database file using NSIS?
It does not seem like there are any SQLite plugins.
Your options are:
Write your own plugin (included for completeness, but almost certainly not a real option)
Use nsExec to run SQLite commands via the command line interface. See discussion on NSIS forums
Write a small app to include with your installer that makes the required changes
Decision probably depends on how well you know the command line interface for SQLite vs. complexity of writing a small app to do what you want.
For #3, it would be similar to what you would do with a third party installer:
ReserveFile "myexe.exe"
...
SetOutPath $TEMP
File "myexe.exe"
ExecWait '"$TEMP\myexe.exe" /parameters"
alt option: http://sourceforge.net/projects/nsissqliteplug/
nsisSqlplugin::executeQuery "sqliteDatabase" "sql_query"
Limitations:
Currently the plugin executes only insert and update queries.
I'm writing a Ruby program to manage courses run at a university, the modules associated with those courses, and the students registered on the courses and modules. I'm using RubyMine to write the program, and I now want to connect what I've written so far to an SQLite database to check that it works as I expect. But I'm not too sure how to do this in RubyMine.
I've opened the database tool window, and it says "No data sources configured", as well as having two "loading" messages at the top of the window.
If I right click on the window, I get a few options, one of which says "Add data source", from which I can add a "DB data source", or a "DDL data source".
I've only ever written one Ruby program before (about 10 months to a year ago), and I used the command line to write it then, also using the command line to create and edit the SQLite database. How can I set up an SQLite database to run with my program from RubyMine?
Also, is there a huge difference between SQLite and SQLite 3? Are there reasons for using one over the other? Which one would people recommend I use?
EDIT 24/08/2012 at 13:50
I tried selecting the option "Run rake tast" from the 'Tools' menu in RubyMine- it then asked me to enter task name or its part, I tried entering "db:migrate", but it said "no matches found".
Any idea what I should do instead?
Please refer to the documentation, RubyMine connects to the database via the JDBC driver. Most likely you want to use SQLite 3 version, as it's the current version at the moment. Check this video for the overview of the Data Sources features.
To connect your program to the database you don't need JDBC driver, it can be done with the sqlite3-ruby gem.
You can also consider some ORM, like Rails ActiveRecord or DataMapper, or Sequel.
I have a Monotouch app which uses a sqlite database. I want to encrypt the database so I am doing this:
_mainConnection = new SqliteConnection("Uri="+finalDB);
_mainConnection.Open();
_mainConnection.ChangePassword("mypassword");
However, its not working (on simulator and iphone). It gets this error:
at (wrapper managed-to-native)
Mono.Data.Sqlite.UnsafeNativeMethods.sqlite3_rekey
(intptr,byte[],int) <0x0005c> at
(wrapper managed-to-native)
Mono.Data.Sqlite.UnsafeNativeMethods.sqlite3_rekey
(intptr,byte[],int) <0x0005c> at
Mono.Data.Sqlite.SQLite3.ChangePassword
(byte[]) <0x00053> at
Mono.Data.Sqlite.SqliteConnection.ChangePassword
(byte[]) <0x0004b> at
Mono.Data.Sqlite.SqliteConnection.ChangePassword
(string) <0x0005b>
Has anyone successfully used password protection on an sqlite database in Monotouch?
As per my research there are a few options for database encryption using MonoTouch. I have a forthcoming blog post on the subject, but for now these are your top two options:
SQLCipher
I've automated the SQLCipher build process substantially. All it takes is a simple make command and you've got a library that you can link into your project. It makes use of the awesome SQLite-NET library. After that, all that's required is to provide the key in the SQLite.cs file.
SQLCipherNet: https://github.com/anujb/SQLCipherNet
CSharp-SQLite
This is a managed port of the SQLite library in C#. Performance is only about ~2x slower, which is pretty awesome considering it's not native code!
Encryption: http://code.google.com/p/csharp-sqlite/wiki/ENCRYPTION
Perf Benchmarks: http://code.google.com/p/csharp-sqlite/wiki/Benchmarks
Try adding ";Password=mypassword" to your connection string, and remove the call to ChangePassword.
Please note that, by default, the iPhone implementation of sqlite does not support encryption, so the sqlite commands for that will be no-ops.
You can get a (paid) copy of the encrypt-able version of sqlite from http://www.hwaci.com/sw/sqlite/see.html, and compile it into your application, making sure to remove the libsqlite3*.dylib from your project if you've linked that in.
You may have to do a bit of digging in the Monotouch documentation and/or experimentation to make sure that the Monotouch library itself is not including the default sqlite implementation, but in fact links to the implementation you specify. Try it first, if things still don't work that's where I'd start looking.
You can do this experiment without paying for the encrypted version, simply using the sqlite3 source code available on the net, with appropriate break points.
Good luck!
PS: Note that there is no comparable solution for Android at this point, this works on iPhone because iPhone runs native C code.
PPS: There is also SQLCipher that claims to encrypt sqlite on iPhone. However I found the configuration requirements to be below my standards for simplicity. I'm also not sure if it will properly insert itself between Monotouch's framework code and the default iPhone sqlite implementation.
SQLCipher for MonoTouch provides full database encryption for SQLite databases.
http://sqlcipher.net/sqlcipher-for-monotouch
There is also a SQLCipher on Mono for Android, which allows you to reuse the same code across Mono Touch and MonoDroid applications
http://sqlcipher.net/sqlcipher-for-monodroid
Just thinking out loud but could this be due to sqlite's dynlib that comes with the iPhoneSDK not being threadsafe?
For an alternative you might try looking at WWDC Vid 209 and just lock/encrypt the DB when you're outside the app.
You can probably do it yourself by issuing a "pragma rekey" in a raw SQLite query -- that is, if the SQLite version installed is actually SqlCipher.
I had the same problem but with a windows form application in C#.
I could not find the solution so i had to encrypt my data manually when saving it and decrypt it when retrieving.