How to manage database changes while upgrading desktop applications?
We have a SQLite database for one of our desktop apps. The uninstaller keeps the database to be used by the next install. But what if the next install has an updated version? How to keep the data but upgrade the tables?
We use .NET and Linq2sql
Here is how I do this:
In my code I define the version of the database
#define DB_VERSION 2
This version number is incremented every time I make a change to the code that 'breaks' the database ( makes an incompatible change to the schema or the semantics of the db contents )
When the code creates a new database, it executes this SQL command
QueryFormat(L"PRAGMA SCHEMA_VERSION = %d;",DB_VERSION);
Note that this must be AFTER all CREATE TABLE commands have been executed, otherwise sqlite increments SCHEMA_VERSION.
When the code opens an existing database, the first thing it does is
Query(L"PRAGMA schema_version;")
The SCHEMA_VERSION that is returned from this query is compared with the DB_VERSION. If they do not match, then the database was created by a different software version. What happens next depends on the details of what you need. Typically:
database was created by a more recent software: inform user and exit
database was created by older software for which there is an upgrade: offer user option to run upgrade code, or re-initialize database
database was created by older software with no upgrade: offer user option to re-initialize database or exit.
The details of how the upgrade code works depends very much on what you need. In general open the old database AND open a new empty database. Read the old tables, convert the data as required, and write to the new database. Close the dbs. Delete the old db. Rename the new db to the standard db name. Open the new db.
If nothing else, the output of echo .dump | sqlite my_database.sqlite is designed to be extremely portable, even to non-SQLite databases.
Or, if I'm misreading your question, you may want alter table.
Related
There are thousands of on-premises servers who are running all different versions of a piece of software including a specific version of a database (all at different companies)
Out of a central development department regularly new versions or pushed of the software with database updates (new tables, new views, new foreign keys, new inserts for enums or specific tables, new stored procedures, etc)
These changes come from many different development branches all bringing in their own bits of sql code that affects the schema
Updating client is handled via a .sql file which verifies the local installed latest database schema release version and the sql code that is annotated as later is ran to update a specific physical machine to a later version (automatically without user involvement)
An idea is now to instead use the EF SSDL description of the specific assembly distributed to bring a database schema (sql server) to the latest version. It would need to compare every table, column, constraint, etc to check if they are equal and if not update them taking into account dependencies.
Question: is there an existing package which would do this? So that e.g. when starting an application it would check the existing physical database versus the SSDL and brings them in sync automagically? It would have to skip database objects that are not in the SSDL since a database could contain tables or views which were added by a specific customer and is not part of the database objects needed by the application or is the current approach the best way to go? (in itself this is another approach than e.g. redgate with comparing the physical database) ?
(the only related question i found here is Purpose of edmgen validation? Comparing SSDL and database schemas?)
We have a Windows 8.1 universal app published on the Windows store. Due to a requirement change now we need to add some fields to an existing table. We have used SQLite for Windows Runtime (Windows 8.1) and SQLite for Windows Phone 8.1. Now if we add some new fields to an existing table (by updating the model classes) will that cause any issues when the existing users update the app?
Instead of adding new columns if we create a new table, then can that cause any issue or will that be safer than adding new fields?
Adding a new column to an existing table is perfectly safe even with an existing SQLite database. Same goes for creating a new table. The concrete process depends on which SQLite library you are using.
If you are querying the database directly using SQL queries, then you will use ALTER TABLE ... ADD COLUMN query. If you are using library like sqlite-net-pcl, you get automatic migrations out of the box when you run the CreateTable method. As documentation says:
The automatic migration, currently, only supports adding new columns. If your classes have new properties that are not associated with columns in the table, then alter table ... add column commands will be executed to bring the database up to date. These new columns will not have default values and will therefore be null.
Tip
When you change the model, always test out the changes against the previous version of the database. This can be done in two ways:
Install the existing version from Store, launch the app and use it so that it contains some data, then close it and go to C:\Users\[UserName]\AppData\Local\Packages. Here navigate to Packages and there find your app's package folder. Inside go to LocalState where you should find your sqlite database. Copy it somewhere, uninstall the Store version and launch the new version for debugging and place a breakpoint in the App.xaml.cs constructor. When the debugger stops there, copy the original database back to the same location in app package LocalState folder and continue debugging. Observe if the database is migrated properly.
Using Git source control, go in history to the previous release commit, install the app for debugging, fill the DB with some data, close the app. Now go to the latest commit and launch app for debugging again. Verify the migration works.
I am creating an app it takes data from the server and storing it in SQLite database but each time I run the app the data gets append in the database. I tried onUpgrade() method to drop table and create a new one, but its not working because db_version is the same. But if I change the version manually then its working... but I want solution so that my version changes each time automatically when I run the app....
OR if there is an another alternative please share it....
You are probably didn't understand what is database Upgrade. In two words you upgrade the version of your database when your DB model changed, not the data!
To clear your database for testing purposes use sqlite DROP TABLE or physically delete the file!
If you upgrade the version of the database each time your app starts - you will lose all the database data each time your app starts, so why do you need the database ?
I have a website solution that uses on SQLite database for each tenant account. Without going into much depth about why we chose this solution, we chose it due to SQLite support on distributed/offline systems.
All databases are manipulated using the same PHP file structure. I wish to update the database version iteratively for all accounts so that they are all at the same version number.
I have a script that loops over each, and can use either PHP(Yii) or the shell to execute queries.
I would like to wrap the manipulation to each database in a transaction. It appears as though DDL commands may already be auto-commit in nature.
Question: How to accomplish a SQLite DB upgrade which, if it fails, will report a failure? Using that report, I could prompt the system to re-attempt or report an error to an admin. Ideally, I would like to wrap the whole upgrade in a transaction to prevent inconsistencies, but I'm fairly certain that this is not possible.
One thought I had was to backup the database temporarily during upgrade, and delete it on success.
As CL stated in the comments, DDL is fully transactional in SQLite. So, for each database in our system, we wrap it in a transaction, and it executes atomically. We leverage the application to ensure that all databases upgrade successfully if any fail during upgrade.
I have an application in C# that uses System.Data.SQLite. In my case I use a recent version of SQL Lite database, by now I can see that the new versión has released, and int sqlite.org webpage says that is recommended to upgrade the database.
My question is how to upgrade without lost the information in my actual database.
How can I chech the version of the data
Thanks.
EDIT: what I mean is when I create a new database with the sqlite3 library, I guess that the database file, my database.db has a version. When I update the sqlite3 library, I am update the sqlite3 command line, but the database file still has the version that had when I created it.
So if in the new versions for example add new features to the database, for example triggers, foreign keys and so on, if I am not wrong, this features must be in the database file, not in the sqlite3 library, because when I access to the database for example with entity framework, I don't use sqlite3 library, I use System.SQLite.Data library.
am I wrong? the datafile is never update and only the library can be updated?
Thanks.
Upgrading the SQLite library will not have any effect on your database file.
Changes like foreign keys do not affect the database file.
The last change that affected the file format was a long time ago.