This question already has answers here:
Room database migration if only new table is added
(7 answers)
Closed 2 years ago.
When using Room whenever new tables added to your database you have to create it in your migration. Unfortunately, Room doesn't have such method to create table by just giving class name. Something below is necessary to have
room.createTable(User::class)
The similar method exists for example in OrmLite
TableUtils.createTable(connectionSource, User::class.java)
The necessity comes from complexity of creating tables with just simple SQLite query. Currently, what you can do is within your migrate you write your create SQLite script
db.execSQL("CREATE TABLE IF NOT EXIST `User` (uid INTEGER NON NULL, PRYMARY KEY (`uid`))")
There is no problem in above method, but it gets complex and long SQLite script if your have for example 50 fields. Obviously, you don't write it by yourself, and there are two methods to get the Create Script automatically generated by Room for you so that you just copy past.
After building the app,AppDatabase_Impl will be generated and it will have all necessary table creations. You can get the query from there
You include exportSchema = true in your #Database annotation and it will create versionNumber.json schema of Room database within your schemas folder. You can get the create script from there.
But, both of above methods will require you to run the app without any proper migration (as you don't know correct query) and it will definitely crash. And after that, you have correct query which you can include in your migration method. I think this is not "professional" way of doing it. Plus even after you get the long SQLite query, it's not PR friendly and it's not just good practice to have long SQLite query which is can not be easily debugged.
So, I wanted to make object oriented way of creating tables while migrating. The only approach that I could think of was, obviously, using model data class and generate query according to each field of model. It will supposedly be like this
fun createTable(db: SupportSQLiteDatabase, clazz: KClass<*>) {
val fields = extractColumns(clazz)
val primaryKeys = fields
.filter { it.primaryKey }
.map { it.name }
val createQuery = "CREATE TABLE IF NOT EXISTS `${clazz.simpleName}` (" +
fields.joinToString(", ") { "`${it.name}` ${it.type} ${it.nonNull}" } +
", PRIMARY KEY (" + primaryKeys.joinToString(",") { "`$it`" } +
"))"
db.execSQL(createQuery)
}
fun extractColumns(clazz: KClass<*>): Array<Column>{
val columns = ArrayList<Column>()
for (field in clazz.declaredMemberProperties){
val name = field.findAnnotation<ColumnInfo>()?.name ?: field.name
val type = getSqlType(field.returnType)
val nonNull = if (field.returnType.isMarkedNullable) "" else "NON NULL"
val primaryKey = field.findAnnotation<PrimaryKey>() != null
columns.add(Column(name, type, nonNull, primaryKey))
}
return columns.toTypedArray()
}
But the problem is Room Annotations are all annotated with
#Retention(RetentionPolicy.CLASS) which is can only be accessed during compile time. They are not available during run time. So all of my findAnnotation methods will return null. I was thinking of doing creating during compile time, but couldn't think of how.
So, my question was is there any way to generate CREATE scripts during compile time and if so how to do it?
Apart from my mentioned way of solving, is there any other way of creating tables which doesn't involve first two methods of copy pasting?
And by the way, I am not considering fallbackToDestructiveMigration. I mean, who would want their users to lose all data?
As of current update of Room there is actually the way to create SQL queries using Annotation Processing. With Annotation processing you have to wrote small library that generate Room queries for you when you build it.
Creating Annotation Processing Library is not straightforward and here is related issue.
Room database migration if only new table is added
Related
I'm new to Android, and rather new to SQL in general.
I have a data model where I have a Text that consists of TextMetadata as well as a long string, which is the text content itself. So
Text {
metadata: {
author: string,
title: string
// more associated metadata
},
textContent: long string, or potentially array of lines or paragraphs
}
I'd like to load a list of the metadata for all texts on the App's landing page, without incurring the cost of reading all the long strings (or having operations be slowed down because the table has a column with a long string?).
What is the proper pattern here? Should I use two tables, and related them? Or can I use one table/one #Entity, with embedded metadata, and do some fancy stuff in the DAO to just list/sort/operate on the embedded metadata?
Most of my background is with NoSQL databases, so I could be thinking about this entirely wrong. Advice on the general best practices here would be helpful, but I guess I have two core questions:
Does having a long/very long string/TEXT column cause performance considerations when operating on that specific table/row?
Is there a clean way using Kotlin annotations to express embedded metadata that would make it easy to fetch in the DAO, without having use a long SELECT for each individual column?
This is a good question that is also relevant to other environments.
The Core Issue: How to store large data without effecting your database?
As a rule of thumb you should avoid storing information in your database that is not queryable. Large strings, images, or event metadata which you will never query - does not belong in your db. I was surprised when I realized how many design patterns there are regarding to mongo db (which are relevant to other noSQL databases as well)
So, we know that this data should NOT be stored in the DB. But, because the alternative (file system) is WAY worse than that (unless you would like to implement your own secured file-system-based store) - we should at least try to minimize its footprint.
Our Strategy: save large data chunks in a different table without defining it as an entity (there is no need to wrap it as entity anyway)
How Are We Going To Do That?
Well, thankfully, android room has a direct access to sqLite and it can be used directly (read the docs). This is the place to remind us that android room is built on-top of sqLite - which is (in my own opinion) a fascinating database. I enjoy working with it very much and it's just getting better as the time goes by (personal opinion). Advantages? we are still using android APIs while storing large data in a performant, unified and secure way. yay
Steps we are going to perform:
Initiate a class which will manage a new database - for storing large data only
Define a command that will create our table - constructed of 2 columns
key (primary key) - the id of the item
value - the item itself
In original db for the Text entity - define a column that will hold the id (key) of the large text stored
Whenever you save an item to your large items table - get the id and store it in your entity
You can of course use only 1 table for this.. but.. I know that sqLite requires a certain amount of understanding and it is NOT as easy as android room so.. it's your choice whenever to use 1 or 2 tables in your solution
Below is a code that demonstrates the main principal of my proposal
object LargeDataContract {
// all tables for handling large data will be defined here
object TextEntry : BaseColumns {
const val TABLE_NAME = "text_entry"
const val COLUMN_NAME_KEY = "key"
const val COLUMN_NAME_VALUE = "value"
}
}
// in the future - append this "create statement" whenever you add more tables to your database
private const val SQL_CREATE_ENTRIES =
"CREATE TABLE ${TextEntry.TABLE_NAME} (" +
"${TextEntry.COLUMN_NAME_KEY} INTEGER PRIMARY KEY," +
"${TextEntry.COLUMN_NAME_VALUE} TEXT)"
// create a helper that will assist you to initiate your database properly
class LargeDataDbHelper(context: Context) : SQLiteOpenHelper(context, DATABASE_NAME, null, DATABASE_VERSION) {
override fun onCreate(db: SQLiteDatabase) {
db.execSQL(SQL_CREATE_ENTRIES)
}
companion object {
// If you change the database schema, you must increment the database version. Also - please read `sqLite` documentation to better understand versioning ,upgrade and downgrade operations
const val DATABASE_VERSION = 1
const val DATABASE_NAME = "LargeData.db"
}
}
// create an instance and connect to your database
val dbHelper = LargeDataDbHelper(context)
// write an item to your database
val db = dbHelper.writableDatabase
val values = ContentValues().apply {
put(TextEntry.COLUMN_NAME_VALUE, "some long value goes here")
}
val key = db?.insert(TextEntry.TABLE_NAME, null, values)
// now take the key variable and store it in you entity. this is the only reference you should need
Bottom Line: This approach will assist you to gain as much performance as possible while using android APIs. Sure thing, not the most "intuitive" solution, but - this is how we gain performance and making great apps as well as educating ourselves and upgrading our knowledge and skillset. Cheers
I have taken over some code and I see that database updates are performed like this:
dbcon = DependencyService.Get<ISQLite>().GetConnection();
public void UpdateAnswered(string id)
{
lock(locker)
{
dbcon.Query<Phrase>("UPDATE Phrase SET Answered = Answered + 1 " +
"WHERE Id = ?", id);
}
}
I am new to using SQLite with Xamarin but it looks strange to me that this update is handled with a dbcon.Query and that the table name is passed as . Can someone confirm is this the optimal way to handle a table update? Also why is it coded as a query with the table name being passed?
Update<T>
This method allows you to pass in an instance of an object that this stored in the database which has a primary key. SQLite then recognizes the primary key and updates the rest of the object's values.
You would just call connection.Update( phrase ); where the phrase is an instance of the Phrase class with properties you want to set. Be aware that all columns except ID will be updated.
Query<T>
Performs a query and returns the results. The type parameter specifies the type of the items returned. This is most appropriate for SELECT queries.
Execute
This returns the number of affected rows by the query as an int. This is probably the best choice for your UPDATE query after the Update<T> method.
ExecuteScalar<T>
Use for queries that return scalar types - like COUNT, etc., where T is the type of the value.
In summary, Update is the most natural way to update a row in the database (with an instance you have), but Query<T> and Execute<T> are very useful if you just want to UPDATE one column like in your example.
I'd like to use flyway for a DB update with the situation that an DB already exists with productive data in it. The problem I'm looking at now (and I did not find a nice solution yet), is the following:
There is an existing DB table with numeric IDs, e.g.
create table objects ( obj_id number, ...)
There is a sequence "obj_seq" to allocate new obj_ids
During my DB migration I need to introduce a few new objects, hence I need new
object IDs. However I do not know at development time, what ID
numbers these will be
There is a DB trigger which later references these IDs. To improve performance I'd like to avoid determine the actual IDs every time the trigger runs but rather put the IDs directly into the trigger
Example (very simplified) of what I have in mind:
insert into objects (obj_id, ...) values (obj_seq.nextval, ...)
select obj_seq.currval from dual
-> store this in variable "newID"
create trigger on some_other_table
when new.id = newID
...
Now, is it possible to dynamically determine/use such variables? I have seen the flyway placeholders but my understanding is that I cannot set them dynamically as in the example above.
I could use a Java-based migration script and do whatever string magic I like - so, that would be a way of doing it, but maybe there is a more elegant way using SQL?
Many thx!!
tge
If the table you are updating contains only reference data, get rid of the sequence and assign the IDs manually.
If it contains a mix of reference and user data, you need to select the id based on values in other columns.
I have map the entities in .hmb.xml and define attribute for all entity in classes.
I have some basic accomplishment and get all the record using below code.
public List<DevelopmentStep> getDevelopmentSteps()
{
List<DevelopmentStep> developmentStep;
developmentStep = Repository.FindAll<DevelopmentStep>(new OrderBy("Id", Order.Asc));
return developmentStep;
}
I have check from net that we can write HQL, Now the problem is how to execute this HQL like..
string hql = "From DevelopmentSteps d inner join table2 t2 d.id=t2.Id where d.id=IDValue";
What additional Classes or other thing I need to add to execute this kind of HQL?
Please help me ---- Thanks
To write dynamic queries, I recommend using the Criteria API. This is dynamic, because you have a single query for several different types and you also want to set the ordering dynamically.
The queries are always object oriented. You don't need to join by foreign keys, you just navigate through the class model. There also no "tables" in the queries, but entities.
Getting (single) instances by ID should always be done using session.Get (or session.Load). Only then NHibernate can also take it directly from the cache without database roundtrip, it it had already been loaded.
for instance:
public IList<T> GetAll<T>(string orderBy)
{
return session.CreateCriteria(typeof(T))
.AddOrder(Order.Asc(orderBy))
.List<T>();
}
I've got a flex/air app I've been working on, it uses a local sqlite database that is created on the initial application start.
I've added some features to the application and in the process I had to add a new field to one of the database tables. My questions is how to I go about getting the application to create one new field that is located in a table that already exists?
this is a the line that creates the table
stmt.text = "CREATE TABLE IF NOT EXISTS tbl_status ("+"status_id INTEGER PRIMARY KEY AUTOINCREMENT,"+" status_status TEXT)";
And now I'd like to add a status_default field.
thanks!
Thanks - MPelletier
I've add the code you provided and it does add the field, but now the next time I restart my app I get an error - 'status_default' already exists'.
So how can I go about adding some sort of a IF NOT EXISTS statement to the line you provided?
ALTER TABLE tbl_status ADD COLUMN status_default TEXT;
http://www.sqlite.org/lang_altertable.html
That being said, adding columns in SQLite is limited. You cannot add a column anywhere but after the last column in your table.
As for checking if the column already exists, PRAGMA table_info(tbl_status); will return a table listing the various columns of your table.
ADD ON:
I've been using a strategy in database design that allows me to distinguish which modifications are required. For this, you will need a new table (call it DBInfo), with one field (Integer, call it SchemaVersion). Alternately, there is also an internal value in SQLite called user_version, which can be set with a PRAGMA command. Your code can, on program startup, check for schema version number and apply changes accordingly, one version at a time.
Suppose a function named UpdateDBSchema(). This function will check for your database schema version, handle DBInfo not being there, and determine that the database is in version 0. The rest of this function could be just a large switch with different versions, nested in a loop (or other structure available to your platform of choice).
So for this first version, have an UpgradeDBVersion0To1() function, which will create this new table (DBInfo), add your status_default field, and set SchemaVersion to 1. In your code, add a constant that indicates the latest schema version, say LATEST_DB_VERSION, and set it to 1. In that way, your code and your database have a schema version, and you know you need to synch them if they are not equal.
When you need to make another change to your schema, set the LATEST_DB_VERSION constant to 2 and make a new UpgradeDBVersion1To2() function that will perform the required changes.
That way, your program can be ported easily, can connect to and upgrade an old database, etc.
I know this is an old question... however.
I've hit this precise problem in the SQLite implementation in Adobe AIR. I thought it would be possible to use the PRAGMA command to resolve, but since adobe air's implementation does not support the PRAGMA command, we need an alternative.
What I did, that I thought would be worth while sharing here, is this:
var sql:SQLStatement = new SQLStatement();
sql.sqlConnection = pp_db.dbConn;
sql.text = "SELECT NewField FROM TheTable";
sql.addEventListener(SQLEvent.RESULT, function(evt:SQLEvent):void {
});
sql.addEventListener(SQLErrorEvent.ERROR, function(err:SQLErrorEvent):void {
var sql:SQLStatement = new SQLStatement();
sql.sqlConnection = pp_db.dbConn;
sql.text = "ALTER TABLE TheTable ADD COLUMN NewField NUMERIC;";
sql.execute();
sql.addEventListener(SQLEvent.RESULT, function (evt:SQLEvent):void {
});
});
sql.execute();
Hope it helps someone.
I solved a similar problem using the answer from this question:
ALTER TABLE ADD COLUMN IF NOT EXISTS in SQLite
Use built in user_version parameter to keep track of your updates. You set it using:
PRAGMA user_version = 1
and you retrieve it using
PRAGMA user_version
So basically retrieve user_version (it's 0 by default), check if it's 0. If yes, perform your updates and set it to 1. If you have more updates in the future, check if it's 1, perform updates and set it to 0. And so on...
In some cases I execute the command and get the exception for "duplicate column". Just a quick solution, not the perfect.