Agenda- Migrate DynamoDB tables to ScyllaDB (Schema as well as data)
Does the Scylla-Migrator - https://github.com/scylladb/scylla-migrator can migrate the table schema as well, or I have to create the exact schema in my ScyllaDB and then it can just migrate the data?
you need to create the keyspace and table in remote cluster,
migrator can map old table to new table layout if needed.
The reason why we don't migrate schema automagically is that most of times you want to have it different (e.g. different compactions strategy, or new columns)
and then having this as manual step makes sure you can review your schema before using it.
That said I think it makes sense to ask for a special flag that will just migrate old schema for you to new cluster - https://github.com/scylladb/scylla-migrator/issues - can you file it there?
Related
We've been using flyway for schema migrations, versions 4.x/5.x - the table is named schema_version. Now we are looking to move to latest version, 7.x. I've seen some information that states the new table name has changed to flyway_schema_history, but then I just saw this where the table name looks to be named flyway_history_schema (https://flywaydb.org/documentation/concepts/migrations#schema-history-table).
Can anyone who uses flyway confirm the schema history table name?
Thanks,
Scott
The default name for the table is indeed flyway_schema_history
The part of the documentation you linked to is referring to the situation when Flyway is not allowed to create new schemas by its configuration, and you need to manually create a schema for that table to live in. "flyway_history_schema" is a suggested name for the schema, not the table.
I have an Ionic App using SQLite. I don't have any problems with implementation.
The issue is that I need to import an SQL file using SQLitePorter to populate the database with configuration info.
But also, on the same database I have user info, so my question is:
Everytime I start the app, it will import the sql file, fill the database and probably overwrite my user data too? Since it is all on the same base?
I assume that you can always init your table using string queries inside your code. The problem is not that you are importing a .sql file. Right?
According to https://www.sqlitetutorial.net/sqlite-create-table/ it is obvious that you always create a table with [IF NOT EXISTS] switch. Writing a query like :
CREATE TABLE [IF NOT EXISTS] [schema_name].table_name (
column_1 data_type PRIMARY KEY);
you let sqlite to decide if it's going to create a table with the risk to overwrite an existing table. It is supposed that you can trust that sqlite is smart enough, not to overwrite any information especially if you use 'BEGIN TRANSACTION' - 'COMMIT' procedure.
I give my answer assuming that you have imported data and user data in distinct tables, so you can manipulate what you populate and what you don't. Is that right?
What I usually do, is to have a sql file like this:
DROP TABLE configutation_a;
DROP TABLE configutation_b;
CREATE TABLE configutation_a;
INSERT INTO configutation_a (...);
CREATE TABLE configutation_b;
INSERT INTO configutation_b (...);
CREATE TABLE IF NOT EXIST user_data (...);
This means that every time the app starts, I am updating with the configuration data I have at that time (that's is why we use http.get to get any configuration file from a remote repo in the future) and create user data only if user_data table is not there (hopefully initial start).
Conclusion: It's always a good practice, in my opinion, to trust a database product 100% and abstractly let it do any transaction that might give you some risk if you implemented your self in your code; since it gives a tool for that.For example, the keyword [if not exists], is always safer than implementing a table checker your self.
I hope that helps.
PS: In case you refer in create database procedure, SQLite, connects to a database file and it doesn't exist, it creates it. For someone comfortable in sqlite command line, when you type
sqlite3 /home/user/db/configuration.db will connect you with this db and if the file is not there, it will create it.
I'm using Flyway to manage db migration on IBM dashDB. This database organizes by default table content 'by column', which in particular makes the creation of the "schema_version" table fail.
To get it to work, the table creation SQL statement should only include the "ORGANIZE BY ROW" directive:
CREATE TABLE (...)
(...)
) ORGANIZE BY ROW
What would be the best approach to handle this issue ? I'm looking for a solution that does not impact the default table organization.
Thanks for helping,
Cheers.
dashDB will perform best when all tables are column-based. When you start to mix row and column based tables, many operations are then performed in "compensation" which basically means they won't take full advantage of the columnar engine.
There are currently some compatibility reasons why a columnar table cannot be created and thus a row based table must be used, but the original DDL nor error are stated so I can't tell in this case. If you can provide the full CREATE TABLE statement and the resulting error (if you have it), I can possibly provide an alternative solution that would allow you to still use all column-based tables.
If you only want to change a particular table from column organized to row organized then a "ORGANIZE BY ROW" on the table definition would be the recommended way to approach this. (This seems to be what you're doing)
Changing the default table org will change how tables are created when you don't put an "ORGANIZE BY " in your table ddl.
If you have admin privileges on your dashDB instance you can change the default table org via 'Run SQL' in the dashDB console or using a dashDB client. (for exampl: clp/clpplus)
Set default table organization to ROW:
call ADMIN_CMD('UPDATE DB CFG USING DFT_TABLE_ORG ROW');
Set default table organization to COLUMN: (default dashDB configuration)
call ADMIN_CMD('UPDATE DB CFG USING DFT_TABLE_ORG COLUMN');
Analytics will perform much better with Column organized tables so it's recommended to have the majority of your tables as column organized.
Anybody know if it's possible to;
Dynamically create LINQ classes for tables/columns that change regularly?
If that creation can be used in DynamicData.
A web app we are developing creates tables and columns in SQL. We want to edit these tables in DynamicData.
Thoughts?
Depending on what type of Database you are running, but you could always have a linq statement that queries the systems schema table and have it return the tables and columns. Then could use what you return and then use another linq query to break out the information from each table.
I used sqlmetal.exe from the SDK, it's a winner.
Can someone help me answer these questions on EntityFramework?
Does it do anything special to the database? (like extra tables)
Can I add data directly with SQL without breaking EF?
Can I add tables and fields without breaking EF?
Yes you can access database with plain SQL when using EF.
No. EF just uses database. There is one exception in code first approach where EF can create one additional table for its own purpose called EdmMetadata.
Yes you can add data directly with SQL. If both your entity model and database are defined correctly it will not break EF.
Yes you can add new tables directly but EF will not know about them. You should not change existing tables because it can break EF.
You can do it with:
var context = new YourObjectContext();
var s = context.ExecuteStoreCommand("some query");
if your query create a table, this only create a table on db and not effected on EF