I'm just in the process of configuring and fully understanding flyway and I came into this situation:
I successfully configured a new project to work with flyway.
I successfully migrated a test database from version 0 to 1.0.3.
Migration to version 1.0.4 failed to execute. (I was trying to add column that was already there, no problems so far, my bad.)
However, once that I made the necessary changes to the corresponding script to work, flyway kept showing this message:
Current schema version: 1.0.4
com.googlecode.flyway.core.migration.MigrationException: Migration to version 1.0.4 failed! Please restore backups and roll back database and code!
Since I didn't want to restore a complete dump and apply every migration again, just to make an alter table script to work, what I finally did were some changes to the 'schema_version' table:
1st I erased the entry for version 1.0.4
2nd I set the 'current_version' field to 1 for version 1.0.3
And then executed the flyway:migrate command again
After this, the migration finally was applied and a success message shown, however I´m not quite sure if this is the right approach to deal kind of this situations. I'm no sure if its right to modify the 'schema_version' table by myself since i think it should only be modified by flyway itself.
So, after explaining what happened to me, my question would be:
Is there a way to 'retry' to apply a failed migration in flyway, without modifying the 'schema_version' table by myself?
Any command I'm not aware of to fulfill this task?
This is answered in the FAQ: http://flywaydb.org/documentation/faq.html#repair
The upcoming Flyway 2.0 will include the repair command. This code is already checked into SCM.
Note: This only deals with Flyway's metadata table. You are still responsible for cleaning up any other effects of a failed migration.
Update: Flyway 2.0 has now been released. You can grab it at http://flywaydb.org
I don't know whether this a good idea or not, but you could try doing a repair() if migrate() fails:
final Flyway flyway = new Flyway();
flyway.setBaselineOnMigrate(true);
flyway.setValidateOnMigrate(false);
flyway.setDataSource(dataSource());
try {
flyway.migrate();
} catch (final Exception e) {
logger.error("Flyway migration failed, doing a repair and retrying ...");
flyway.repair();
flyway.migrate();
}
Full example, this will always try to repair before running a migration, the rest of configuration in in a config file.
#Configuration
public class PersistanceConfiguration {
protected final Logger log = LoggerFactory.getLogger(this.getClass());
#Bean
public FlywayMigrationStrategy cleanMigrateStrategy() {
FlywayMigrationStrategy strategy = new FlywayMigrationStrategy() {
#Override
public void migrate(Flyway flyway) {
flyway.repair();
flyway.migrate();
}
};
return strategy;
}
}
Related
We have a Xamarin.Forms project that needed to use the sqlite local db to store date. EF Core's sqlite library was used to set this up and by different developers from different PCs (vs 2019). Initially, it was used with the Database.EnsureCreated() function and later with ef core's migrations. All went smooth for more than a month.
Last week all of a sudden the android app wouldn't start on any one's PC due to some error with sqlite. It showed the following error:
Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
I spent a while trying all kinds of fixes and rollbacks thinking it was an issue with the code. This included the following:
Deleted obj and bin folders, cleaned and rebuilt for all below steps.
Downgraded the version of ef to 2.2.3 (the version we started with)
Rolled back to git commits up to a week back
Upgraded the versions of dependencies of ef core
Removed the past few migrations
Downgraded xamarin forms to 3.6.x
After trying the above and several other fixes, finally upgrading the versions of java and android SDK worked last Friday (on all PCs). Once this fix worked everything went smooth that day. On Tuesday once again the issue was back (no library updates or code changes). A deeper look at EF Cores logging shows that it crashes the moment it attempts to connect to a db.
The issue can be replicated on the android devices but not the emulators. I am not sure if there is some new permission in android we need to request for.
I finally created a new xamarin forms project with sqlite setup. I used the pcl library and ef core. I still found the same error.
Here is the git hub that replicates the issue https://github.com/neville-nazerane/xamarin-site
Update
Just something i noticed. eariler my database file was named "main.db". Now no matter what i change this file name to or no matter what variables i change. it always shows database name as "main" in logs. Not sure if changing the db name would fix the issue. However, never found a way to change this db name. I tried different connection strings, it just said "database" and "db" were unknown keys
Update
Steps to replicate:
using (var db = new AppDbContext())
{
db.Add(new Person {
Age = 55,
Name = "Neville"
});
db.SaveChanges();
Person[] alldata = db.People.ToArray();
}
The definitions of Person and AppDbContext are quite obvious. So, with the spirit of not making the question too long, I am not posting it here. However, if required I can post them too.
This is a bug with the Xamarin.Forms and Mono.
It was detected since a couple of months ago, it was fixed but then there was some regression (with VS 2019 v16.1).
Even with the latest release (v16.1.2) the bug still happens, so we need to wait for a fix.
Sources:
https://github.com/mono/mono/issues/14170
https://github.com/xamarin/xamarin-android/issues/3112
https://github.com/xamarin/xamarin-android/issues/2920
Due to slight differences of the particular file systems on the native side, I would suggest creating an interface to handle the actual database file handling on the native application level.
So here is how I implemented SQLite using the nuget package SQLite.Net-PCL:
In the shared project, create a new interface, for instance FileHandler.cs
public interface IFileHandler
{
SQLite.SQLiteConnection GetDbConnection();
}
You may want to extend that with more methods to save or retrieve various files, but for now we will just have the GetDbConnection Method to retrieve a working SQLite Connection.
Now in your Android implementation, we add the native implementation to that interface:
Create a new class called FileHandler.cs in your android project:
[assembly: Dependency(typeof(FileHandler))]
namespace YourProjectName.Droid
{
public class FileHandler : IFileHandler
{
public SQLite.SQLiteConnection GetDbConnection()
{
string sqliteFilename = "YourDbName.db3";
string path = Path.Combine(GetPersonalPath(), sqliteFilename);
SQLiteConnectionString connStr = new SQLiteConnectionString(path, true);
SQLiteConnectionWithLock connection = new SQLiteConnectionWithLock(connStr, SQLiteOpenFlags.Create | SQLiteOpenFlags.ReadWrite | SQLiteOpenFlags.NoMutex);
return connection;
}
private string GetPersonalPath()
{
return Environment.GetFolderPath(Environment.SpecialFolder.Personal);
}
}
}
Now back again in your shared code you can access that connection with the following code:
using (SQLiteConnection connection = DependencyService.Get<IFileHandler>().GetDbConnection())
{
// Do whatever you want to do with the database connection
}
Alright mate, I don't understand what issue you are facing. It might be an issue with your machine, I'd suggest using another computer/laptop.
I took the exact code that you shared on the Github. I was able to build it on my Mac computer in VS 2019 and installed the application in debug mode on my phone. I was able to add a date successfully, as you can see in the picture, and I placed an Exception Catchpoint and faced no exceptions.
I then proceeded to add another entry with the same details and it errored out with the message that you can see here
I would also suggest using Xamarin Profiler or any other Android logger to see the Stack Trace that you aren't able to see in your application output. It will give you details of the error, that you can share here for us to understand better.
Before I explain my issue, I have some experience with entity framework 5 and 6 code first migrations, running add-migration/update-database and a few more specific commands from the Package Manager console. All of the migration history was handled out of the box in the __MigrationHistory table.
I am now writing a UWP app and using EntityFrameworkCore sqlite. The app is set up to scaffold new migrations and does so correctly.
When applying migrations the app needs to automatically deduce, on install and first startup, if the database exists, and the current database migration version. It can then apply the relevant migration procedures, including creating the database if required.
Currently, I attempt to perform the migrations in my DbContext on startup:
public class MyContext : DbContext
{
public DbSet<SomeEntity> MyEntities { get; set; }
static MyContext()
{
using(var db = new MyContext())
{
db.Database.Migrate();
}
}
This works perfectly for a new app on first startup. On second startup however, or after the addition of a new migration, the Migrate() method fails as the tables it is attempting to create already exist.
SQLite Error 1: 'table \"MyEntities\" already exists'
This error comes from rerunning the migration that has been previously applied. The database itself needs to be aware of it's migration history as was previously handled with __EFMigrationHistory. Currently this table is not being created for me.
I am suspecting that I need to manually build a solution to this, maybe creating my own __MigrationHistory table and keeping it up to date, as per this post here
I wondered what solutions people have used for this issue, or if there is anything out of the box that I'm being silly and missing.
Let me know if more detail needed.
I solved this in a manner of speaking but I'm still not sure why the __MigrationHistory table was not automatically generated...
I couldn't find any evidence of other people struggling with this issue in UWP apps, so it is likely project specific and something caused by how the solution was set up.
Anyway, the changes I made:
I created a MigrationHistory model and added it as a DbSet to my DbContext.
Model
namespace MyApp.Shared.Models.Infrastructure
{
public class MigrationHistory
{
public string MigrationId { get; set; }
public string ProductVersion { get; set; }
}
}
DbContext additions
public class MyContext : DbContext
{
public DbSet<MigrationHistory> __MigrationHistory { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
#region Primary Keys
modelBuilder.Entity<MigrationHistory>().HasKey(mh => mh.MigrationId);
At this stage, on running add-migration entity framework attempted to create the __MigrationHistory table. This would result in an error if I ran my application, as applying the migration would result in the error:
SQLite Error 1: 'table \"__MigrationHistory\" already exists'
So I added the following code to my MyContextModelSnapshot class
modelBuilder.Entity("MyApp.Shared.Models.Infrastructure.MigrationHistory", b =>
{
b.Property<string>("MigrationId")
.ValueGeneratedOnAdd();
b.Property<string>("ProductVersion");
b.HasKey("MigrationId");
b.ToTable("__MigrationHistory");
});
Once it is in the snapshot, it can stay there, and it acts to prevent entity framework attempting to add this table in future migrations.
On startup my app now runs
using (var db = new Assessment.Data.WindowsUniversal.AssessmentContext())
{
db.MigrateDatabase();
}
And it works perfectly, consulting the table and applying migrations where necessary.
I feel like this is a solution to a problem that doesn't really exist, and that is of my own making, but I'll leave this here in case it's relevant to somebody else.
As far as i've come up with while having your same issue, i found out the debug database (inside your \bin\Debug folder) won't have the __EFMigrationsHistory table, while the production database (root of your launching project) has it.
Maybe could be of help for somebody else.
I encountered the same issue with Sqlite In-Memory databases in my Tests. I have found the following thread https://github.com/dotnet/efcore/issues/4922. The point is ones all connections to the database are closed, the database is being removed from the memory. I have not solved the issue yet as I started using physical databases instead. But I will update my answer ones I find the solution.
My problem is even if I call db.Database.Migrate() first time without any other connections opened before, it still throws such error.
So, I'm trying out EF Code First, so that I can have the code drive updates to the database. I'm working on a ClickOnce app using LocalDB, so figured this may be the best solution for me, since otherwise changes to the MDF file will cause it to be overwritten on the client when deployed, thus losing everything entered before.
However, I'm now having my fair share of all new problems around Code First Migrations. I've followed through a Code First Migrations on MSDN, and I've managed to get the initial Configuration created, as well as the initial database creation.
The problems begin when I try to make my first actual migration. I added one single field to one of my models, and tried to make an explicit migration to handle that schema change for the next time I publish. Well...
PM> Add-Migration AddIsPercentField
Unable to generate an explicit migration because the following explicit migrations are pending: [201601052011180_InitialCreate]. Apply the pending explicit migrations before attempting to generate a new explicit migration.
Ok... I'll run update and try again:
PM> Update-Database
Specify the '-Verbose' flag to view the SQL statements being applied to the target database.
Applying explicit migrations: [201601052011180_InitialCreate].
Applying explicit migration: 201601052011180_InitialCreate.
Unable to update database to match the current model because there are pending changes and automatic migration is disabled. Either write the pending model changes to a code-based migration or enable automatic migration. Set DbMigrationsConfiguration.AutomaticMigrationsEnabled to true to enable automatic migration.
You can use the Add-Migration command to write the pending model changes to a code-based migration.
PM> Add-Migration AddIsPercentField
Unable to generate an explicit migration because the following explicit migrations are pending: [201601052011180_InitialCreate]. Apply the pending explicit migrations before attempting to generate a new explicit migration.
That's familiar, as that's the error (blatant lie?) it just told me earlier. Well, maybe if I undo my changes and update again, it will move to a valid state:
PM> Update-Database
Specify the '-Verbose' flag to view the SQL statements being applied to the target database.
Applying explicit migrations: [201601052011180_InitialCreate].
Applying explicit migration: 201601052011180_InitialCreate.
Running Seed method.
Ok, no warning this time. Should be golden. Field added back, project rebuilt. Here we go:
PM> Add-Migration AddIsPercentField
Unable to generate an explicit migration because the following explicit migrations are pending: [201601052011180_InitialCreate]. Apply the pending explicit migrations before attempting to generate a new explicit migration.
So... is there actually a working way to generate explicit migrations for any changes beyond the first?
EDIT: I made some forward progress on this, I believe. I did notice that the __MigrationHistory table was not generated in my .mdf after running Update-Database, even though it said everything completed just fine. I believe the issue is actually around how the local database works within the application. The connection string references AttachDbFilename=|DataDirectory|. What I think is going on is that it is deploying the .mdf temporarily, updating that temporary deployment, thus ultimately not committing the changes.
I'm working on a solution I have in mind, which is to have migrations work against a copy of the blank .mdf put in a static location, so that the static .mdf will be used to track and determine changes, while the blank .mdf will be what goes out to clients with the deployment.
I found that the root of my problem was that the console commands were not actually able to make changes, thus track migrations, to my data file. This was due to the connection string to the data file referencing a deployed location, so the file being updated was merely temporary.
This was in part a good thing, because the whole point of using Code First Migrations in my project was to avoid a hash signature change to my .mdf (which should have simply remained blank, as a placeholder) when publishing, so that the data from previous versions would never be overridden and discarded. However, that also introduced the obvious (in retrospect) problem that EF could not track changes due to there never being a __MigrationHistory table.
The solution upon which I arrived was to have two .mdf files. The blank one, for deployments, and a second one, to which I would interact with Code First Migrations. So, I have the initial MyData.mdf of Build Action Content, and a second MyDataDesignTime.mdf of Build Action None. (The "Design Time" migration database shouldn't be deployed.)
Using this approach, I found that I could now work successfully with migrations, calling Update-Database and Add-Migration, making sure to pass the -ConnectionString parameter with AttachDbFilename pointed to the full path to my design time database.
Later, getting lazy to supply a long -ConnectionString parameter on every migration command, I added the design time path to my config connection strings, and updated my DbContext so that it uses the design time path initially, but which I would change at the beginning of run-time to use my actual target data file:
public partial class MyData : DbContext
{
public const string DesignTimeConnection = "MyDataConnectionStringDesignTime";
public static string ConnectionName { get; set; } = DesignTimeConnection;
public MyData()
: base("name=" + ConnectionName)
{
}
...
}
And at application initialization:
MyData.ConnectionName = "MyDataConnectionString";
This works, and it makes things simpler on me. However, the one minor issue I'm left with is that I have a full static path which applies only to my environment left in the app.config file. Not currently an issue, as I'm the only dev on this project, but it's a code smell that I'm not happy with. Is there some path variable that I can use, such that it still points to the actual design time data (not any temporary, deployed file), but does so relative to the active, open project?
Found non-empty schema "public" without metadata table! Use init() or set initOnMigrate to true to initialize the metadata table.
I'm using Postgres 9.2 with Postgis 2.0. This means that by default when I create a new database there will be a table created in public schema called spatial_ref_sys.
When I run flyway migrate on this database, I get the above error. Running init seems to create the public.schema_version table and mark version 1 as SUCCEDED without actually running the the migration file. I've also tried combinations of initOnMigrate with no success. Flyway is not configured to manage any schemas.
Any ideas on how I can run a migration in this scenario?
The title is somewhat contradictory, as the database is indeed not virgin as you installed, through the PostGIS extension, a number of objects in the public schema.
You can either
set flyway.schemas to a new schema, say my_app, which will then be created automatically by Flyway. Your application should then use this one instead of public (recommended)
set flyway.baselineOnMigrate to true or invoke flyway.baseline() against the public schema. This will work, but public will then contain a mix of both your application objects and the PostGIS objects
If you are using Gradle you can run
./gradlew -Dflyway.schemas=public flywayClean flywayMigrate
Where public is the name of the database containing the schema_versions table. That should delete the table and metadata as well as running the migrations to get it back up to date.
Caution!
This will delete all data in public schema
I think this error comes only with latest version of Flyway i.e. above 4.03. I didn't received in the earlier project but got it when I am using Flyway version 5.07 in my latest project. Putting the code here that resolve my issues
public class FlywayConfig {
#Autowired
DataSource dataSource;
#Autowired
Config config;
#Bean
public Flyway flyway(){
Flyway flyway = new Flyway();
flyway.setDataSource(dataSource);
flyway.setSqlMigrationPrefix("V");
flyway.setLocations(new String[] { config.getSqlLocation() });
flyway.setBaselineOnMigrate(true);
// *******************flyway.clean(); ********************// this will wipe out the DB, be careful
flyway.migrate();
return flyway;
}
}
this work for me , i were figthin with the same problema a lot of time
my project was building on maven
Flyway flyway = new Flyway();
flyway.setDataSource(dataSource);
flyway.setLocations("db/your_db");
flyway.setTable("name_of_schema");
next a added this line
flyway.setBaselineOnMigrate(true);
flyway.clean();
next this lines
MigrationInfo migrationInfo = flyway.info().current();
flyway.migrate();
and i let you the URL of my references from flyway.org
Flyway.org/documentation/commandline/baseline
In my case the problem started when I deleted all the rows in the table myschema.schema_version
./gradlew flywayInit did the trick and the error is not showed anymore.
I am following the offical asp.net "Getting started with EF 5 using MVC 4". In that tutorial, the database is created when the migrations are performed(in my understanding). When I was looking at the EF 5 with Mvc 5 tutorial they didn't use migrations to create a database. They use database initializer. So, I was wondering could create a database for your project without using migrations in EF 5? Also, what would the difference be with both these approaches?
Code first Migrations and using Package Manager Console Commands to do upgrades can get a bit confusing at first.
You can use the initializer to CreateDatabaseIfNotExists , DropCreateIfModelChanges, DropCreateDatabaseAlways and to MigrateDatabaseToLatestVersion
See the interface IDatabaseInitializer<TContext>.
CreateDatabaseIfNotExists // is the Default initializer.
So this is why it appears EF just does things for you sometimes.
So the answer is "YES you can "Create a Database without Migrations"
But the difference is not obvious and if you would do that long term is another question.
If you are using migrations. It would Update the Db to match the code first model.
If there is NO database, then that means creating the database.
So Thats why Automated migrations and CreateDB look confusing since they can result in same outcome sometimes. But technically they are different.
So generally it is sufficient to use code first automatic "migrations" only.
Migrations can be either Automatic or "managed".
The managed migrations approach invovles generating code , tweaking the code and running PM commandlet or POwershell command to actually perform the migration.
With Automated migrations you just need set the intitializer and Access the DBContext.
There are 2 parts to the process.
a) The DB Initializer step.
do this immediately before instantiating YourDBContext.
//eg
// DONT TOUCH MY DB or i break your back!
Database.SetInitializer(new ContextInitializerNone<YourDbContext>()); // Do Nothing,
// OR
// yes migrate my db to match my code please.
Database.SetInitializer(new MigrateDatabaseToLatestVersion<YourDbContext, YourMigrationConfiguration>()); // Set to migration is requested, see config class below
The Confirguration class specified when using Migration initializer looks like this
public class YourMigrationConfiguration<TContext> : DbMigrationsConfiguration<TContext>
where TContext : DbContext{
protected YourMigrationConfiguration() {
AutomaticMigrationsEnabled = true; // run it when needed. Do not wait for my PM Command
AutomaticMigrationDataLossAllowed = true; // if the new db look means dropping tables or columns go ahead and kill my data. So use this option with caution.
}
then just trigger the migration in code when required.
Context.Database.Initialize(true); // i place this inside a method on my UoW class
Code first Db initialization strategies.
Code first migrations recommended reading
Managed Migrations
There are many articles on the web on this topic.