Data Migration into Service Fabric Stateful Service - .net-core

Question

I have a stateful service that stores a bunch of data about my users that is stored in a reliable dictionary and obviously also retrieves it from there too.
However, I also have a SQL database that used to store this info. On initialization of a new stateful service instance, I need to migrate that info from my SQL database into the new reliable storage mechanism. From that point on, the stateful service is the source of truth. Ideally, I'd like to delay availability of my stateful service until this initialization process is completed.
Are there any suggestions on an approach for how to do this?

Answer 1

Something like does will do the trick:
public interface IStateful1 : IService
{
Task MyMethod();
}
internal sealed class Stateful1 : StatefulService, IStateful1
{
private bool isReady = false;
public Stateful1(StatefulServiceContext context)
: base(context)
{ }
public Task MyMethod()
{
if(!isReady)
throw new NotImplementedException(); // Probably throw or return something more meaningful :-)
return Task.CompletedTask; // Do your thing here
}
protected override IEnumerable<ServiceReplicaListener> CreateServiceReplicaListeners()
{
return new ServiceReplicaListener[0];
}
protected override async Task RunAsync(CancellationToken cancellationToken)
{
await Task.Run(() => {
// Simulation of some work
Thread.Sleep((int)TimeSpan.FromMinutes(5).TotalMilliseconds);
});
isReady = true;
}
}
In this setup the import from the DB into the reliable collection is done in the RunAsync method.
Unfortunately, AFAIK, there is not way to plug in the communication listeners at a later time. That would make things way easier.
If CreateServiceReplicaListeners would be an async operation we could await the initialization task here, but we can't right now. Using .Wait() is not going to work as it will report that the instance is taking to long to get running and will mark the instance as unhealthy.
A complete overview of the lifecycle of a service can be found in the docs

Answer 2

I am not sure if I got you right. But based on your comment I would suggest the following solution for returning the 'Not ready' response during the migration.
public interface IMigrationService
{
bool IsDone();
}
public class MigrationService : IMigrationService
{
private bool migrating = tu;
public bool BeginMigration()
{
this.migrating = true;
}
public bool EndMigration()
{
this.migrating = false;
}
public bool IsDone()
{
return this.migrating;
}
}
// WebHost startup class
public class Startup
{
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
// Register a middle-ware that would short circuit responses
// while migration is in progress.
app.Use(
async (context, next) =>
{
var migrationService =
context.RequestServices.GetService<IMigrationService>();
if (!migrationService.IsDone())
{
/* short circuit the response with approriate error message */
}
await next();
});
app.UseMvc();
}
}
public class Stateful : StatefulService
{
private readonly IMigrationService migrationService;
public Stateful(StatefulServiceContext context)
: base(context)
{
this.migrationService = new MigrationService();
}
protected override IEnumerable<ServiceReplicaListener>
CreateServiceReplicaListeners()
{
/*
Create a listener here with WebHostBuilder
Use Startup class with the middle-ware defined and
add configure services -> .ConfigureServices()
with services.AddSingleton<IMigrationService>(this.migrationService)
*/
}
protected override async Task
RunAsync(CancellationToken cancellationToken)
{
this.migrationService.StartMigration();
/* Migration code */
this.migrationService.EndMigration();
}
}
This would allow you to roll-out a new version of the service that would short circuit all requests with appropriate error message while the migration is in progress.
Hope this helps.