DbUpdateConcurrencyException with EntityFramework on RemoveRange - .net-core

Question

We are experiencing something very curious with EntityFramework that we are having a hard time debugging and understanding.
We have a service, which debounces index updates, such that we'll update the index when all changes are made and not for every change. The code looks something like this:
var messages = await _debouncingDbContext.DebounceMessages
.Where(message => message.ElapsedTime <= now)
.Take(_internalDebouncingOptionsMonitor.CurrentValue.ReturnLimit)
.ToListAsync(stoppingToken)
.ConfigureAwait(false);
if (!messages.Any())
return;
var tasks = messages.Select(m =>
_messageSession.Send(m.ReplyTo, new HandleDebouncedMessage {Message = m.OriginalMessage}))
.ToList();
try
{
await Task.WhenAll(tasks).ConfigureAwait(false);
}
catch (Exception e)
{
//Exception handling
}
_debouncingDbContext.DebounceMessages.RemoveRange(messages);
await _debouncingDbContext.SaveChangesAsync().ConfigureAwait(false);
While this is being run, we have another thread that can update the ElapsedTime on the entries. This happens if a new event comes in, before the debounce timer expires.
What we experience is the that the await _debouncingDbContext.SaveChangesAsync().ConfigureAwait(false); throws a DbUpdateConcurrencyException.
The result is that the entries are not being deleted and consequently being queried out over and over again in the initial query. This leads to an exponential growth in our index updates where the same few items are being updated over and over again. Eventually, the system dies.
The only fix we have for this right now, is restarting the service. Once that is done, the next iteration picks up the troublesome messages just fine and everything works again.
I am having a hard time understanding how this can happen. It appears that the dbcontext thinks that the entries are deleted, when in fact that are not. Somehow the DBContext gets decoupled from the database state.
I cannot understand how this can happen, when the only thing potentially being changed on the database entry it self is a timestamp and not the actual ID by which it is deleted.
EDIT 18th of November.
Adding a little more context.
The database model, looks like this:
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
public string Key { get; set; }
public string OriginalMessage { get; set; }
public string ReplyTo { get; set; }
public DateTimeOffset ElapsedTime { get; set; }
The only thing configured on the dbcontext is two indexes:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Entity<DebounceMessageWrapper>()
.HasIndex(m => m.Key);
modelBuilder.Entity<DebounceMessageWrapper>()
.HasIndex(m => m.ElapsedTime);
}
The flow is quite simple.
We have a dotnet hosted service extending the abstract BackgroundService class from dotnet core. This runs in a while(!stoppingToken.IsCancellationRequested) loop with the initial code above and a Task.Delay(Y) in each loop. All the above code does, is it queries all the messages with an ElapsedTime greater than the allowed timespan. For each of those messages, it returns it to the ReplyTo and then deletes all the corresponding database entries. It is this delete that fails.
Secondly, we have a MessageHandler listening for events on a RabbitMQ. This spawns a thread per physical core on the host. Each of these threads receives messages and looks up the messages based of the key on the database model. If the message already exist, the ElapsedTime is updated, if not, the message is inserted in the database.
This gives us X+1 threads, where X equals the number of physical cores on the host, that potentially can alter the database. Each of these threads uses it's own scope and thereby unique instance of the DBContext.
The idea of this service, is as mentioned to debounce index updates. The nature of our system makes these index updates come in bulks and there is no reason to update the index for each of the updates, if it can be done by one index update when all the changes are finished.