I need to rewrite an old web service because the team that supported the old one no longer wants to support it with their new tool. So I'm rewriting it using ASP.NET Core WebAPI and EF Core 3.1.
The majority of the logic for the service is stuck in stored procedures written years ago. Nothing super complex, but don't really think it's a good idea to start rewriting everything.
The problem is that EF Core's support for stored procs seems lacking at best. I got one working using Database.ExecuteSqlRaw which returns the results as output parameters, but I'm having trouble with another that returns the results as a dataset. (Actually two, but let's not get ahead of ourselves... I've commented it so it's only returning one right now.)
The (current) problem with the .FromSqlRaw query is that it doesn't appear to be querying the database at all when I watch for it in XEvent Profiler. (SQL Server 2016.)
Here's the code I'm using to call the proc:
var bundle_id = new SqlParameter("bundle_id", bundleID) { Direction = ParameterDirection.Input };
var result = this.BundleUserGuideDetails.FromSqlRaw("EXEC dbo.p_fetch_user_guide_details #bundle_id", bundle_id);
var deets = result.FirstOrDefault<BundleUserGuideDetail>();
I did create a DBSet for it in the DBContext class:
public DbSet<BundleUserGuideDetail> BundleUserGuideDetails { get; set; }
And since it's a keyless type I've got this per Microsoft's Keyless Entity guide:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<BundleUserGuideDetail>(eb =>
{
eb.HasNoKey();
});
}
The model I created has all the same field names as the data being returned as well.
So why is the DB not even getting called for this?
EDIT: Forgot to write that the call to FirstOrDefault is throwing "System.InvalidOperationException: 'Sequence contains no elements'"
EDIT: Here is the full exception text:
System.InvalidOperationException: Sequence contains no elements
at System.Linq.ThrowHelper.ThrowNoElementsException()
at System.Linq.Enumerable.Aggregate[TSource](IEnumerable`1 source, Func`3 func)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.EntityMaterializerInjectingExpressionVisitor.ProcessEntityShaper(EntityShaperExpression entityShaperExpression)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.EntityMaterializerInjectingExpressionVisitor.VisitExtension(Expression extensionExpression)
at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
at System.Linq.Expressions.ExpressionVisitor.VisitBinary(BinaryExpression node)
at System.Linq.Expressions.BinaryExpression.Accept(ExpressionVisitor visitor)
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block)
at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node)
at System.Linq.Expressions.BlockExpression.Accept(ExpressionVisitor visitor)
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
at System.Linq.Expressions.ExpressionVisitor.VisitLambda[T](Expression`1 node)
at System.Linq.Expressions.Expression`1.Accept(ExpressionVisitor visitor)
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.EntityMaterializerInjectingExpressionVisitor.Inject(Expression expression)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.InjectEntityMaterializers(Expression expression)
at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.VisitShapedQueryExpression(ShapedQueryExpression shapedQueryExpression)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression)
at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
at System.Linq.Queryable.FirstOrDefault[TSource](IQueryable`1 source)
at MandTBank.BBFB.BPS.BPSWebAPI.Data.BundleSystemContext.FetchUserGuideDetails(String bundleID) in C:\Users\tdevmcr\Source\Workspaces\BBFB\NeedsAssessmentSystem\Main\Sourcecode\BBFB NAS - I3 - BundleUserGuide Service\MandTBank.BBFB.BPS.BPSWebAPI\Data\BundleSystemContext.cs:line 65
Turns out I had made a simple mistake, and failed to make the fields in the model public
The exception being thrown certainly doesn't make that obvious though...
I also needed to add AsEnumerable() after the FromSqlRaw call for it to work, as per this question: Include with FromSqlRaw and stored procedure in EF Core 3.1
Related
We are querying database using LINQ-SQL and then storing resulting master table objects in HTTP cache.
Later, the master objects are being used to query its children, using lazy loading. Here are the relevant pieces of code - I have recreated the scenario in a new proof-of-concept app:
if (HttpRuntime.Cache["c"] == null)
{
LockApp.Models.DBDataContext db = new Models.DBDataContext();
var master = db.Masters.ToList();
HttpRuntime.Cache.Add("c", master,
null, DateTime.Now.AddMonths(1),
TimeSpan.Zero, CacheItemPriority.Normal, null);
}
ViewBag.Data = (List<LockApp.Models.Master>)HttpRuntime.Cache["c"];
And here's the razor view that is iterating over master and detail objects:
#foreach(var m in ViewBag.Data){
#m.Id<nbsp></nbsp>
foreach(var d in m.Details){
#d.Id<nbsp></nbsp>
}
<br />
}
It works perfectly fine and caches the data correctly. However, it fails when there are multiple requests trying to hit the web site after cache is cleared - I am testing this using JMeter, basically hitting the site with many (50) parallel threads, and then touching web.config - I immediately start seeing one of the following two errors:
Index was outside the bounds of the array
at foreach(var d in m.Details)
this error never goes away, i.e. some data gets corrupted in cache
with following stack:
System.Collections.Generic.List`1.Add(T item) +34
System.Data.Linq.SqlClient.SqlConnectionManager.UseConnection(IConnectionUser user) +305
System.Data.Linq.SqlClient.SqlProvider.Execute(Expression query, QueryInfo queryInfo, IObjectReaderFactory factory, Object[] parentArgs, Object[] userArgs, ICompiledSubQuery[] subQueries, Object lastResult) +59
System.Data.Linq.SqlClient.SqlProvider.ExecuteAll(Expression query, QueryInfo[] queryInfos, IObjectReaderFactory factory, Object[] userArguments, ICompiledSubQuery[] subQueries) +118
System.Data.Linq.SqlClient.CompiledQuery.Execute(IProvider provider, Object[] arguments) +99
System.Data.Linq.DeferredSourceFactory`1.ExecuteKeyQuery(Object[] keyValues) +402
System.Data.Linq.DeferredSourceFactory`1.Execute(Object instance) +888
System.Data.Linq.DeferredSource.GetEnumerator() +51
System.Data.Linq.EntitySet`1.Load() +107
System.Data.Linq.EntitySet`1.GetEnumerator() +13
System.Data.Linq.EntitySet`1.System.Collections.IEnumerable.GetEnumerator() +4
ASP._Page_Views_Home_Index_cshtml.Execute() in c:\Users\prc0092\Documents\Visual Studio 2012\Projects\LockApp\LockApp\Views\Home\Index.cshtml:16
or this error
ExecuteReader requires an open and available Connection. The connection's current state is open.
at the same line foreach(var d in m.Details)
this error does go away after a while if I stop hitting the site with parallel requests
with following stack
System.Data.SqlClient.SqlConnection.GetOpenConnection(String method) +5316460
System.Data.SqlClient.SqlConnection.ValidateConnectionForExecute(String method, SqlCommand command) +7
System.Data.SqlClient.SqlCommand.ValidateCommand(String method, Boolean async) +155
System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite) +82
System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method) +53
System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method) +134
System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior) +41
System.Data.Common.DbCommand.ExecuteReader() +12
System.Data.Linq.SqlClient.SqlProvider.Execute(Expression query, QueryInfo queryInfo, IObjectReaderFactory factory, Object[] parentArgs, Object[] userArgs, ICompiledSubQuery[] subQueries, Object lastResult) +1306
System.Data.Linq.SqlClient.SqlProvider.ExecuteAll(Expression query, QueryInfo[] queryInfos, IObjectReaderFactory factory, Object[] userArguments, ICompiledSubQuery[] subQueries) +118
System.Data.Linq.SqlClient.CompiledQuery.Execute(IProvider provider, Object[] arguments) +99
System.Data.Linq.DeferredSourceFactory`1.ExecuteKeyQuery(Object[] keyValues) +402
System.Data.Linq.DeferredSourceFactory`1.Execute(Object instance) +888
System.Data.Linq.DeferredSource.GetEnumerator() +51
System.Data.Linq.EntitySet`1.Load() +107
System.Data.Linq.EntitySet`1.GetEnumerator() +13
System.Data.Linq.EntitySet`1.System.Collections.IEnumerable.GetEnumerator() +4
ASP._Page_Views_Home_Index_cshtml.Execute() in c:\Users\prc0092\Documents\Visual Studio 2012\Projects\LockApp\LockApp\Views\Home\Index.cshtml:16
Different things I tried
Double locking
Doesn't help
private static object ThisLock = new object();
public ActionResult Index()
{
if (HttpRuntime.Cache["c"] == null)
{
lock (ThisLock)
{
if (HttpRuntime.Cache["c"] == null)
{
Loading child data upfront
Works, but requires constant maintenance as not all children should be loaded upfront, plus see next note
DataLoadOptions dlo = new DataLoadOptions();
dlo.LoadWith<Master>(b => b.Details);
db.LoadOptions = dlo;
Locking the master object while trying to access its children
Again, requires maintenance as all initial places where child is accessed need to be found - we are struggling with this as there are different entry paths into the site
#foreach(var m in ViewBag.Data){
#m.Id<nbsp></nbsp>
lock (m){
foreach(var d in m.Details){
#d.Id<nbsp></nbsp>
}
}
<br />
}
Switching to entity framework
This seems to still have (sometimes - much better than linq-sql) the "open connection" issue at certain number of parallel requests (50+ on core i7) - it does go away after a while as I mentioned and I haven't seen data corruption yet.
We may end up switching to EF completely as this seems to be the only viable path - assuming data corruption doesn't show up - that is to be tested on my actual project.
I am not optimistic though, as EF data context is not thread safe either, and I think the EF data objects carry their context with them. This is probably the only question that I don't have answer to yet.
Theories on why it's broken
It looks like storing LINQ-SQL object in http cache carries the data context with it. When this context is later used by multiple threads to access children, there is some type of concurrency issue that manifests itself in either temporary connectivity issue or complete data corruption of the child object. As there's no way to disconnect/reconnect the context from the LINQ object, it looks like the only suggestion is not to cache LINQ objects that need lazy-loading of their children - a substantial number of google searches I did does not seem to give you that advice, in fact sometimes it's opposite.
I have uploaded the complete project (for Visual Studio 2012 and SQL Server 2012)
https://docs.google.com/file/d/0B8CQRA9dD8POb3U5RGtCV3BMeU0/edit?usp=sharing
and a simple JMeter script that will hit your local machine with parallel requests:
https://docs.google.com/file/d/0B8CQRA9dD8POd1VYdGRDMEFQbEU/edit?usp=sharing
to test, start the site and run the test - then touch the web.config on the site
LockApp.Models.DBDataContext db = new Models.DBDataContext();
var master = db.Masters.ToList();
You should have a call to db.ObjectTrackingEnabled = false in between these two calls. Otherwise all of the objects will be tracked by the datacontext so that changes can be written back into the database. Since you're caching these objects to be read by multiple threads, you do not want this. (It's also more expensive even in single-threaded cases to track objects you won't change, so worth doing in other places).
Also, use LoadWith to eagerly load any properties you might want to access of these cached entities, so they are all loaded on the initial caching thread, rather than with (potentially mulitple) threads that try to access them.
Ultimately I am trying to address the same issue that is referenced in Loading any MVC page fails with the error “An item with the same key has already been added.” and An item with the same key has already been added. A duplicate of the first link is All MVC pages fail with the message an item with the same key has already been added but it has some additional pertinent information, confirming that it only effects MVC pages, while webforms and other aspects of the application that deal with appSettings continue to work without error.
I have now seen this four times in production and it has not been seen in any other environment (dev, test, UAT). I have closely examined and debugged through the source code of System.Web.WebPages and the relevant sections of the MVC stack but did not run into anything that stood out. This problem has persisted through a migration from MVC3 to MVC4, and the latest changeset from aspnetwebstack.codeplex.com does not appear to address this issue.
A quick summary of the issue:
Every MVC page is affected and completely unusable
WebForms and other aspects of the application that use appSettings continue to work just fine
Only an appPool restart will correct this issue
At least once and as referenced in an article above, this has happened after a regular time interval recycle of the appPool by IIS
This has happened during both low and high volume traffic periods
This has happened on multiple production servers, but the issue only affects a single server at any given time, as other servers continue serving MVC pages
The offending line of code is var items = new Lazy<Dictionary<object, object>>(() => appSettings.AllKeys.ToDictionary(key => key, key => (object)appSettings[key], comparer));, but it occurs when the lazy initialization is forced by requesting a value from items The appSettings variable is from System.Web.WebConfigurationManager.AppSettings which is a direct static reference to System.Configuration.ConfigurationManager.AppSettings. So I beleive the line is equivalent to: var items = new Lazy<Dictionary<object, object>>(() => System.Configuration.ConfigurationManager.AppSettings.AllKeys.ToDictionary(key => key, key => (object)appSettings[key], comparer));
I rarely suspect framework issues, but it appears that appSettings has two distinct keys that are the same (not the same as a NameValueCollection supporting multiple values for the same key). The comparer being used in the MVC stack is the StringComparer.OrdinalIgnoreCase which appears to match what is used by the configuration system. If this is a framework issue, the MVC stack appears to be very unforgiving when it forces the NameValueColleciton into a dictionary using the ToDictionary() extension method. I believe using appSettings.AllKeys.Distinct().ToDictionary(...) would probably allow the MVC stack to operate normally as the rest of the application does and be oblivious to the possibility of duplicate keys. This unforgiving nature appears to also contribute to the issue described in NullReferenceException in WebConfigScopeDictionary
Server stack trace:
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
at System.Web.WebPages.Scope.WebConfigScopeDictionary.<>c__DisplayClass4.<.ctor>b__0()
at System.Lazy`1.CreateValue()
Exception rethrown at [0]:
at System.Lazy`1.get_Value()
at System.Web.WebPages.Scope.WebConfigScopeDictionary.TryGetValue(Object key, Object& value)
at System.Web.Mvc.ViewContext.ScopeGet[TValue](IDictionary`2 scope, String name, TValue defaultValue)
at System.Web.Mvc.ViewContext.ScopeCache..ctor(IDictionary`2 scope)
at System.Web.Mvc.ViewContext.ScopeCache.Get(IDictionary`2 scope, HttpContextBase httpContext)
at System.Web.Mvc.ViewContext.GetClientValidationEnabled(IDictionary`2 scope, HttpContextBase httpContext)
at System.Web.Mvc.Html.FormExtensions.FormHelper(HtmlHelper htmlHelper, String formAction, FormMethod method, IDictionary`2 htmlAttributes)
at ASP._Page_Areas_Client_Views_Equipment_Index_cshtml.Execute()
at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1a.<InvokeActionResultWithFilters>b__17()
at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation)
at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultWithFilters(ControllerContext controllerContext, IList`1 filters, ActionResult actionResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass25.<BeginInvokeAction>b__22(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c__DisplayClass1d.<BeginExecuteCore>b__18(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar)
at System.Web.Mvc.MvcHandler.<>c__DisplayClass8.<BeginProcessRequest>b__3(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
To separate my question from the questions already asked, I will ask has anyone seen the configuration system be corrupted with multiple duplicate keys or where a NameValueCollection.AllKeys returns two identical keys? I know you can have multiple keys defined in the config file, but the last key wins, and that scenario does not reproduce this issue.
Although I am not alone in seeing this behavior multiple times, there are relatively few posts describing this issue, so I also suspect that it might be a configuration/environmental issue, but again, servers will run for months without experiencing this issue, and an appPool restart immediately corrects the problem.
I have mitigated this issue by forcing an appPool restart if a server starts seeing this error, but management is not happy about this "hacky" solution because some user will still experience an error.
Help?!?!?
EDIT:
Here is a contrived, cheezy test that can reproduce the scenario, but doesn't help in solving the issue. It happens during approx. 20% of the test runs. The code will blow up for other threading reasons, but it is the "Same key has already been added" error that is of interest.
[TestClass]
public class UnitTest1
{
readonly NameValueCollection _nameValueCollection = new NameValueCollection();
private Lazy<Dictionary<object, object>> _items;
[TestMethod]
public void ReproduceSameKeyHasAlreadyBeenAdded()
{
Thread[] threads = new Thread[1000];
for (int i = 0; i < 1000; i++)
{
ThreadStart threadStart = AddEntry;
Thread thread = new Thread(threadStart);
threads[i] = thread;
}
foreach (var thread in threads)
thread.Start();
Thread.Sleep(100);
_items = new Lazy<Dictionary<object, object>>(() => _nameValueCollection.AllKeys.ToDictionary(key => key, key => (object)_nameValueCollection[key], ScopeStorageComparer.Instance));
object value;
_items.Value.TryGetValue("4", out value); // approx. 20% of time, blows up here with mentioned error
Assert.IsTrue(value != null);
}
private int _counter;
private void AddEntry()
{
_counter++;
try
{ // add a bunch of even keys, every other one a duplicate
_nameValueCollection.Add((_counter%2) == 0 ? _counter.ToString() : (_counter + 1).ToString(), "some value");
}
catch {}
}
}
StackTrace:
at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
at UnitTestProject1.ReproduceSameKeyHasAlreadyBeenAdded.<TestMethod1>b__0() in c:\Git\AspNetWebStack\aspnetwebstack\UnitTestProject1\UnitTest1.cs:line 37
at System.Lazy`1.CreateValue()
We came across this same issue and finally tracked it down to dynamic updating of the ConfigurationManager.AppSettings collection. I have posted a full answer here: https://stackoverflow.com/a/17415830/2423407
Are you absolutely positive the error is occurring on the line where you are initializing items and not the line on which items is being used to add to some other dictionary (Static I would presume).
To me the most likely way this would occur is if the code was executed in parallel (by two concurrent users) and the second one executing causing the exception.
As a general workaround I usually initialize a strongly typed class with all configuration parameters as an auto-start provider (and add some strongly-typed type checking of the values as well, such as checking that an int is an int, etc) to avoid run-time errors.
This has the double benefit of loading these at warmup, not when the users want the info (better response performance) and thread-safety is supposedly guaranteed with them as well.
See if that doesn't fix your issue. If you do not want to do that I would at the very least try to execute the code that is failing for you with multiple threats hitting it, as my guess would be that you will see your error occur fairly reliably.
Calling FluentMigrator's builder methods while inside the action that I pass to Execute.WithConnection causes a null reference exception to be thrown.
What I am trying to do is select some data so that I may manipulate it in c#, as that is easier than manipulating it in T-SQL, and use the result of my c# operations to update the data or insert new data (to be more specific, I need to pick one query string parameter out of a stored url string and insert it somewhere else).
The only way I see to select data within a migration is to use Execute.WithConnection and retrieve the data myself (FluentMigrator provides no helpers for selecting data), but if I try to use any fluent migrator expression in the action I pass to Execute.WithConnection a null reference exception is thrown.
Here is a boiled down version of my code:
[Migration(1)]
public class MyMigration : Migration
{
public void Up()
{
Execute.WithConnection(CustomDml);
}
public void CustomDml(IDbConnection conn, IDbTransaction tran)
{
var db = new NPoco.Database(conn).SetTransaction(tran); // NPoco is a micro-ORM, a fork of PetaPoco
var records = db.Fetch<Record>("-- some sql"); // this is immediately evaluated, no reader is left open
foreach (var r in records) {
var newValue = Manipulate(r.OriginalValue);
Insert.IntoTable("NewRecords").Row(new { OriginalValueId = r.Id, NewValue = newValue }); // <-- this line causes the exception
}
}
public void Down() {}
}
The line that calls Inser.IntoTable causes a null exception to be thrown from line 36 of FluentMigrator\Builders\Insert\InsertExpressionRoot.cs - it appears that the _context variable may be null at this point but I do not understand why this is. (when testing Create.Table, e.g., it occurs on line 49 of FluentMigrator\Builders\Create\CreateExpressionRoot.cs)
Any help would be appreciated. Perhaps there is disagreement on whether DML is appropriate in a migration, and I am open to suggestions, but this scenario has come up twice this week alone. For now I am simply performing the insert using my micro-ORM within the action rather than FluentMigrator and that does work, but it seems like what I am trying to do should work.
When using the Execute.WithConnection expression all you get is the db connection and the transaction.
Using Execute.WithConnection creates an PerformDBOperationExpression expression. When processing the expression, a processor calls the Operation property (an example in the SqlServerProcessor) and the processor does not have a reference to the MigrationContext. But even if it did have access to the MigrationContext, when FluentMigrator has come to the processing stage, it is already too late. You would be trying to process expressions in a expression and at the moment FluentMigrator is not built to handle that type of nesting.
An alternative would be to make the connection string available in the migration context, see this issue: https://github.com/schambers/fluentmigrator/issues/240
Would that be a better approach?
as part of a more complex project we work on our own workflow persistence layer for workflow foundation.
I got load and save running but have a problem that only get unusable workflows back. I am stuck somewhere and just fail to see where.
Any workflow I load I load like this:
WorkflowApplication wf2App = new WorkflowApplication(new WorkflowInstanceStoreTestsSimplePersistence());
wf2App.InstanceStore = store;
wf2App.Load(wfApp.Id);
This looks nice - I get a workflow back. I hook up the handlers and when I do Run ()... I get...
...Abort.
The reason is:
An error processing the current work item has caused the workflow to abort. See the
inner exception for details.
The inner exception is:
The persistence provider implementation of InstanceStore doesn't support the command
named {urn:schemas-microsoft-com:System.Activities.Persistence/command}SaveWorkflow.
Either choose a different provider, or ensure that this persistence command isn't
attempted.
The real problem with that is that I fail to see that in my implementation. I simply never return an error and every call into a command handler returns without errors.
The stack trace is not helpfull either:
at System.Runtime.AsyncResult.End[TAsyncResult](IAsyncResult result)
at System.Runtime.DurableInstancing.InstancePersistenceContext.ExecuteAsyncResult.End(IAsyncResult result)
at System.Runtime.DurableInstancing.InstancePersistenceContext.EndOuterExecute(IAsyncResult result)
at System.Runtime.DurableInstancing.InstanceStore.EndExecute(IAsyncResult result)
at System.Activities.WorkflowApplication.PersistenceManager.EndSave(IAsyncResult result)
at System.Activities.WorkflowApplication.UnloadOrPersistAsyncResult.OnPersisted(IAsyncResult result)
at System.Runtime.AsyncResult.SyncContinue(IAsyncResult result)
at System.Activities.WorkflowApplication.UnloadOrPersistAsyncResult.Persist()
at System.Activities.WorkflowApplication.UnloadOrPersistAsyncResult.CollectAndMap()
at System.Activities.WorkflowApplication.UnloadOrPersistAsyncResult.Track()
at System.Activities.WorkflowApplication.UnloadOrPersistAsyncResult.EnsureProviderReadyness()
at System.Activities.WorkflowApplication.UnloadOrPersistAsyncResult.InitializeProvider()
at System.Activities.WorkflowApplication.UnloadOrPersistAsyncResult..ctor(WorkflowApplication instance, TimeSpan timeout, PersistenceOperation operation, Boolean isWorkflowThread, Boolean isInternalPersist, AsyncCallback callback, Object state)
at System.Activities.WorkflowApplication.BeginInternalPersist(PersistenceOperation operation, TimeSpan timeout, Boolean isInternalPersist, AsyncCallback callback, Object state)
at System.Activities.WorkflowApplication.OnBeginPersist(AsyncCallback callback, Object state)
at System.Activities.Runtime.ActivityExecutor.PersistenceWaiter.PersistWorkItem.Execute(ActivityExecutor executor, BookmarkManager bookmarkManager)
All my command operations are in the InstanceStore override for TryCommand and that just works without fault.
The handler for the SaveWorkflowCommand is:
void Pro
cessSaveWorkflow (InstancePersistenceContext context, SaveWorkflowCommand command)
{
if (command.CompleteInstance)
{
DataStore.DeleteInstance(context.InstanceView.InstanceId);
DataStore.DeleteInstanceAssociation(context.InstanceView.InstanceId);
return;
}
if (command.InstanceData.Count > 0 || command.InstanceKeyMetadataChanges.Count > 0)
{
if (!DataStore.SaveAllInstanceData(context.InstanceView.InstanceId, command))
{
DataStore.SaveAllInstanceMetaData(context.InstanceView.InstanceId, command);
}
if (command.InstanceKeysToAssociate.Count > 0)
{
foreach (var entry in command.InstanceKeysToAssociate)
{
DataStore.SaveInstanceAssociation(context.InstanceView.InstanceId, entry.Key, false);
}
}
return;
}
}
and works without issues (datastore calls I jsut don't publish here).
I start hinking I may forget some call to set a ok status, but I follow the examples from Pro WF (for 4.0) (the book) and it just does not work.
Anyone an idea?
A WF4 custom instance store is a very tricky thing to write and there is very little documentation :-(
Besides the samples Jota mentioned, which are useful but not the easiest to get started with, there is a bit of documentation here. Take a good look at the XmlWorkflowInstanceStore.BeginTryCommand() and the way it checks for the command with code like if (command is SaveWorkflowCommand) and finally returns a new CompletedAsyncResult<bool>(true, callback, state)
I'm using the mvc-mini-profiler in my project built with ASP.Net MVC 3 and Entity Framework code-first.
Everything works great until I attempt to add database profiling by wrapping the connection in the ProfiledDbConnection as described in the documentation. Since I'm using a DbContext, the way I am attempting to provide the connection is through the constructor using a static factory method:
public class MyDbContext : DbContext
{
public MyDbContext() : base(GetProfilerConnection(), true)
{ }
private static DbConnection GetProfilerConnection()
{
// Code below errors
//return ProfiledDbConnection.Get(new SqlConnection(ConfigurationManager.ConnectionStrings["MyConnectionName"].ConnectionString));
// Code below works fine...
return new SqlConnection(ConfigurationManager.ConnectionStrings["MyConnectionName"].ConnectionString);
}
//...
}
When using the ProfiledDbConnection, I get the following error:
ProviderIncompatibleException: The provider did not return a ProviderManifestToken string.
Stack Trace:
[ArgumentException: The connection is not of type 'System.Data.SqlClient.SqlConnection'.]
System.Data.SqlClient.SqlProviderUtilities.GetRequiredSqlConnection(DbConnection connection) +10486148
System.Data.SqlClient.SqlProviderServices.GetDbProviderManifestToken(DbConnection connection) +77
System.Data.Common.DbProviderServices.GetProviderManifestToken(DbConnection connection) +44
[ProviderIncompatibleException: The provider did not return a ProviderManifestToken string.]
System.Data.Common.DbProviderServices.GetProviderManifestToken(DbConnection connection) +11092901
System.Data.Common.DbProviderServices.GetProviderManifestToken(DbConnection connection) +11092745
System.Data.Entity.DbModelBuilder.Build(DbConnection providerConnection) +221
System.Data.Entity.Internal.LazyInternalContext.CreateModel(LazyInternalContext internalContext) +61
System.Data.Entity.Internal.RetryLazy`2.GetValue(TInput input) +1203482
System.Data.Entity.Internal.LazyInternalContext.InitializeContext() +492
System.Data.Entity.Internal.InternalContext.GetEntitySetAndBaseTypeForType(Type entityType) +26
System.Data.Entity.Internal.Linq.InternalSet`1.Initialize() +89
System.Data.Entity.Internal.Linq.InternalSet`1.get_InternalContext() +21
System.Data.Entity.Infrastructure.DbQuery`1.System.Linq.IQueryable.get_Provider() +44
System.Linq.Queryable.Where(IQueryable`1 source, Expression`1 predicate) +135
I have stepped through and the type returned by ProfiledDbConnection.Get is of type ProfiledDbConnection (Even if the current MiniProfiler is null).
The MiniProfiler.Start() method is called within the Global Application_BeginRequest() method before the DbContext is instantiated. I am also calling the Start method for every request regardless but calling stop if the user is not in the correct role:
protected void Application_BeginRequest()
{
// We don't know who the user is at this stage so need to start for everyone
MiniProfiler.Start();
}
protected void Application_AuthorizeRequest(Object sender, EventArgs e)
{
// Now stop the profiler if the user is not a developer
if (!AuthorisationHelper.IsDeveloper())
{
MvcMiniProfiler.MiniProfiler.Stop(discardResults: true);
}
}
protected void Application_EndRequest()
{
MiniProfiler.Stop();
}
I'm not sure if this affects things but I'm also using StructureMap as IoC for the DbContext using the following initialiser:
For<MyDbContext>().Singleton().HybridHttpOrThreadLocalScoped();
I understand that there is a similar question on here with a good explanation of what's happening for that user, however it doesn't seem to solve my problem.
EDIT:
For clarity. I am attempting to pass the connection as ProfiledDbConnection in order to profile the generated sql from Entity Framework Code First.
The Entity Framework is expecting a connection with type SqlConnection which of course this isn't.
Here is an example of my connection string (notice the providerName)
<add name="MyDbContext" connectionString="Server=.\SQLEXPRESS; Database=MyDatabase;Trusted_Connection=true;MultipleActiveResultSets=true" providerName="System.Data.SqlClient" />
I attempted to create my own version of the ProfiledDbConnection inheriting from SqlConnection but it is a sealed class.
If there is some way of telling Entity Framework about the custom connection type then perhaps this would work. I tried setting the providerName in the connection string to MvcMiniProfiler.Data.ProfiledDbConnection but that didn't work.
So. Perhaps an evolution of the question would be: How can you pass a custom connection type to Entity Framework Code First?
This is now fully supported, check out the latest source or grab the package from nuget.
You will need the MiniProfiler.EF package if you are using nuget. (1.9.1 and up)
Supporting this involved a large set of modifications to the underlying proxy object to support acting as EF code first proxies.
To add this support:
During your Application_Start run:
MiniProfilerEF.Initialize();
Note: EF Code First will store table metadata in a table called: EdmMetadata. This metadata uses the provider as part of the entity key. If you initialized your provider as a non-profiled provider, you will have to re-build this metadata. Deleting all the rows from EdmMetadata may do the trick, alternatively some smarter providers are able to handle this transparently.
I was still having problems getting this to work and found that I needed to rename or remove the connection string to get Database.DefaultConnectionFactory to work.
Please refer to this answer for more detail.
This error in my experience has always been an invalid connection string, or a lack of connection to the DB, like "A network service error occurred while connecting...".
Also note that the DbContext just needs the "connectionStringKey" in the constructor, like
public MyDbContext() :
base("MyConnectionName", true)
{ }