Some questions concerning the combination of ADO.NET, Dapper QueryAsync and Glimpse.ADO - asp.net

I have been experimenting with a lightweight solution for handling my business logic. It consists of a vanilla ADO.NET connection that is extended with Dapper, and monitored by Glimpse.ADO. The use case for this setup will be a web application that has to process a handful of queries asynchronously per request. Below a simple implementation of my setup in an MVC controller.
public class CatsAndDogsController : Controller
{
public async Task<ActionResult> Index()
{
var fetchCatsTask = FetchCats(42);
var fetchDogsTask = FetchDogs(true);
await Task.WhenAll(fetchCatsTask, fetchDogsTask);
ViewBag.Cats = fetchCatsTask.Result;
ViewBag.Dogs = fetchDogsTask.Result;
return View();
}
public async Task<IEnumerable<Cat>> FetchCats(int breedId)
{
IEnumerable<Cat> result = null;
using (var connection = CreateAdoConnection())
{
await connection.OpenAsync();
result = await connection.QueryAsync<Cat>("SELECT * FROM Cat WHERE BreedId = #bid;", new { bid = breedId });
connection.Close();
}
return result;
}
public async Task<IEnumerable<Dog>> FetchDogs(bool isMale)
{
IEnumerable<Dog> result = null;
using (var connection = CreateAdoConnection())
{
await connection.OpenAsync();
result = await connection.QueryAsync<Dog>("SELECT * FROM Dog WHERE IsMale = #im;", new { im = isMale });
connection.Close();
}
return result;
}
public System.Data.Common.DbConnection CreateAdoConnection()
{
var sqlClientProviderFactory = System.Data.Common.DbProviderFactories.GetFactory("System.Data.SqlClient");
var dbConnection = sqlClientProviderFactory.CreateConnection();
dbConnection.ConnectionString = "SomeConnectionStringToAwesomeData";
return dbConnection;
}
}
I have some questions concerning the creation of the connection in the CreateAdoConnection() method. I assume the following is happening behind the scenes.
The call to sqlClientProviderFactory.CreateConnection() returns an instance of System.Data.SqlClient.SqlConnection passed as a System.Data.Common.DbConnection. At this point Glimpse.ADO.AlternateType.GlimpseDbProviderFactory kicks in and wraps this connection in an instance of Glimpse.Ado.AlternateType.GlimpseDbConnection, which is also passed as a System.Data.Common.DbConnection. Finally, this connection is indirectly extended by the Dapper library with its query methods, among them the QueryAsync<>() method used to fetch the cats and dogs.
The questions:
Is the above assumption correct?
If I use Dapper's async methods with this connection - or create a System.Data.Common.DbCommand with this connection's CreateCommand() method, and use it's async methods - will those calls internally always end up using the vanilla async implementations of these methods as Microsoft has written them for System.Data.SqlClient.SqlConnection and System.Data.SqlClient.SqlCommand? And not some other implementations of these methods that are actually blocking?
How much perf do I lose with this setup compared to just returning a new System.Data.SqlClient.SqlConnection directly? (So, without the Glimpse.ADO wrapper)
Any suggestions on improving this setup?

Yes pretty much. GlimpseDbProviderFactory wraps/decorates/proxies all the registered factories. We then pass any calls we get through to the factory we wrap (in this case SQL Server). In the case of CreateConnection() we ask the inner factory we have, to create a connection, when we get that connection, we wrap it and then return it to the originating caller
Yes. Glimpse doesn't turn what was an async request into a blocking request. We persevere the async chain all the way though. If you are interested, the code in question is here.
Very little. In essence, using a decorator pattern like this adds only one or two frames to the call stack. Compared to most operations performed during the request lifecycle, the time to observe whats happening here is extremely minimal.
What you have looks great. Only suggestion is to maybe us this code to build the factory. This code means that you can shift your connection string, etc to the web.config.

Related

how to work with XREAD .NET Redis to read actual changes

When i am set stop point at line with XREAD programm doing nothing. Maybe i need to configure this XREAD command?
public async void ListenTask()
{
var readTask = Task.Run(async () =>
{
while (!Token.IsCancellationRequested)
{
var result = db.StreamRead(streamName, "$", 1);
if (result.Any())
{
var dict = ParseResult(result.Last());
var sb = new StringBuilder();
foreach (var key in dict.Keys)
{
sb.Append(dict[key]);
}
Console.WriteLine(sb.ToString());
}
await Task.Delay(1000);
}
});
}
It's an illegal operation on StackExchange.Redis
Because of its unique multiplexed architecture, StackExchange.Redis (the library you appear to be using) does not support blocking XREAD operations. This is because all the commands going over the interactive interface (basically everything non-pub/sub) uses the same connection. If you block one of those connections, everything else in your app dependent on the multiplexer will be backed up awaiting the block to complete. The StackExchange.Redis library actually goes so far as to consider the $ id an illegal id, it's only purpose after all is to block. What's most likely happening (you don't see it happen because it's being swallowed up by the synchronization context) is that var result = db.StreamRead(streamName, "$", 1); is throwing an InvalidOperationException: System.InvalidOperationException: StreamPosition.NewMessages cannot be used with StreamRead.
Work Arounds
There are 2 potential workarounds in this case, first, you can use poll with XRANGE command rather than using blocking reads.
var readTask = Task.Run(async () =>
{
var lastId = "-";
while (!token.IsCancellationRequested)
{
var result = await db.StreamRangeAsync("a-stream", lastId, "+");
if(result.Any()
{
lastId = result.Last().Id;
}
await Task.Delay(1000);
}
});
You're already effectively doing a thread sleep so this polling operation is probably good enough for what you're looking for.
If you really need to do blocking operations, you'll have to use a different library, if you try to use StackExchange.Redis (you can possibly force the issue with the Execute/ExecuteAsync commands) you can seriously negatively degrade its performance.
Articles on doing so with ServiceStack.Redis and CsRedis are available on the redis developer site (I'm the author of them)
One final thing
Probably want to make sure that when you are issuing these commands that you are being as async as possible, you're using the sync XREAD command in an Async context (mostly every command in StackExchange.Redis has a sync/async version you can use - use the async when possible).

SignalR return value from client method

Hello I'm developing a Server-Client application that communicate with SignalR. What I have to implement is a mechanism that will allow my server to call method on client and get a result of that call. Both applications are developed with .Net Core.
My concept is, Server invokes a method on Client providing Id of that invocation, the client executes the method and in response calls the method on the Server with method result and provided Id so the Server can match the Invocation with the result.
Usage is looking like this:
var invocationResult = await Clients
.Client(connectionId)
.GetName(id)
.AwaitInvocationResult<string>(ClientInvocationHelper._invocationResults, id);
AwaitInvocationResult - is a extension method to Task
public static Task<TResultType> AwaitInvocationResult<TResultType>(this Task invoke, ConcurrentDictionary<string, object> lookupDirectory, InvocationId id)
{
return Task.Run(() =>
{
while (!ClientInvocationHelper._invocationResults.ContainsKey(id.Value)
|| ClientInvocationHelper._invocationResults[id.Value] == null)
{
Thread.Sleep(500);
}
try
{
object data;
var stingifyData = lookupDirectory[id.Value].ToString();
//First we should check if invocation response contains exception
if (IsClientInvocationException(stingifyData, out ClientInvocationException exception))
{
throw exception;
}
if (typeof(TResultType) == typeof(string))
{
data = lookupDirectory[id.Value].ToString();
}
else
{
data = JsonConvert.DeserializeObject<TResultType>(stingifyData);
}
var result = (TResultType)data;
return Task.FromResult(result);
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
});
}
As you can see basically I have a dictionary where key is invocation Id and value is a result of that invocation that the client can report. In a while loop I'm checking if the result is already available for server to consume, if it is, the result is converted to specific type.
This mechanism is working pretty well but I'm observing weird behaviour that I don't understand.
If I call this method with await modifier the method in Hub that is responsible to receive a result from client is never invoked.
///This method gets called by the client to return a value of specific invocation
public Task OnInvocationResult(InvocationId invocationId, object data)
{
ClientInvocationHelper._invocationResults[invocationId.Value] = data;
return Task.CompletedTask;
}
In result the while loop of AwaitInvocationResult never ends and the Hub is blocked.
Maby someone can explain this behaviour to me so I can change my approach or improve my code.
As it was mentioned in the answer by Brennan, before ASP.NET Core 5.0 SignalR connection was only able to handle one not streaming invocation of hub method at time. And since your invocation was blocked, server wasn't able to handle next invocation.
But in this case you probably can try to handle client responses in separate hub like below.
public class InvocationResultHandlerHub : Hub
{
public Task HandleResult(int invocationId, string result)
{
InvoctionHelper.SetResult(invocationId, result);
return Task.CompletedTask;
}
}
While hub method invocation is blocked, no other hub methods can be invoked by caller connection. But since client have separate connection for each hub, he will be able to invoke methods on other hubs. Probably not the best way, because client won't be able to reach first hub until response will be posted.
Other way you can try is streaming invocations. Currently SignalR doesn't await them to handle next message, so server will handle invocations and other messages between streaming calls.
You can check this behavior here in Invoke method, invocation isn't awaited when it is stream
https://github.com/dotnet/aspnetcore/blob/c8994712d8c3c982111e4f1a09061998a81d68aa/src/SignalR/server/Core/src/Internal/DefaultHubDispatcher.cs#L371
So you can try to add some dummy streaming parameter that you will not use:
public async Task TriggerRequestWithResult(string resultToSend, IAsyncEnumerable<int> stream)
{
var invocationId = InvoctionHelper.ResolveInvocationId();
await Clients.Caller.SendAsync("returnProvidedString", invocationId, resultToSend);
var result = await InvoctionHelper.ActiveWaitForInvocationResult<string>(invocationId);
Debug.WriteLine(result);
}
and on the client side you will also need to create and populate this parameter:
var stringResult = document.getElementById("syncCallString").value;
var dummySubject = new signalR.Subject();
resultsConnection.invoke("TriggerRequestWithResult", stringResult, dummySubject);
dummySubject.complete();
More details: https://learn.microsoft.com/en-us/aspnet/core/signalr/streaming?view=aspnetcore-5.0
If you can use ASP.NET Core 5, you can try to use new MaximumParallelInvocationsPerClient hub option. It will allow several invocations to execute in parallel for one connection. But if your client will call too much hub methods without providing result, connection will hang.
More details: https://learn.microsoft.com/en-us/aspnet/core/signalr/configuration?view=aspnetcore-5.0&tabs=dotnet
Actually, since returning values from client invocations isn't implemented by SignalR, maybe you can try to look into streams to return values into hubs?
This is supported in .NET 7 now https://learn.microsoft.com/en-us/aspnet/core/signalr/hubs?view=aspnetcore-7.0#client-results
By default a client can only have one hub method running at a time on the server. This means that when you wait for a result in the first hub method, the second hub method will never run since the first hub method is blocking the processing loop.
It would be better if the OnInvocationResult method ran the logic in your AwaitInvocationResult extension and the first hub method just registers the id and calls the client.

Parallel httprequest in UWP app

I'm creating an app that requires todo parallel http request, I'm using HttpClient for this.
I'm looping over the urls and foreach URl I start a new Task todo the request.
after the loop I wait untill every task finishes.
However when I check the calls being made with fiddler I see that the request are being called synchronously. It's not like a bunch of request are being made, but one by one.
I've searched for a solution and found that other people have experienced this too, but not with UWP. The solution was to increase the DefaultConnectionLimit on the ServicePointManager.
The problem is that ServicePointManager does not exist for UWP. I've looked in the API's and I thought I could set the DefaultConnectionLimit on HttpClientHandler, but no.
So I have a few Questions.
Is DefaultConnectionLimit still a property that could be set somewhere?
if so, where do i set it?
if not, how do I increase the connnectionlimit?
Is there still a connectionlimit in UWP?
this is my code:
var requests = new List<Task>();
var client = GetHttpClient();
foreach (var show in shows)
{
requests.Add(Task.Factory.StartNew((x) =>
{
((Show)x).NextEpisode = GetEpisodeAsync(((Show)x).NextEpisodeUri, client).Result;}, show));
}
}
await Task.WhenAll(requests.ToArray());
and this is the request:
public async Task<Episode> GetEpisodeAsync(string nextEpisodeUri, HttpClient client)
{
try
{
if (String.IsNullOrWhiteSpace(nextEpisodeUri)) return null;
HttpResponseMessage content; = await client.GetAsync(nextEpisodeUri);
if (content.IsSuccessStatusCode)
{
return JsonConvert.DeserializeObject<EpisodeWrapper>(await content.Content.ReadAsStringAsync()).Episode;
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
return null;
}
Oke. I have the solution. I do need to use async/await inside the task. The problem was the fact I was using StartNew instead of Run. but I have to use StartNew because i'm passing along a state.
With the StartNew. The task inside the task is not awaited for unless you call Unwrap. So Task.StartNew(.....).Unwrap(). This way the Task.WhenAll() will wait untill the inner task is complete.
When u are using Task.Run() you don't have to do this.
Task.Run vs Task.StartNew
The stackoverflow answer
var requests = new List<Task>();
var client = GetHttpClient();
foreach (var show in shows)
{
requests.Add(Task.Factory.StartNew(async (x) =>
{
((Show)x).NextEpisode = await GetEpisodeAsync(((Show)x).NextEpisodeUri, client);
}, show)
.Unwrap());
}
Task.WaitAll(requests.ToArray());
I think an easier way to solve this is not "manually" starting requests but instead using linq with an async delegate to query the episodes and then set them afterwards.
You basically make it a two step process:
Get all next episodes
Set them in the for each
This also has the benefit of decoupling your querying code with the sideeffect of setting the show.
var shows = Enumerable.Range(0, 10).Select(x => new Show());
var client = new HttpClient();
(Show, Episode)[] nextEpisodes = await Task.WhenAll(shows
.Select(async show =>
(show, await GetEpisodeAsync(show.NextEpisodeUri, client))));
foreach ((Show Show, Episode Episode) tuple in nextEpisodes)
{
tuple.Show.NextEpisode = tuple.Episode;
}
Note that i am using the new Tuple syntax of C#7. Change to the old tuple syntax accordingly if it is not available.

Asychronous Task works with HttpClient object but not with project's class in Web Generic Handler

I found an interesting differences between .NET Framework's HttpClient class/objects and the VS-2013 Project PhotoServer (DLL) class/objects. It made me wonder if there's a bug with the script.
I'm using .NET Framework v4.5.1.
I'm using the HttpClient script in the sychronous Web Generic Handler. Noticed that I'm using the ".Result" for the asynchronous POST to wait for response. So, looking at HttpClient which works is
using (var httpClient = new HttpClient())
{
var response = httpClient.PostAsync(
_baseUrl,
new FormUrlEncodedContent
(
new List<KeyValuePair<string, string>>
{
new KeyValuePair<string, string>("Vin", parmVin),
new KeyValuePair<string, string>("ImageSize", parmImageSize)
}.ToArray()
)
).Result;
//returned string[] datatype...
var photoUrls = response.Content.ReadAsStringAsync().Result;
}
I'm using the "GetPhotoUrlsAsync" script in the sychronous Web Generic Handler. This "GetPhotoUrlsAsync" object comes from the Project class (DLL). Again, I'm using the ".Result" and it doesn't work, it just deadlocked and hung. What I wanna know is why is that and was there a bug with the script?
//[Scripts in Web Generic Handlers]...
var managerVehiclePhoto = new ManagerVehiclePhoto();
var photoUrls = managerVehiclePhoto.GetPhotoUrlsAsync("12345678901234567").Result;
//[Project Class]...
namespace BIO.Dealer.Integration.PhotoServer
{
public seal class VehiclePhotoManager
{
public async Task<string[]> GetPhotoUrlsAsync(string vin)
{
var listResponse = await _client.ListAsync(vin);
return listResponse.ToArray();
}
}
}
Thanks...
Edit #1
//Synchronous API Call...
public string[] GetPhotoUrls(string vin)
{
return GetPhotoUrlsAsync(vin).Result;
}
Using .Result like this is actually a bug in both cases; it just happens not to deadlock in the HttpClient case. Note that the same HttpClient library on other platforms (notably Windows Phone, IIRC) will deadlock if used like this.
I describe the deadlock in detail on my blog, but the gist of it is this:
There's an ASP.NET "request context" that is captured by default every time you use await. When the async method resumes, it will resume within that context. However, types such as HttpContext are not multithread-safe, so ASP.NET restricts that context to one thread at a time. So if you block a thread by calling .Result, it's blocking a thread inside that context.
The reason GetPhotoUrlsAsync deadlocks is because it's an async method that is attempting to resume inside that context, but there is already a thread blocked in that context. The reason HttpClient happens to work is because GetAsync etc. are not actually async methods (note that this is an implementation detail and you should not depend on this behavior).
The best way to fix this is to replace .Result with await:
var managerVehiclePhoto = new ManagerVehiclePhoto();
var photoUrls = await managerVehiclePhoto.GetPhotoUrlsAsync("12345678901234567");
Me again (re: same quest WebPages async) :) That said, I'm one of those that Stephen Cleary identifies as "trying to migrate into async", so all this is (still) a learning moment.
The issue is SynchronizationContext in GUI/ASP.Net. I won't mangle Stephen's explanation so the link is your best bet to grok.
Given the best practices in that article, here's my way (consequently what I use in WebPages for the "top level call") to "mock" awaiting a PostAsync call like what you're doing. In this case I'm using ConfigureAwait (call is from WebPages, not MVC):
public static async Task<string> PostToRequestBin()
{
var _strContent = new FormUrlEncodedContent(new[] {new KeyValuePair<string,string>("fizz","buzz")});
using (var client = new HttpClient())
{
/*
* See http://requestb.in/ for usage
*/
var result = await client.PostAsync("http://requestb.in/xyzblah", _strContent).ConfigureAwait(false);
return await result.Content.ReadAsStringAsync();
}
}
In my WebPages page:
#{
//Class2 is a mock "library" in App_Code where the above async code lives
var postcatcher = Class2.PostToRequestBin();
}
And to make use of it somewhere in the page (where I make use of Task<string>.Result:
<p>#postcatcher.Result</p>
Again, this is a learning moment and I hope it helps/guides you. I fully expect the SO community to comment and or correct/improve on this like:
"Why don't I have to ConfigureAwait on ReadAsStringAsync" (it works either way)?
because at this point, it's "async all the way". I could have awaited some other async method...
...so the learning moments continue :)

How do I set up Moq so that I can unit test adding multiple groups and clients in SignalR?

I've been working on coming with a SignalR Unit testing framework using Moq.
I have been able things to get reasonably well with the 1 group - 1 client (connection) scenario.
How do I set up Moq so I can:
1) Add/remove multiple clients from the same group?
2) Add/remove multiple groups on the same mocked hub?
I'm relatively new to the world of Moq and SignalR combination.
Thanks in advance,
JohnB
Here is an example testing adding a client to multipe groups using Moq and xUnit.net:
[Fact]
public async Task MyHubAddsConnectionToTheCorrectGroups()
{
// Arrange
var groupManagerMock = new Mock<IGroupManager>();
var connectionId = Guid.NewGuid().ToString();
var groupsJoined = new List<string>();
groupManagerMock.Setup(g => g.Add(connectionId, It.IsAny<string>()))
.Returns(Task.FromResult<object>(null))
.Callback<string, string>((cid, groupToJoin) =>
groupsJoined.Add(groupToJoin));
var myHub = new MyHub();
myHub.Groups = groupManagerMock.Object;
myHub.Context = new HubCallerContext(request: null,
connectionId: connectionId);
// Act
await myHub.AddToGroups();
// Assert
groupManagerMock.VerifyAll();
Assert.Equal(3, groupsJoined.Count);
Assert.Contains("group1", groupsJoined);
Assert.Contains("group2", groupsJoined);
Assert.Contains("group3", groupsJoined);
}
public class MyHub : Hub
{
public async Task AddToGroups()
{
await Groups.Add(Context.ConnectionId, "group1");
await Groups.Add(Context.ConnectionId, "group2");
await Groups.Add(Context.ConnectionId, "group3");
}
}
The basic idea is to define a Callback along with your Setup that stores arguments important to your test inside a collection. You can then use the collection verify that the method you mocked was called the right number of times with the right arguments. I don't verify the order of the calls to Groups.Add in my example test, but you can test that as well.
This pattern extends pretty trivially to testing the adding/removing of multiple clients. Basically, you would just need a second collection to store the connectionId arguments passed to Groups.Add.

Resources