Error calling HttpClient.GetAsync: The underlying connection was closed - asp.net

I have an asp.net MVC website which is consuming a rest api to receive it's data. I'm using asynchronous tasks to perform the requests as there can be many on each page. After a while of uptime the website has been throwing the following error when trying to receive data.
The underlying connection was closed: An unexpected error occurred on a send.
I read that this could be due to the maxconnection settings on the web.config but increasing this doesn't seem to make much difference.
I'm also using caching to reduce the load on the api. The task is cached so the result can be used later.
The only way I've found to fix this is by recycling the application pool. Any help would be appreciated.
/* Code from page_load */
var currenciesTask = ApiClient.GetAsync<CurrencyListWrapper>("currencies");
var blogArticleTask = ApiClient.GetAsync<BlogArticleListWrapper>("blog/articles", "limit=10");
var seoPageTask = ApiClient.GetAsync<SEOPageListWrapper>("seopages");
await Task.WhenAll(currenciesTask, blogArticleTask, seoPageTask);
/* Code from data access later */
public class ApiClient : HttpClient
{
public static Task<T> GetAsync<T>(string operation, string query = null, bool cache = true)
{
// Check if task is in cache
string cacheName = null;
if (cache)
{
cacheName = String.Format("{0}_{1}_{2}", operation, query ?? String.Empty, App.GetLanguage());
var cachedTask = HttpRuntime.Cache[cacheName];
if (cachedTask != null)
{
return (Task<T>)cachedTask;
}
}
// Get data task
var task = GetAsyncData<T>(operation, query);
// Add to cache if required
if (task != null && cache)
{
App.AddToCache(cacheName, task);
}
return task;
}
public static async Task<T> GetAsyncData<T>(string operation, string query = null)
{
using (ApiClient client = new ApiClient())
{
string url;
if (query != null)
{
url = String.Format("{0}?{1}", operation, query);
}
else
{
url = String.Format("{0}", operation);
}
var response = await client.GetAsync(url);
return (await response.Content.ReadAsAsync<T>());
}
}
}

This is wrong,
The task is cached so the result can be used later.
You are supposed to cache result, not the task. At end of first execution, your HttpClient is closed and when you try to retrieve cached task, it will not work.
public class ApiClient : HttpClient
{
public static async Task<T> GetAsync<T>(string operation, string query = null, bool cache = true)
{
// Check if task is in cache
string cacheName = null;
if (cache)
{
cacheName = String.Format("{0}_{1}_{2}", operation, query ?? String.Empty, App.GetLanguage());
T cachedResult = (T)HttpRuntime.Cache[cacheName];
if (cachedResult!= null)
{
return Task.FromResult(cachedResult);
}
}
// Get data task
var result = await GetAsyncData<T>(operation, query);
// Add to cache if required
if (result != null && cache)
{
App.AddToCache(cacheName, result);
}
return result;
}
public static async Task<T> GetAsyncData<T>(string operation, string query = null)
{
using (ApiClient client = new ApiClient())
{
string url;
if (query != null)
{
url = String.Format("{0}?{1}", operation, query);
}
else
{
url = String.Format("{0}", operation);
}
var response = await client.GetAsync(url);
return (await response.Content.ReadAsAsync<T>());
}
}
}

Akash could be right.
But it seems more or less connection issue with application pool. Set the connection limit 0 to make it unlimited at application pool.
Have a finally block in you code, and
gc.collect();
garbage collection method to be called to remove unused connections to make space for other connection.

Related

Record request and response from client layer for audit logging in .net core without using middleware

Is there a way of logging the request and response from the client layer(not from controller as we can use middleware to log the same there).
I am looking to eliminate developer code for audit log here (//log request ,//log response and and creating a provider context ) instead move them to a common handler , may be inherit from delegating handler delegating handler and have the Audit log code there.
Any ideas ?
Currently we have audit logging in the client where another service is called but the developer has to do the following :
Client layer code:
{
IRestResponse response = null;
ConnectorHTMLResponse CCMSResponse = null;
request.Validate(request.TemplateName);
var providerContext = _messageTracker.CreateProviderContext(correlationId, "MailTrigger", "GetHTML", OperationProtocols.HTTPS);
//log request
await providerContext.StartAsync(request, param => request.TemplateName);
var bodyJson = ToBodyJson(request, TemplateType.HTML);
try
{
response = await ExecuteAsync(bodyJson, correlationId);
}
catch (Exception ex)
{
await providerContext.RaiseExceptionAsync(ex);
throw;
}
Response = ConstructHTMLDocumentDetails(ValidateResponse(response));
//log response
await providerContext.CompletedAsync(Response);
return Response;
}
//and in the message tracker(Common code )
public static ProviderContext CreateProviderContext(this IMessageTracker messageTracker, string correlationId, string systemId, string operationName, OperationProtocols protocol)
{
var context = new ProviderContext(
messageTracker,
correlationId,
systemId,
operationName,
Assembly.GetCallingAssembly().GetName().Name,
protocol
);
return context;
}
public async Task StartAsync<T>(T payload, Func<T, string> primaryIdentifierFunc = null, Func<T, string> secondaryIdentifierFunc = null)
{
await StartAsync(payload, primaryIdentifierFunc?.Invoke(payload), secondaryIdentifierFunc?.Invoke(payload));
}
public async Task CompletedAsync<T>(T payload, Func<T, string> primaryIdentifierFunc = null, Func<T, string> secondaryIdentifierFunc = null)
{
_source.Payload = payload.AsPayload();
_source.PrimaryIdentifier = primaryIdentifierFunc?.Invoke(payload) ?? _source.PrimaryIdentifier;
_source.SecondaryIdentifier = secondaryIdentifierFunc?.Invoke(payload) ?? _source.SecondaryIdentifier;
await _tracker.TrackProviderResponseAsync(
//track in cloud
);
}``

SocketError when creating multiple HttpClient Connections with the Load balancer(F5)

This is the exact exception in the error logs.
I have a requirement where I should be creating a set of service calls and wait till all of them are completed successfully before moving on to do a different set of service calls.
I have two services Service1 and Service2.
There are 2 instances of Service2 in 2 different servers and we have set up an F5(Load Balancer) to distribute the load evenly.
Let's say I have 10 service calls to be made from Service1 to Service2 at a time and F5 will share those 10 calls among the 2 servers. i.e. 5 Calls to Service2 of each server.
But I observed that if any one of those 10calls is taking lot of time to complete the work it should do(The work to be done in Service2 has some heavy lifting) then i get a socket exception thrown and entire process gets stopped.
However when I dont use the F5 load balancer and just use 1 instance of the Service2. Then however long the process takes for any of those 10 calls it doesn't throw any exception.
I am not sure if this is an issue with F5 configuration or with the way connections are made with the F5 from .Net code.
Please go through the below code to get some idea of what i am trying to do and let me know if any code change would help me resolve it.
for (int i=0 ; i< ReqList.Count;i++)
{
maxTasks++;
ClassA reqList = new ClassA();
reqList = ReqList[i];
List<ClassA> recsByReq = recs.Where(x => x.ReqId == reqList.ReqtId).ToList();
ClassC service2Input = new ClassC();
service2Input.DetaiList = listofRecs;
service2Input.RecList = recsByReq;
taskList.Add(_service2.Service2MethodCall(service2Input, service2Resource));
if(maxTasks == 10 || ReqList.Count == 10 || i==ReqList.Count-1)
{
Task.WaitAll(taskList.ToArray(),-1);
maxTasks = 0;
taskList.Clear();
}
}
The Service2MethodCall is where I am creating a HttpClient to make connections with the 2nd service i.e. Service2,
public class Service2: IService2 {
private ServiceClient GetService2(string resource)
{
return new ServiceClient(_service2BaseUrl, TimeSpan.FromMinutes(60))
{
Resource = resource,
};
}
public async Task Service2MethodCall(ClassC service2Input, string resource)
{
try
{
var client = GetService2(resource);
await client.PostAsync(JsonConvert.SerializeObject(service2Input).ToString());
}
catch (Exception ex)
{
throw new Exception("The Service encountered an error during the Service2 call for Req ID : " +
service2Input.RecsList.Find(x => x.ReqId != "").ReqId.ToString(), ex);
}
}
}
The PostAsync() method creates a new HttpClient with HttpClientHandler object for each of the call.
public async Task PostAsync(object data) {
using(var httpClientHandler = new HttpClientHandler()) {
httpClientHandler.ServerCertificateCustomValidationCallback = (message, cert, chain, sslPolicyErrors) => {
if (sslPolicyErrors == SslPolicyErrors.None) {
return true; //Is valid
}
if (cert.GetCertHashString().ToUpper() == _acceptedThumbprint.ToUpper()) {
return true;
}
return false;
};
using(var client = new HttpClient(httpClientHandler)) {
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(ResponseFormatter.MediaType);
client.Timeout = Timeout.InfiniteTimeSpan;
Uri uri = BuildUri();
if (!String.IsNullOrWhiteSpace(SignatureKey)) {
client.DefaultRequestHeaders.TryAddWithoutValidation("Authorization",
RequestFormatter.GenerateAuthHeaderEncodedUserSignature(uri, AuthUserName, SignatureKey, data));
}
//_logger.WriteUsage(client.BaseAddress.ToString());
HttpResponseMessage response = await RequestFormatter.PostAsync(client, uri, data);
string content = await response.Content.ReadAsStringAsync();
try {
response.EnsureSuccessStatusCode();
}#
pragma warning disable CS0168 // The variable 'rex' is declared but never used
catch (HttpRequestException rex)# pragma warning restore CS0168 // The variable 'rex' is declared but never used
{
//_logger.WriteUsage("Response for POST to: {0} did not yield a successful status code. Message: {1}" + uri.ToString() + content);
throw new ApiHttpException(response.StatusCode, content);
}
}
}
}
Is there some thing I can do within the code to avoid this situation once and for all?

Execute Dynamic Entity in Database using Dapper

My user send dynamic entity from client-project so, I have to write methods like this
public Task<TUser> FindByNameAsync(string normalizedUserName, CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
throw new NotImplementedException();
//string sql = "SELECT * FROM \"IdentityUsers\" WHERE \"NormalizedUserName\" = #NormalizedUserName;";
//using (var connection = _databaseConnectionFactory.CreateConnectionAsync())
//{
// connection.QueryFirstOrDefaultAsync<TUser>(sql,
// new { NormalizedUserName = normalizedUserName });
//}
}
My IDatabaseConnectionFactory class bind ConnectionString like below:
public interface IDatabaseConnectionFactory
{
Task<IDbConnection> CreateConnectionAsync();
}
public class ConnectionFactory : IDatabaseConnectionFactory
{
private readonly string _connectionString;
public ConnectionFactory(string connectionString) => _connectionString = connectionString ?? throw new ArgumentNullException(nameof(connectionString));
public async Task<IDbConnection> CreateConnectionAsync()
{
try
{
var connString = new NpgsqlConnection(_connectionString);
await connString.OpenAsync();
return connString;
}
catch
{
throw;
}
}
}
Now, how can I execute following query using generic-type entity TUser
string sql = "SELECT * FROM \"IdentityUsers\" WHERE \"NormalizedUserName\" = #NormalizedUserName;";
using (var connection = _databaseConnectionFactory.CreateConnectionAsync())
{
connection.QueryFirstOrDefaultAsync<TUser>(sql,
new { NormalizedUserName = normalizedUserName });
}
Note: QueryFirstOrDefaultAsync not found under connection here
You aren't awaiting the CreateConnectionAsync. Unfortunately it isn't obvious in this case, because Task<T> is disposable (so the using doesn't complain); try instead:
using (var connection = await _databaseConnectionFactory.CreateConnectionAsync())
{
var user = await connection.QueryFirstOrDefaultAsync<TUser>(sql,
new { NormalizedUserName = normalizedUserName });
}
As a tip: the compiler output (against the original code) helps make this clear:
Error CS1929 'Task<IDbConnection>' does not contain a definition for 'QueryFirstOrDefaultAsync' and the best extension method overload 'SqlMapper.QueryFirstOrDefaultAsync<TUser>(IDbConnection, string, object, IDbTransaction, int?, CommandType?)' requires a receiver of type 'IDbConnection'
which tells us that:
it found some QueryFirstOrDefaultAsync method, but it wasn't usable, because
the target expression is a Task<IDbConnection>, not an IDbConnection
As a side note: it is worth knowing that if you're only doing one operation with the connection, Dapper can deal with opening and closing the connection for you - which can help reduce the number of async/await operations. Consider, for example, if you had a CreateClosedConnection() method that did not open the connection, and thus had no need to be async; the following would still work:
using (var connection = _databaseConnectionFactory.CreateClosedConnection())
{
var user = await connection.QueryFirstOrDefaultAsync<TUser>(sql,
new { NormalizedUserName = normalizedUserName });
}
with Dapper dealing with the await OpenAsync() for you as part of the QueryFirstOrDefaultAsync.

ASP.NET Core RC-1 file upload

I am currently uploading a file via the kendo fileuploader to an api controller using ASP.NET core RC-1. I am receiving a periodic error of "object reference not set to instance of object" when attempting to read the stream following opening the stream with IFormFile.OpenReadStream().
My controller is:
[HttpPost]
[Route("api/{domain}/[controller]")]
public async Task<IActionResult> Post([FromRoute]string domain, [FromForm]IFormFile file, [FromForm]WebDocument document)
{
if (ModelState.IsValid)
{
if (file.Length > 0)
{
var userName =
Request.HttpContext.User.Claims
.FirstOrDefault(c => c.Type == ClaimTypesEx.FullName)?
.Value;
var uploadedFileName =
ContentDispositionHeaderValue.Parse(file.ContentDisposition).FileName.Trim('"');
document.Domain = domain;
document.MimeType = file.ContentType;
document.SizeInBytes = file.Length;
document.ChangedBy = userName;
document.FileName = (string.IsNullOrEmpty(document.FileName)) ? uploadedFileName : document.FileName;
try
{
document = await CommandStack.For<WebDocument>()
.AddOrUpdateAsync(document, file.OpenReadStream()).ConfigureAwait(false);
}
catch (Exception e)
{
return new HttpStatusCodeResult(500);
}
return Ok(document);
}
}
return new BadRequestResult();
}
And the error is being thrown when I actually try to read the stream when it is going into blob storage:
public async Task<Uri> CreateOrUpdateBlobAsync(string containerName, string fileName, string mimeType,
Stream fileStream)
{
var container = Client.GetContainerReference(containerName);
var blob = container.GetBlockBlobReference(fileName);
//Error HERE
await blob.UploadFromStreamAsync(fileStream);
blob.Properties.ContentType = mimeType;
await blob.SetPropertiesAsync();
return blob.Uri;
}
What I am having trouble with is this is sporadic and there seems to be no defined pattern of which files are accepted and which ones generate the error. At first I thought it might be a size issue but that is not the case as I have several larger files uploaded successfully and then one small file will throw the error. Images seem to work fine and it is hit or miss on other file types with no rhyme or reason that I can figure out.

Async calls in WP7

I have been experimenting with WP7 apps today and have hit a bit of a wall.
I like to have seperation between the UI and the main app code but Ive hit a wall.
I have succesfully implemented a webclient request and gotten a result, but because the call is async I dont know how to pass this backup to the UI level. I cannot seem to hack in a wait for response to complete or anything.
I must be doing something wrong.
(this is the xbox360Voice library that I have for download on my website: http://www.jamesstuddart.co.uk/Projects/ASP.Net/Xbox_Feeds/ which I am porting to WP7 as a test)
here is the backend code snippet:
internal const string BaseUrlFormat = "http://www.360voice.com/api/gamertag-profile.asp?tag={0}";
internal static string ResponseXml { get; set; }
internal static WebClient Client = new WebClient();
public static XboxGamer? GetGamer(string gamerTag)
{
var url = string.Format(BaseUrlFormat, gamerTag);
var response = GetResponse(url, null, null);
return SerializeResponse(response);
}
internal static XboxGamer? SerializeResponse(string response)
{
if (string.IsNullOrEmpty(response))
{
return null;
}
var tempGamer = new XboxGamer();
var gamer = (XboxGamer)SerializationMethods.Deserialize(tempGamer, response);
return gamer;
}
internal static string GetResponse(string url, string userName, string password)
{
if (!string.IsNullOrEmpty(userName) && !string.IsNullOrEmpty(password))
{
Client.Credentials = new NetworkCredential(userName, password);
}
try
{
Client.DownloadStringCompleted += ClientDownloadStringCompleted;
Client.DownloadStringAsync(new Uri(url));
return ResponseXml;
}
catch (Exception ex)
{
return null;
}
}
internal static void ClientDownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
{
if (e.Error == null)
{
ResponseXml = e.Result;
}
}
and this is the front end code:
public void GetGamerDetails()
{
var xboxManager = XboxFactory.GetXboxManager("DarkV1p3r");
var xboxGamer = xboxManager.GetGamer();
if (xboxGamer.HasValue)
{
var profile = xboxGamer.Value.Profile[0];
imgAvatar.Source = new BitmapImage(new Uri(profile.ProfilePictureMiniUrl));
txtUserName.Text = profile.GamerTag;
txtGamerScore.Text = int.Parse(profile.GamerScore).ToString("G 0,000");
txtZone.Text = profile.PlayerZone;
}
else
{
txtUserName.Text = "Failed to load data";
}
}
Now I understand I need to place something in ClientDownloadStringCompleted but I am unsure what.
The problem you have is that as soon as an asynchronous operation is introduced in to the code path the entire code path needs to become asynchronous.
Because GetResponse calls DownloadStringAsync it must become asynchronous, it can't return a string, it can only do that on a callback
Because GetGamer calls GetResponse which is now asynchronous it can't return a XboxGamer, it can only do that on a callback
Because GetGamerDetails calls GetGamer which is now asynchronous it can't continue with its code following the call, it can only do that after it has received a call back from GetGamer.
Because GetGamerDetails is now asynchronous anything call it must also acknowledge this behaviour.
.... this continues all the way up to the top of the chain where a user event will have occured.
Here is some air code that knocks some asynchronicity in to the code.
public static void GetGamer(string gamerTag, Action<XboxGamer?> completed)
{
var url = string.Format(BaseUrlFormat, gamerTag);
var response = GetResponse(url, null, null, (response) =>
{
completed(SerializeResponse(response));
});
}
internal static string GetResponse(string url, string userName, string password, Action<string> completed)
{
WebClient client = new WebClient();
if (!string.IsNullOrEmpty(userName) && !string.IsNullOrEmpty(password))
{
client.Credentials = new NetworkCredential(userName, password);
}
try
{
client.DownloadStringCompleted += (s, args) =>
{
// Messy error handling needed here, out of scope
completed(args.Result);
};
client.DownloadStringAsync(new Uri(url));
}
catch
{
completed(null);
}
}
public void GetGamerDetails()
{
var xboxManager = XboxFactory.GetXboxManager("DarkV1p3r");
xboxManager.GetGamer( (xboxGamer) =>
{
// Need to move to the main UI thread.
Dispatcher.BeginInvoke(new Action<XboxGamer?>(DisplayGamerDetails), xboxGamer);
});
}
void DisplayGamerDetails(XboxGamer? xboxGamer)
{
if (xboxGamer.HasValue)
{
var profile = xboxGamer.Value.Profile[0];
imgAvatar.Source = new BitmapImage(new Uri(profile.ProfilePictureMiniUrl));
txtUserName.Text = profile.GamerTag;
txtGamerScore.Text = int.Parse(profile.GamerScore).ToString("G 0,000");
txtZone.Text = profile.PlayerZone;
}
else
{
txtUserName.Text = "Failed to load data";
}
}
As you can see async programming can get realy messy.
You generally have 2 options. Either you expose your backend code as an async API as well, or you need to wait for the call to complete in GetResponse.
Doing it the async way would mean starting the process one place, then return, and have the UI update when data is available. This is generally the preferred way, since calling a blocking method on the UI thread will make your app seem unresponsive as long as the method is running.
I think the "Silverlight Way" would be to use databinding. Your XboxGamer object should implement the INotifyPropertyChanged interface. When you call GetGamer() it returns immediately with an "empty" XboxGamer object (maybe with GamerTag=="Loading..." or something). In your ClientDownloadStringCompleted handler you should deserialize the returned XML and then fire the INotifyPropertyChanged.PropertyChanged event.
If you look at the "Windows Phone Databound Application" project template in the SDK, the ItemViewModel class is implemented this way.
Here is how you can expose asynchronous features to any type on WP7.

Resources