I'm currently working on a project where I have to make a post request to another API, which takes a significant amount of time (~30-60 seconds) to return. When I make the post request from my controller, I usually (90% of the time) get a TaskCancelledException when the request times out. I've tried using NoAsyncTimeout and AsyncTimeout with a large number but it doesn't seem to be working. The exception happens at the PostAsJsonAsync line of code. The code is below:
[HttpPost]
[ValidateAntiForgeryToken]
[NoAsyncTimeout]
public async Task<ActionResult> Create(...)
{
// processing code
HttpClient httpClient = new HttpClient(new HttpClientHandler() { UseDefaultCredentials = true });
HttpResponseMessage response = await httpClient.PostAsJsonAsync(requestUri, data);
var jsonResult = JObject.Parse(await response.Content.ReadAsStringAsync());
// processing result
}
Is there anything I should do to increase the timeout time? Or is there another problem with this post request?
NoAsyncTimeout and AsyncTimeout set timeouts for the request that is serviced by this action. It's probable that the POST to requestUri is the one that is timing out. Try setting HttpClient.Timeout.
Related
I'm buildin a console Web API to communicate with a localhost server, hosting computer games and highscores for them. Every time I run my code, I get this charming error:
fail:
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
An unhandled exception has occurred while executing the request.
System.NotSupportedException: Deserialization of types without a
parameterless constructor, a singular parameterized constructor, or a
parameterized constructor annotated with 'JsonConstructorAttribute' is
not supported. Type 'System.Net.Http.HttpContent'. Path: $ |
LineNumber: 0 | BytePositionInLine: 1.
This is the method I'm using to post to the database. Note that this method is not in the console application. It is in the ASP.NET Core MvC application opening a web browser and listening for HTTP requests (which can come from the console application).
[HttpPost]
public ActionResult CreateHighscore(HttpContent requestContent)
{
string jasonHs = requestContent.ReadAsStringAsync().Result;
HighscoreDto highscoreDto = JsonConvert.DeserializeObject<HighscoreDto>(jasonHs);
var highscore = new Highscore()
{
Player = highscoreDto.Player,
DayAchieved = highscoreDto.DayAchieved,
Score = highscoreDto.Score,
GameId = highscoreDto.GameId
};
context.Highscores.Add(highscore);
context.SaveChanges();
return NoContent();
}
I'm sending POST requests in a pure C# console application, with information gathered from user input, but the result is exactly the same when using Postman for post requests - the above NotSupportedException.
private static void AddHighscore(Highscore highscore)
{
var jasonHighscore = JsonConvert.SerializeObject(highscore);
Uri uri = new Uri($"{httpClient.BaseAddress}highscores");
HttpContent requestContent = new StringContent(jasonHighscore, Encoding.UTF8, "application/json");
var response = httpClient.PostAsync(uri, requestContent);
if (response.IsCompletedSuccessfully)
{
OutputManager.ShowMessageToUser("Highscore Created");
}
else
{
OutputManager.ShowMessageToUser("Something went wrong");
}
}
I'm new to all this HTTP requests stuff, so if you spot some glaring errors in my code, that would be appreciated. Though, the most important question is, what am I missing, and how can I read from the HttpContent object, to be able to create a Highscore object to send to the database?
It seems to be the string jasonHs... line that is the problem, since the app crashed in exactly the same way, when I commented out the rest of the ActionResult method.
Based on your code, we can find that you make a HTTP Post request with a json string data (serialized from a Highscore object) from your console client to Web API backend.
And in your action method, you create an instance of Highscore manually based on received data, so why not make your action accept a Highscore type parameter, like below. Then the model binding system would help bind data to action parameter(s) automatically.
[HttpPost]
public ActionResult CreateHighscore([FromBody]Highscore highscore)
{
//...
I have 2 asp.net MVC web applications , as follow:-
ApplicationA . which is an Asp.net mvc-4 deployed under iis-8.
ApplicationB. which is an Asp.net mvc-5 deployed under iis-8.
now inside my ApplicationA i have the following method,which will call an action method (home/sync) on applicationB , as follow:-
public List<Technology> GetTechnology(int? currentfiltertype)
{
try
{
using (WebClient wc = new WebClient())
{
string url = currentURL + "home/sync?filtertype=" + currentfiltertype;
wc.Headers.Add("Authorization", token);
string json = wc.DownloadString(url);
List<Technology> result = JsonConvert.DeserializeObject<List<Technology>>(json);
return result;
}
}
catch (Exception e){}
}
now i have noted that when the WebClient calls the action method, and the method did not receive a response within around 2 minutes it will raise a timeout exception. But since the home/sync action method on web application B needs around 30 minutes to complete.. so i was searching for a solution to extend the web-client timeout period. so i tried changing my code to use async methods as follow,mainly by replacing wc.DownloadString with wc.DownloadStringTaskAsync as follow:-
public async Task<List<Technology>> GetTechnology(int? currentfiltertype)
{
try
{
using (WebClient wc = new WebClient())
{
string url = currentURL + "home/sync?filtertype=" + currentfiltertype;
wc.Headers.Add("Authorization", token);
string json = await wc.DownloadStringTaskAsync(url);
List<Technology> result = JsonConvert.DeserializeObject<List<Technology>>(json);
return result;
}
}
catch (Exception e) {}
}
and now seems the WebClient will never expired ... i tried calling the action method and the web client keep waiting for a response for more than 20 minutes without raising any timeout exception, then it received the response from web applicationB and everything worked well..
so can anyone advice why changing my code to use async methods as shown in the above code, caused the WebClient to not timeout ?? i can not understand the relation between using async logic and extending the timeout period for the web-client (not sure if the WebClient will ever timeout inside async methods!!)?
can anyone advice why changing my code to use async methods as shown in the above code, caused the WebClient to not timeout ??
The answer is a bit convoluted: WebClient is based on WebRequest, and HttpWebRequest's Timeout property is only honored for synchronous requests.
(noy sure if the WebClient will ever timeout inside async methods!!)?
It does not directly support asynchronous timeouts, but it does support (its own kind of) cancellation, which you can trigger after a timer.
I have been trying to find a way to make this task more efficient. I am consuming a REST based web service and need to update information for over 2500 clients.
I am using fiddler to watch the requests, and I'm also updating a table with an update time when its complete. I'm getting about 1 response per second. Are my expectations to high? I'm not even sure what I would define as 'fast' in this context.
I am handling everything in my controller and have tried running multiple web requests in parallel based on examples around the place but it doesn't seem to make a difference. To be honest I don't understand it well enough and was just trying to get it to build. I suspect it is still waiting for each request to complete before firing again.
I have also increased connections in my web config file as per another suggestion with no success:
<system.net>
<connectionManagement>
<add address="*" maxconnection="20" />
</connectionManagement>
</system.net>
My Controllers action method looks like this:
public async Task<ActionResult> UpdateMattersAsync()
{
//Only get matters we haven't synced yet
List<MatterClientRepair> repairList = Data.Get.AllUnsyncedMatterClientRepairs(true);
//Take the next 500
List<MatterClientRepair> subRepairList = repairList.Take(500).ToList();
FinalisedMatterViewModel vm = new FinalisedMatterViewModel();
using (ApplicationDbContext db = new ApplicationDbContext())
{
int jobCount = 0;
foreach (var job in subRepairList)
{
// If not yet synced - it shouldn't ever be!!
if (!job.Synced)
{
jobCount++;
// set up some Authentication fields
var oauth = new OAuth.Manager();
oauth["access_token"] = Session["AccessToken"].ToString();
string uri = "https://app.com/api/v2/matters/" + job.Matter;
// prepare the json object for the body
MatterClientJob jsonBody = new MatterClientJob();
jsonBody.matter = new MatterForUpload();
jsonBody.matter.client_id = job.NewClient;
string jsonString = jsonBody.ToJSON();
// Send it off. It returns the whole object we updated - we don't actually do anything with it
Matter result = await oauth.Update<Matter>(uri, oauth["access_token"], "PUT", jsonString);
// update our entities
var updateJob = db.MatterClientRepairs.Find(job.ID);
updateJob.Synced = true;
updateJob.Update_Time = DateTime.Now;
db.Entry(updateJob).State = System.Data.Entity.EntityState.Modified;
if (jobCount % 50 == 0)
{
// save every 50 changes
db.SaveChanges();
}
}
}
// if there are remaining files to save
if (jobCount % 50 != 0)
{
db.SaveChanges();
}
return View("FinalisedMatters", Data.Get.AllMatterClientRepairs());
}
}
And of course the Update method itself which handles the Web requesting:
public async Task<T> Update<T>(string uri, string token, string method, string json)
{
var authzHeader = GenerateAuthzHeader(uri, method);
// prepare the token request
var request = (HttpWebRequest)WebRequest.Create(uri);
request.Headers.Add("Authorization", authzHeader);
request.Method = method;
request.ContentType = "application/json";
request.Accept = "application/json, text/javascript";
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(json);
request.ContentLength = bytes.Length;
System.IO.Stream os = request.GetRequestStream();
os.Write(bytes, 0, bytes.Length);
os.Close();
WebResponse response = await request.GetResponseAsync();
using (var reader = new System.IO.StreamReader(response.GetResponseStream()))
{
return JsonConvert.DeserializeObject<T>(reader.ReadToEnd());
}
}
If it's not possible to do more than 1 request per second then I'm interested in looking at an Ajax solution so I can give the user some feedback while it is processing. In my current solution I cannot give the user feedback while the action method hasn't reached 'return' yet can I?
Okay it's taken me a few days (and a LOT of trial and error) but I've worked this out. Hopefully it can help others. I finally found my silver bullet. And it was probably the place I should have started:
MSDN: Consuming the Task-based Asynchronous Pattern
In the end this following line of code is what brought it all to light.
string [] pages = await Task.WhenAll(from url in urls select DownloadStringAsync(url));
I substituted a few things to make it work for a Put request as follows:
HttpResponseMessage[] results = await Task.WhenAll(from p in toUpload select client.PutAsync(p.uri, p.jsonContent));
'toUpload' is a List of MyClass:
public class MyClass
{
// the URI should be relative to the base pase
// (ie: /api/v2/matters/101)
public string uri { get; set; }
// a string in JSON format, being the body of the PUT request
public StringContent jsonContent { get; set; }
}
The key was to stop trying to put my PutAsync method inside a loop. My new line of code IS still blocking until ALL responses have come back, but that is what I wanted. Also, learning that I could use this LINQ style expression to create a Task List on the fly was immeasurably helpful. I won't post all the code (unless someone wants it) because it's not as nicely refactored as the original and I still need to check whether the response of each item was 200 OK before I record it as successfully saved in my database. So how much faster is it?
Results
I tested a sample of 50 web service calls from my local machine. (There is some saving of records to a SQL Database in Azure at the end).
Original Synchronous Code: 70.73 seconds
Asynchronous Code: 8.89 seconds
That's gone from 1.4146 requests per second down to a mind melting 0.1778 requests per second! (if you average it out)
Conclusion
My journey isn't over. I've just scratched the surface of asynchronous programming and am loving it. I need to now work out how to save only the results that have returned 200 OK. I can deserialize the HttpResponse which returns a JSON object (which has a unique ID I can look up etc.) OR I could use the Task.WhenAny method, and experiment with Interleaving.
I want to modify the response body from the token endpoint response.
I've tried to intercept the /Token request with a MessageHandler but it doesn't work.
I'm able to add some additional informations to the response by overriding the OAuthAuthorizationServerProvider.TokenEndpointmethod, but I'm not able to create my own response body.
Is there a way to intercept the /Token request?
Edit
I found out how to remove the response body content from the token endpoint response, like this: HttpContext.Current.Response.SuppressContent = true;
It seems the right way to achieve my goal, but now when I use the context.AdditionalResponseParameters.Add() method to add my custom information, the SuppressContent block any alterations.
Now I have something like this:
// Removing the body from the token endpoint response
HttpContext.Current.Response.SuppressContent = true;
// Add custom informations
context.AdditionalResponseParameters.Add("a", "test");
To simply add new items to the JSON token response, you can use TokenEndpointResponse instead of the TokenEndpoint notification.
If you're looking for a way to completely replace the token response prepared by the OAuth2 authorization server by your own one, there's sadly no easy way to do that because OAuthAuthorizationServerHandler.InvokeTokenEndpointAsync doesn't check the OAuthTokenEndpointContext.IsRequestCompleted property after invoking the TokenEndpointResponse notification.
https://github.com/aspnet/AspNetKatana/blob/dev/src/Microsoft.Owin.Security.OAuth/OAuthAuthorizationServerHandler.cs
This is a known issue, but it was too late to include it in Katana 3 when I suggested to fix it.
You should give Owin.Security.OpenIdConnect.Server a try: it's an a fork of the OAuthAuthorizationServerMiddleware designed for Katana 3.0 and 4.0.
https://www.nuget.org/packages/Owin.Security.OpenIdConnect.Server/1.0.2
Of course, it includes the correct check to allow bypassing the default token request processing (this was even one of the first things I fixed when forking it).
You were almost there +Samoji #Samoji and really helped/inspired me to get the answer.
// Add custom informations
context.AdditionalResponseParameters.Add("a", "test");
// Overwrite the old content
var newToken = context.AccessToken;
context.AdditionalResponseParameters.Add("access_token", newToken);
I found it just replaced my old token with my new.
This question is similar to How to extend IdentityServer4 workflow to run custom code
So you can create custom middleware and register it before OAuth2 service in Startup:
public void Configuration(IAppBuilder app)
{
....
app.Use(ResponseBodyEditorMiddleware.EditResponse);
app.UseOAuthAuthorizationServer(...);
...
}
where custom middleware is:
public static async Task EditResponse(IOwinContext context, Func<Task> next)
{
// get the original body
var body = context.Response.Body;
// replace the original body with a memory stream
var buffer = new MemoryStream();
context.Response.Body = buffer;
// invoke the next middleware from the pipeline
await next.Invoke();
// get a body as string
var bodyString = Encoding.UTF8.GetString(buffer.GetBuffer());
// make some changes to the body
bodyString = $"The body has been replaced!{Environment.NewLine}Original body:{Environment.NewLine}{bodyString}";
// update the memory stream
var bytes = Encoding.UTF8.GetBytes(bodyString);
buffer.SetLength(0);
buffer.Write(bytes, 0, bytes.Length);
// replace the memory stream with updated body
buffer.Position = 0;
await buffer.CopyToAsync(body);
context.Response.Body = body;
}
The best way to intercept request and response is via MessageHandler if you want to avoid doing so after a request has reached the IControllerFactory handler in the pipeline - obviously in that case use a custom 'Attribute'
I have used MessageHandlers in the past to intercept request to api/token, create a new request and get the response, create a new response.
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request, CancellationToken cancellationToken)
{
//create a new auth request
var authrequest = new HttpRequestMessage();
authrequest.RequestUri = new Uri(string.Format("{0}{1}", customBaseUriFromConfig, yourApiTokenPathFromConfig));
//copy headers from the request into the new authrequest
foreach(var header in request.Headers)
{
authrequest.Headers.Add(header.Key, header.Value);
}
//add authorization header for your SPA application's client and secret verification
//this to avoid adding client id and secret in your SPA
var authorizationHeader =
Convert.ToBase64String(Encoding.UTF8.GetBytes(string.Format("{0}:{1}", _clientIdFromConfig, _secretKeyFromConfig)));
//copy content from original request
authrequest.Content = request.Content;
//add the authorization header to the client for api token
var client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(request.Headers.Authorization.Scheme, authorizationHeader);
var response = await client.PostAsync(authrequest.RequestUri, authrequest.Content, cancellationToken);
if(response.StatusCode == HttpStatusCode.OK)
{
response.Headers.Add("MyCustomHeader", "Value");
//modify other attributes on the response
}
return response;
}
This works for me perfectly. There is, however, the configuration for this handler required in the WebApiConfig.cs file (RouteConfig.cs if you're using ASP.NET MVC).
Can you elaborate on what it is that does not work for you on the handler?
I have a web service (made in Asp.net Web API) that returns an xml file of about 10MB size.
The service has been tested with Fiddler and it is working
I am trying to download the file using HttpClient class. The problem is that the compilator never gets outside the await client.GetAsync() method, even if the API project returned the HttpResponseMessage.
This is my function
public async Task<XDocument> DownloadXmlAsync(string xmlFileName)
{
using (HttpClient client = new HttpClient())
{
client.BaseAddress = new Uri("http://localhost:51734/");
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("text/xml"));
// When the copiler enters this next command, it doesn't get outside anymore
HttpResponseMessage response = await client.GetAsync("api/applications/ApplicationXml?fileName=" + xmlFileName);
response.EnsureSuccessStatusCode();
string stringResponse = await response.Content.ReadAsStringAsync();
XDocument xDoc = new XDocument(stringResponse);
return xDoc;
}
}
I updated also the maxRequestLength in web.config
<httpRuntime maxRequestLength="15360" />
What i am doing wrong?
Edit
Calling the function
public async Task<ActionResult> Index()
{
var xmlTask = DownloadXmlAsync("1.xml");
// doesn't reach here
var result = xmlTask.Result;
return View();
}
You're causing a classic deadlock by calling Result. Instead, you should await the task:
public async Task<ActionResult> Index()
{
var xmlTask = DownloadXmlAsync("1.xml");
// doesn't reach here
var result = await xmlTask;
return View();
}
I explain this deadlock in full on my blog, but the general idea is like this:
ASP.NET only allows one thread to be processing a request at a time.
When you await a Task, the compiler will capture a "context" and use it to resume the method when the Task completes. In the case of ASP.NET, this "context" is a request context.
So when DownloadXmlAsync (asynchronously) waits for GetAsync to complete, it returns an incomplete task to Index.
Index synchronously blocks on that task. This means the request thread is blocked until that task completes.
When the file is received, GetAsync completes. However, DownloadXmlAsync cannot continue because it's trying to resume that "context", and the "context" already has a thread in it: the one blocked on the task.
Hence, deadlock.