We've been experiencing some deadlocks when working with interconnected ASP.NET WebApis on the same IIS server. We'd like to know if this is somehow an expected behavior, due to hosting all APIs on the same server and same Application Pool, since we have managed to avoid the issue by moving either WebApi to a different pool; or if there's something wrong with our code.
For production, we will probably host the APIs on different server or pools, but still we'd like to understand why this is happening. Our main concern is that, if it's our faulty code, the issue may be reproduced on a larger scale, even if the hosting setup is correct.
We have created a little solution to reproduce the deadlock, hosted in GitHub.
The reproduction steps are as follow:
WebClient executes multiple HTTP request in parallel WebApi1.
WebApi1 executes an HTTP request to WebApi2.
WebApi2 executes an HTTP request to WebApi3.
WebApi3 simply returns a string.
The expected behavior would be that all requests are eventually resolved.
The actual behavior is that, certain requests gets completed, while some others will fail, due to a TaskCancelledException which seems to be due to the requests timing out.
The only article that I was able to find that seems to mention the same issue is from 2014: "Do Not Send ServerXMLHTTP or WinHTTP Requests to the Same Server", I believe that this is the issue we are experiencing, how can we confirm this?
Context
We've been assigned the task to create a centralized authentication server for multiple internal APIs of the company we work at.
We are using IdentityServer3 with reference tokens, so when some API requests a second API using reference tokens, the second API will request the authentication server for token validation which reproduces the issue.
I have added the IdentityServer tag, since this could be a common issue when doing multiple APIs communication and using reference tokens. Sample on GitHub.
Just one observation: you are using HttpClient as a static member for every Controller and according to this HttpClient is not guaranteed to be thread-safe
Related
In IIS, I have an ASP.net web application that makes many behind-the-scenes calls to an IIS-hosted WCF service.
I never had any issues during development. However, since deploying to a few test servers at work, our tester was reporting that some of the calls were seemingly randomly failing. I verified that indeed, some of those behind the scenes webapp->service calls were failing with a "(417) Expectation Failed". Hitting the back-button and attempting the action again always worked.
A common solution I kept seeing online was https://stackoverflow.com/a/7358457/1669011
After adding that to my web config, the error continued to happen, but instead of returning a 417, would end up returning the results of my web.config's applicationInitialization remapManagedRequestsTo page.
I feel like the web.config fix I mentioned above has just allowed my web application to accept the full body of the response rather than failing when it realised it wasn't the wcf response it was expecting.
So if random services in WCF are returning the results of remapped requests that occur during Application Initialization, does that indicate that my WCF services in IIS are constantly totally shutting down?
What might cause my WCF service to randomly be in a state of "application initialization"? I'm hoping this is just a server issue and nothing to do with my application, as this has come out of nowhere and is risking an upcoming deploy.
Thanks for any assistance
My guess would be time (or very serious exceptions).
Specifically, if the server hosting the WCF service is not being hit often enough, it will shut down the application. Follow the instructions here to ensure the WCF site is not being unloaded: How to keep a WCF site online?.
If it's exceptions, you have not provided enough information to diagnose the issue.
There's one point in ASP.NET Core that I believe I didn't fully understand yet and that is the idea of request features. As explained in the docs:
Feature interfaces define specific HTTP features that a given request may support. Servers define collections of features, and the initial set of features supported by that server, but middleware can be used to enhance these features.
My initial understanding about this was that request features are all things a server should expose to be used on the application pipeline. That is, behaviors that a server should perform like sending a file.
On the other hand, there's, for example, the authentication request feature. Now, I'm not sure authentication falls into this category. It doesn't seem like some server behavior that the application should call, but rather, a concern of the application itself.
This makes me wonder what really makes something be a request feature. So, what makes something be a request feature in ASP.NET Core? Is my initial understanding wrong? What is behind the decision of making something a request feature?
My initial understanding about this was that request features are all things a server should expose to be used on the application pipeline. That is, behaviors that a server should perform like sending a file.
That's one use of http features. It's also a way to augment or light up behaviors on the HttpContext, like buffering, send file, authentication, websockets.
Middleware can also add features specific to that middleware, you can see examples of this:
The exception handler middleware flows the exception that occurred via a request feature - https://github.com/aspnet/Diagnostics/blob/dev/src/Microsoft.AspNetCore.Diagnostics.Abstractions/IExceptionHandlerFeature.cs.
The routing middleware adds route data to the current http context via a request feature - https://github.com/aspnet/Routing/blob/dev/src/Microsoft.AspNetCore.Routing.Abstractions/IRoutingFeature.cs
Generally it's a way to flow per request behavior and state from the server, through middleware, to the application.
I'm receiving an error when I'm attempting to consume a web service:
Cannot read the token from the 'Timestamp' element with the 'http://docs.oasis- open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd' namespace for BinarySecretSecurityToken, with a '' ValueType.
Not quite sure
The client is an asp.net web application, making a call. From Wireshark, one can see the post going in, and the response coming back, but then it errors out like this.
To give some background, this is a WCF calling on a java served web service.
You may need to add a security timestamp soap header to the message. Look at this SO question where they had the opposite problem but it may be helpful to look at their configuration. Also, you may save yourself some grief if you can use one of the WCF Interop Express bindings for accessing a java service implementing WS-Security.
I'm looking into building an ASP.NET MVC application that exposes (other than the usual HTML pages) JSON and XML REST services, as well as Web Sockets.
In a perfect world, I would be able to use the same URLs for the Web Sockets interface as I do for the other services (and determine which data to return by what the user agent requests) but, knowing that IIS wasn't built for persistent connections, I need to know if there's a way that I can accept (and possibly even handshake) the Web Sockets connection and then pass the connection off to another service running on the server.
I do have a workaround in mind if this isn't possible that basically involves using ASP.NET to check for the Web Sockets connection upgrade headers, and responding with a HTTP/1.1 302 Found that points to a different host that has my Web Sockets service configured to directly listen to the appopriate endpoint(s).
If I completely understand your goal, I believe you can use the IIS7/7.5 Application Request Routing module to accomplish this.
Here's a quick reference: http://learn.iis.net/page.aspx/489/using-the-application-request-routing-module/
Rather than 302 responses you could use ISAPI_rewrite to direct to an appropriate endpoint (and manipulate the HTTP header to get it there)
http://www.isapirewrite.com/docs/
Otherwise no, IIS cannot natively pass off an HTTP connection. The current MSFT method is to use a 302 or something else that is intercepting the raw socket and performing header manipulation prior to sending to IIS (or whatever other application)
It strikes me that this would be a better question to ask Microsoft than to ask us. Web Sockets is new technology, and rather than looking for a hack, you might want to ask Microsoft how they plan to support it. IIS is their software. Poke around on http://iis.net (maybe in http://forums.iis.net) and see what you learn.
The way to do this is to use a unique Session ID that is associated with the Http Session. From the description, it seems like you might want to scope this to a single HttpApplication instance, but this is not necessary (you may also persist a session across many application instances). Anyway, this session ID needs to be attached somehow to each Http Request (either with a cookie, querystring, static variable with the HttpApplication instance, form data). Then you persist the identifying information about the Http session somewhere with the ID.
This identifying information may vary depending on your needs but could entail the entire http request or just some stripped down representation that serves your particular purpose.
Using this SessionID somewhere in the Http request allows you to restore whatever information you need to call and interact with the appropriate services. The instances of the services may also need to be scoped to the session as well.
Basically, what I am suggesting is that you NOT directly pass the Http connection to an external process, but instead pass the necessary data to the external process, and allow create a mechanism for sending callback data. I think looking into the mediator pattern may be helpful for you in understanding what I mean here. http://en.wikipedia.org/wiki/Mediator_pattern . I hope this helps.
I have a .NET 2.0 ClickOnce application that runs in a corporate environment and it's setup to work via Windows Authentication.
So the users download the app via ClickOnce and then it connects to an IIS server and communicates via Web Service calls.
The problem is that for every single call there is a double handshake. The first call yields a 401 and invites the client to negotiate. The second call succeeds since the client sends the authentication token.
Is there anyway to eliminate this double handshake? Because it put a tremendous overhead on the latency of the application and makes it seem very sluggish.
Thanks
There actually is a solution, it turns out: WebRequest.PreAuthenticate
Check out an article by Rick Strahl.
This "double handshake" is a integral part of NTLM's challenge response model. Your only option would be to change the authentication type.
Alternatively, you try re-using the connection for web service calls.