I have a weird behavior going on with SignalR, I have a hub defined like so:
public class NotificationHub : Hub
{
//Empty
}
on my js I have the following:
function blabla {
// bla bla
$.connection.NotificationHub.client.AppendNewStory = function (story) {
// appends a new story, long jquery code
};
$.connection.hub.start().done(function () {
_ConnectionStarted = true; // there must be some built in way
});
}
I call js from a class on my mvc project
public SomeClass
{
private IHubContext _context;
public SomeClass()
{
_context = GlobalHost.ConnectionManager.GetHubContext<NotificationHub>();
}
public void Notify(SomeData ...)
{
_context.Clients.User(userId).AppendNewStory(data);
}
}
Problem has two symptoms:
Server to client class fail intermittently (can't figure out a scenario).
My web server seems to get VERY SLOW, subsequent ajax calls and regular webrequests timeout.
Using fiddler I found the following:
SignalR/hubs call succeeds.
SignalR negotial call succeeds.
connect with WebSockets transport failed with HTTP 504!
subsequent connect calls try foreverFrame and long polling, both succeed with 200.
a poll request is issued and NEVER RETURNS! after that everything becomes slow or hangs.
Something that might aid the debugging, server hangs if i'm opening 2 tabs or more. a single tab seems ok.
I'm also using IE 11 and chrome, same problem except chrome tries serverSentEvents instead of foreverFrame.
Any ideas on how to debug this issue, see why I get 504? or why the server hangs? I have Windows 8.1 Update 1, IIS 8, SignalR 2.0, .NET 4.5.1.
Did you install WebSockets in Windows 8.1? Do this under add/remove features. I had similar random issues connecting until I did this. I also had to enable the following setting in the web.config:
<add key="aspnet:UseTaskFriendlySynchronizationContext" value="true"/>
Is there anything in between your server and browser. Some proxies have trouble with web sockets and server sent events. Try choosing the transports in the javascript. I saw similar things with Firefox over OpenIG. OpenIG didn't like server sent events and just hung.
One possibility that you'll want to investigate is whether you're running into a limit on the number of connections allowed to IIS. I'm not sure whether websocket connections are included in that limit, but normal connections - and a long polling connection will consume one of those - are typically limited to "3" on a the "Basic" version of Windows 8 (and are limited to 10 on the Professional and other versions). I'm guessing that your long polling connection(s) are consuming all the available connections, and that's what is causing everything else to time out.
See here for more details:
http://www.jpelectron.com/sample/WWW%20and%20HTML/IIS-%20OS%20Version%20Limits.htm
I'd either upgrade to Professional (if you're already at "Professional", then discount this answer), or install Websockets and see if that helps things.
Ok thanx to everyone who answered, you all helped debug the issue, unfortunately nothing worked.
problem turned out I needed to set the targetFramework attribute in web.config:
<httpRuntime targetFramework="4.5.1">
Everything worked smoothly afterwards.
Related
My function invocations keep failing with the error message An existing connection was forcibly closed by the remote host. According to this to mitigate this problem I am to use static Http Clients, which I am already doing, but still the error keeps occurring.
However, I do have 4 of those static clients, which I guess is more than strictly 1. But still I would have thought to fare better with 4 static ones than with multiple (non-static) instances.
I would like to really have only 1 client but I do not know how to make it work. There is one function app which as the below helper class defined in it. In Addition there are 3 services which have been extracted to separate class libraries (because they are used in different projects also) I include in my function app and use via DI. The function app and all of the 3 libs include the below helper class.
public static class Http
{
private static HttpClient _client = new HttpClient();
public static async Task<HttpResponseMessage> GetAsync(string url, Dictionary<string, string>? headers = null)
{
var req = new HttpRequestMessage(HttpMethod.Get, url);
if (headers != null) req._addRequestHeaders(headers);
var response = await _client.SendAsync(req);
response.EnsureSuccessStatusCode();
return response;
}
}
The services in the libs use HTTP to do their own stuff. How would I be able to have the app and all of the libs it references to use the same static http client?
Or do you think the multiple static http classes are not the cause of the problem? There is no such error when running the function app locally in vs code (i.e outside of Azure).
Cheers
Or do you think the multiple static http classes are not the cause of the problem?
This. The error code you're seeing is WSAECONNRESET (10054):
Connection reset by peer.
An existing connection was forcibly closed by the remote host. This normally results if the peer application on the remote host is suddenly stopped, the host is rebooted, the host or remote network interface is disabled, or the remote host uses a hard close (see setsockopt for more information on the SO_LINGER option on the remote socket). This error may also result if a connection was broken due to keep-alive activity detecting a failure while one or more operations are in progress. Operations that were in progress fail with WSAENETRESET. Subsequent operations fail with WSAECONNRESET.
The fact that you're seeing an exception for this indicates it's an unexpected reset (there are other times when WSAECONNRESET is normal, and HttpClient handles those for you).
There's a lot of possible reasons for unexpected resets. If it's always happening, it could be a security (TLS) setting mismatch, or a malformed request. If it's sometimes happening, it could just be another service restarting/crashing at just the wrong time. Which happens in the cloud, and your service should retry.
There is no such error when running the function app locally in vs code (i.e outside of Azure).
The next thing I'd try is to turn the logging up to 11.
We are running an Owin applications on IIS 8.5 on Win 2012 R2 behind a loadbalancer. One some occations, requests to certain URLs hang indefinitely. If the user selects cancel in the browser, and reloads the page, everything is OK and the server responds quickly.
In IIS manager, we can see the requests hanging:
The hang seems to occur inside the Owin pipeline. We are running .NET 4.5.2 and the latest Owin packages.
Here's the code for the /api/whoami endpoint:
[Authorize]
public class WhoAmIController : ApiController
{
[Route("whoami")]
[HttpGet]
public IHttpActionResult Whoami()
{
return Json(ClaimsPrincipal.Current.Identity.Name);
}
}
An observation is that requests to this endpoint hangs in the AuthenticateRequest stage in the IIS pipeline, not PreExecuteRequestHandler which is the default IIS stage for the OWIN pipeline. Our only authentication method is cookie authentication, which executes in the AuthenticateRequest stage.
Any ideas?
Edit: adjusted screenshot
The problem was that we had code executing on the IIS event PreSendRequestHeaders, which apperently is bad according to
http://www.asp.net/aspnet/overview/web-development-best-practices/what-not-to-do-in-aspnet,-and-what-to-do-instead#presend
Our intention was to adjust HTTP headers on the way out on all request. The fix was to move the code to the BeginRequest event.
Have you tried adding .ConfigureAwait(false) to your async calls?
Doing so will make the continuation continue on the "correct" thread.
I keep getting intermittent No transport could be initialized successfully. error in my OWIN self hosted application with SignalR and I haven't been able to find a problem yet, all transport types just fail silently during initialization sometimes.
I'm trying to enable logging and tracing on server side, but can't find how I can do it.
I've seen this article http://www.asp.net/signalr/overview/testing-and-debugging/enabling-signalr-tracing
but I don't think I can do it with OWIN
How can I set up SignalR server side logging? I'm interested in iniitialization tracing - my client and server side method calls work fine.
This isn't ideal, but I'm at least getting some trace after installing Microsoft.Owin.Diagnostics nuget package and then:
public void Configuration(IAppBuilder app)
{
app.UseCors(CorsOptions.AllowAll);
app.Properties["host.AppMode"] = "development";
app.UseErrorPage(new Microsoft.Owin.Diagnostics.ErrorPageOptions { ShowExceptionDetails = true });
app.MapSignalR();
}
This returns HTML to the client with a stack trace and some other useful info. Hope it helps.
I'm using SignalR to push updates out to connected web clients. I listen to the disconnected event in order to know when I should start my reconnection logic
$.connection.hub.disconnected(function() {
// Initiate my own reconnection logic
});
The SignalR hub is hosted in in IIS (along with my site)
[assembly: OwinStartup(typeof(Startup))]
namespace MyNamespace.SignalR
{
public class Startup
{
public void Configuration(IAppBuilder app)
{
app.MapSignalR();
}
}
}
Upon connection, the client calls a server method to join a group
public class MyHub : Hub
{
public void JoinGroup(string groupName)
{
Groups.Add(Context.ConnectionId, groupName);
}
}
And then I push messages to this group:
context.Clients.Group(groupName).sendMessage();
If I manually recycle the application pool in IIS, SignalR starts trying to reconnect and I eventually receive a disconnected event on the client side if it fails (after the timeout).
However, my problem is that if I manually restart the website in IIS, I do not receive any disconnected event at all, and I can't see in the logs that SignalR has detected any connection problem at all. How can I detect that I have been disconnected?
I know I should probably persist the group connections somehow, since that is saved in memory I guess. But that shouldn't affect the initial problem that the client receives no notification of the disconnection? Shouldn't the client side signalr code throw some kind of exception/event?
disconnected fires first when the built in logic for reconnection have timed out. You also need to listen to the recconect event, something like i did here
https://github.com/AndersMalmgren/SignalR.EventAggregatorProxy/blob/ReconnectOnClosed/SignalR.EventAggregatorProxy.Client.JS/jquery.signalR.eventAggregator.js#L157
So I finally found out what the problem is and how to solve it. Some background though:
At the moment we manually release new versions of our application by going to "Basic settings..." under the website in IIS and changing the "Physical Path" from C:\websites\version1 to C:\websites\version2. Apparently this gives the same behavior as doing a restart of the website in IIS (not a hard reset, not stopping the website, not recycling the app pool) and according to this: "does NOT shut the site down, it merely removes the Http.sys binding for that port". And no matter how long we wait, the connected clients never receive any kind of indication that they should reconnect.
So the solution is to recycle the application pool after each release. The "lost" clients will receive disconnected events and reconnect to the new version of the site.
I have run into an issue which can be replicated in the following way (you need IIS8 so must be on Windows 8+ or Windows Server 2012 R2+):
Create a new website in IIS Manager, say TestWs on port 8881, pointing to a new folder, say C:\temp\testws, and add the following Web.config file in there
<?xml version="1.0"?>
<configuration>
<system.web>
<compilation targetFramework="4.5"/>
<httpRuntime targetFramework="4.5"/>
</system.web>
</configuration>
Now add the following WsHandler.ashx file in the same folder
<%# WebHandler Language="C#" Class="WsHandler" %>
using System;
using System.Threading;
using System.Web;
public class WsHandler : IHttpHandler
{
public void ProcessRequest(HttpContext context)
{
context.AcceptWebSocketRequest(async webSocketContext =>
{
while (true)
{
await webSocketContext.WebSocket.ReceiveAsync(new ArraySegment<byte>(new byte[1024]), CancellationToken.None);
}
});
}
public bool IsReusable { get { return true; } }
}
Then create a websocket from within the developer toolbar in your browser like so
var ws = new WebSocket("ws://localhost:8881/wshandler.ashx");
ws.onclose = function() { console.log('closed'); };
In task manager you will see there is a w3wp.exe process for this application, if you kill it the client get the onclose event raised and the closed text will be printed.
However if you create a websocket as described above and go to IIS manager and recycle the application pool, the websocket will not be closed, and there will now be two w3wp.exe processes.
Closing the web socket ws.close(); or refreshing the browser will cause the original w3wp.exe process to be shut down.
It seems the presence of the open websocket is causing IIS to be unable to recycle the app pool correctly.
Can anyone figure out what to change in my code or what to change in IIS to get this to work?
As far as I know, while a WebSocket is open, IIS won't tear down the app domain, so you see this behaviour exhibited.
The best I can suggest is that you do some cross process signalling to force the old instance to shutdown. You could achieve this with an EventWaitHandle:
Create a named EventWaitHandle in your web application, and signal it at startup.
On a separate thread, wait on the wait handle
When it is signalled, call HostingEnvironment.InitiateShutdown to force any running old instance to shutdown.
Try setting "Shutdown Time Limit" to 1 second (App Pool > Advanced Settings > Process Model) [PS: I don't have IIS8. I'm checking the properties in IIS7.]
This property defines the time given to worker process to finish processing requests and shutdown. If the worker process exceeds the shutdown time limit, it is terminated.
I can see default value in IIS7 is 90 seconds. If that's the same value in IIS8 too, then it might be giving that much time to earlier worker process to finish it's work. After 90 seconds (1.5 mins) it will terminate that process and your web socket will get closed. If you change it to 1 second it will terminate the earlier worker process will get terminated almost instantly (as soon as you recycle app pool) and you will get the expected behavior.
As I had the same problem, here's the solution I figured out:
In your IHttpHandler you should have an static object which inherits IStopListeningRegisteredObject. Then use HostingEnvironment.RegisterObject(this) to get notified (via StopListening) when the Application Pool is recyled.
You'll also need a CancellationTokenSource (static, too), which you'll hand over in ReceiveAsync. In StopListening() you can then use Cancel() of the CancellationTokenSource to end the await. Then catch the OperationCanceledException and call Abort() on the socket.
Don't forget the Dispose() after the Cancel() or the App-Pool will still wait.