I create an application using WebRTC data channels (ordered, reliable) and socket.io WebSockets as signalling server.
Assumptions (please confirm / neglect them)
WebRTC SCTP & socket.io send heartbeats, but their interval may be different
if a crash is detected using heartbeats, the connections (DataChannel and WebSockets) will be CLOSED and can NOT be reused if the peer was just temporarily disconnected
--> the temporarily crashed peer MUST reconnect
Is it possible to configure the WebRTC heartbeat interval / timeout or is there any default value used in Chrome and Firefox, that I can rely on?
(related to this question:
https://softwareengineering.stackexchange.com/questions/255057/p2p-locking-using-webrtc-and-socket-io-for-streaming-incremental-game-state
)
Related
While using a MQTT server a question popped up in my head, would it be better to use Protobuf+Websockets in my case?
I am making an instant messaging platform... clientA to server to database and clientB.
information that will be sent: conversation ID and message.
When you include the size of the HTTP header in the initial GET request and then the UPGRADE request to the server to start a Websocket connection the total network traffic is normally orders of magnitude more that of opening a MQTT connection and subscribing to a topic.
Sending data over the network uses the radio, which burns through battery, so the more data you send the more battery you burn. With MQTT more of the data sent and received is your actual payload and not the extra associated with the protocol.
While the difference caused by the initial handshake will even out for longer more stable connections as more data is sent/received, this is not always the case for mobile applications which tend to drop network connections more frequently as they move from 4G to 3G when moving between cell sites or switching between mobile data and WiFi.
Websockets are way better than HTTP long polling, but there is still that large initial network hit on the start of a connection.
MQTT with compression / protobuf => best
MQTT => good
Websockets over HTTP with compression / protobuf => ok
Websockets over HTTP => so so
Only use Websockets if using something like JavaScript on the client side or if you want to be going over HTTP (e.g. to traverse firewalls more easily).
I'm creating an IoT Device + Server system using .NET Micro Framework and ASP.NET WebAPI (Probably in Azure).
The IoT device needs to be able to frequently update the server with stats whilst also being able to receive occasional incoming commands from the server that would change its behaviour. In this sense, the device needs to act as both client and server itself.
My concern is getting the best balance between the security of the device and the load on the server. Furthermore, there must be a relatively low amount of latency between the server needing to issue a command and the device carrying it out; of the order of a few seconds.
As I see it my options are:
Upon connection to the internet, the device establishes a persistent TCP connection to the server which is then used for both polling and receiving commands.
The device listens on a port (e.g. HttpListener) for incoming commands whilst updating the server via frequent HTTP requests.
The device only ever polls the server with HTTP requests. The server uses the response to give the device commands.
The 2nd option seems to be the least secure as the device would have open incoming ports. The 1st option looks the most difficult to reliably implement as it would require low level socket programming. The 3rd option seems easy and secure but due to the latency requirements the device would need to poll every few seconds. This impacts the scalability of the system.
So at what frequency does HTTP polling create more overhead than just constantly keeping a TCP connection open? 5s? 3s? 1s? Or am I overstating the overhead of keeping a TCP connection open in ASP.NET? Or is there a completely different way that this can be implemented?
Thanks.
So at what frequency does HTTP polling create more overhead than just constantly keeping a TCP connection open? 5s? 3s? 1s?
There is nothing to do to keep a TCP connection open. The only thing you might need to do is to use TCP keep-alive (which have nothing to do with HTTP keep-alive!) in case you want to keep the connection idle (i.e no data to send) for a long time.
with HTTP your overhead already starts with the first request, since your data need to be encapsulated into a HTTP message. This overhead can be comparable small if the message is large or it can easily be much larger than the message itself for small messages. Also, HTTP server close the TCP connection after some idle time so you might need to re-establish the TCP connection for the next data exchange which is again overhead and latency.
HTTP has the advantage to pass through most firewalls and proxies, while plain TCP does not. You also get encryption kind of free with HTTPS, i.e. there are established standards for direct encrypted connection and for tunneling through a proxy.
WebSockets is something in between: you do a HTTP request and then upgrade HTTP to WebSocket. The initial overhead is thus as large as for HTTP but for the next messages the overhead is not that much higher than TCP. And you can do also WebSockets with HTTPS (i.e. wss:// instead of ws://). It passes through most simple firewalls and proxies, but more deeper inspection firewalls might still have trouble with it.
Setting up a TCP listener will be a problem if you have your IoT device behind some NAT router, i.e. the usual setup inside private or SoHo networks. To reach the device one would need to open a tunnel at the router from outside into the network, either by administrating the router by hand or with UPnP (which is often switched off for security reasons). So you would introduce too much problems for the average user.
Which means that the thing which the fewest problems for the customer is probably HTTP polling. But this is also the one with the highest overhead. Still mostly compatible are WebSockets which have less overhead and more problems but even less overhead can be reached with simple TCP to the server. TCP listener instead would cause too much trouble.
As for resources on the server side: each HTTP polling request might use new TCP connection but you can also reuse an existing one. In this case you could decide between more overhead and latency one the client side (new TCP connection for each request) which needs few resources on the server side and less overhead and latency on the client side which needs more resources on the server side (multiple HTTP requests per TCP connection). With WebSockets and plain TCP connection you always need more server side resources, unless your client will automatically re-establish the connection on loss of connectivity.
These days you should use a IOT Specific communication protocol over TLS 2.0 for secure light weight connections. For example AWS uses MQTT http://mqtt.org/ and Azure uses AMQP https://www.amqp.org/
The idea is you get a broker you can connect to securely then you use a messaging protocol with a topic to route messages to the proper devices. Also IBM has been using MQTT for a long time and routers now typically come with port 8883 open which is MQTT over TLS.
Good Luck!
Simply use SignalR to connect client and server. It provides you minimal latency without polling. The API is very simple to use.
Physically, this runs over WebSockets which are scalable to a large number of concurrent connections. If you don't have a need for more than 100k per Windows server this would not be a concern.
Question
is there a possibility pings have a 100% success rate but other tcp traffic may fail randomly? If so why and how to resolve the issue?
Background
We are binding to a ussd gateway however we sporadically lose connectivity and are forced to rebind. Now it is my understanding that tcp conections are left open unless closed by the applicaion layer or a node between the ussd client closes the connection such as a Nat router or a Firewall render them stale and close them. The counter intuitive part is that pings between the client and ussd have 0% failure rate.
So the question is there a possibility pings never fail but other tcp traffic may fail? If so why?
Yes. They are different protocols and are handled by different software.
IME, the usual reason for such problems is that the TCP server and/or client are buggy.
The Arduino Ethernet (board or shield) supports a maximum of 4 connections. If acting as a server, it is very easy to cause a denial of service: just open 4 connections to the Arduino without sending any data. The server code will never realize that connections were established, as the EthernetServer::available() method only returns an EthernetClient when the connection has data available. The connections will thus remain open for as long as the client maintains them, preventing the Arduino from processing other requests, without an opportunity for the server to close them after some time.
So the question is: is there a way to set a timeout on idle connections, maybe at the Wiznet chip level?
As background information, the GET HTTP model of a dialog is called "stateless" what this means is that the connection can be closed and then re-opened with no loss of information.
However, there is an HTTP keyword where the browser can ask to keep a connection open.
The server is allowed to close a connection, in fact HTTP/1.1 defines a message from the server to browser that does just that.
So, in an Arduino where you control all of the software (TCP/IP stack and server code), then you should be to modify the WizNEt libs or add your own wrappers that time-out a dialog and close it, independent of what the browser is doing.
Thus, no DNS (or at least the keep the line open style of DNS).
Get the HTTP/1.1 spec (it is on the web) for more information and specifics.
Can anyone let me know how SignalR works internally in a high level way?
I am guessing it is flushing the data using Response.Flush and at client side it is sending Ajax requests at certain intervals. Is it correct?
No, SignalR is an abstraction over a connection. It gives you two programming models over that connection (hubs and persistent connections). SignalR has a concept of transports, each transport decides how data is sent/received and how it connects and disconnects.
SignalR has a few built in transports:
WebSockets
Server Sent Events
Forever Frame
Long polling
SignalR tries to choose the "best" connection supported by server and client (you can also force it to use a specific transport).
That's the high level. If you want to see how each transport is implemented, you can look at the source code.
There's also client code for each transport:
https://github.com/SignalR/SignalR/tree/master/src/Microsoft.AspNet.SignalR.Client.JS
If you're asking about how the long polling transport works in particular:
It sends an ajax request to the server that's waiting asynchronously for a signal to respond. When there is a signal or the request times out, it returns from the server and sends another request and the process continues. (I left some details out about how the client it keeps track of what it saw so it doesn't miss messages)
Hopefully that answers most of your question.
#davidfowl has already answered the major portion. However, to provide some more details regarding the difference in behavior of transports, specifically between WebSocket and other transports; below are some points.
WebSocket is the only transport that establishes a true persistent, two-way connection between client and server. However, WebSocket is supported only by IIS 8 or above, and the latest versions of Internet Explorer, Google Chrome and Mozilla Firefox.
While Server Sent Events, Forever Frame and Long polling, all three follow a one-way communication, and are supported by most of the browsers.