I am working on a project that involves triggering actions on various internet connected devices such as phones, computers, home automation devices, etc. What technologies/general actions are available to connect devices like that? From my limited knowledge I can think of constant poling by all the devices, or making use of a prolonged HTTP connection (I believe that is somewhere along the lines of how Facebook does their notifications).
What other options do I have? What frameworks are there out there? Which option is the best when it comes to time from when the server wants to contact the device to when the device is actually contacted? Which option is easiest on the battery life of the devices?
For battery life, there's not much that beats a plain TCP socket connection. If you can do all the device's polling over a single connection, there is very little overhead besides a few bytes every x seconds for a keepalive packet (which you probably should either enable in the TCP stack or generate as a part of your protocol) This is afaik similar to what Apple uses for iOS notifications.
If you're in an environment where socket communication may be blocked by a firewall, I'd go for websockets with a fallback of long polling (which you mentioned in the question). Combined with proxy support, this should be able to traverse just about any firewall without much complication.
Related
My knowledge about network programming is limited, so, all the comments are more than welcome. Essentially my question boils down to the following question:
Q1. Is there really such a thing as decentralized asynchronous cross-platform peer-to-peer communication?
Let me explain myself.
If we have two http servers running on computers with actual IP addresses, then clearly the answer is yes, assuming one writes a protocol for the interaction.
To go one step further, if one of them (or both) is (are) behind a router, then, with port forwarding the communication can still be established. However, here the problems start because if someone wants to run such a server on the background, say in a mobile phone, the app that is relying on this server really works when one is at home (we can not really expect to request port forwarding everywhere we go).
But even beyond that,
Q2. do mobile phones obtain an actual IP address from telecommunication companies when someone is not using a wi-fi?
If this is true, then clearly one can have cross-platform asynchronous peer-to-peer communication at the expense of not using wi-fi by running an http server on a smartphone. (I understand that this is not convenient, but it is certainly doable.)
Concluding, the two (perhaps there are more) relevant questions that I can think of are:
Q3. How does Skype really work?
Q4. How does Viber really work?
Based on the answer for Skype, it says: If one of the callee or both of them do not have a public IP, then they send voice traffic to another online Skype node over UDP or TCP.
So, it appears that there is no direct communication in Skype, because they have to use a man-in-the-middle for such a scenario.
Regarding Viber, I could not find a good-thorough answer to this particular question. Do people talk to each other through a Viber centralized server, or, do they establish a direct connection? Of course if they do establish a direct connection, then I really want to know how they manage such a thing since a mobile phone may or may not have a physical address. How is a Viber message routed to my cell phone from a friend of mine even when Viber is not running and I am behind a router?
I guess the answer to Viber is really push notifications, but as far as I can understand, all the variations of push notifications rely on open connections, and then the servers of the applications send the notifications to the clients through such connection(s). So, this approach gives us the feeling that it is asynchronous, but essentially it is not. We are cheating, in the sense that there is a constantly open connection to a server, and moreover, as far as I can understand, the application server has to push the notification through that server. Schematically:
A > Central App Server > Central Server w/ open connection to my cellphone > me
So, this seems to be once again a centralized approach.
Honestly, the only approach that I can think of that is both decentralized and asynchronous (on mobile phones as well) is to run an http server on every platform/device, but this comes at the expense of not using Wi-Fi and assuming that a telecommunication company really assigns a physical IP address to every mobile phone (which I do not know if it is true, do you?).
What about WASTE, darknets, F2Fs, etc? Do they offer advantages in the sense of a more direct asynchronous communication between some interested parties? Are there real-world applications (also including mobile phones) using such approaches for communication.
Really, this is not the actual problem that I would like to work on, but I would like to know what the state of the art is so that I can figure out how I can proceed from there. So, all comments are really more than welcome. If you have references for the state of the art I would like to know about them as well, but a brief description would also be nice.
I appreciate all your time and effort in advance.
You asked many questions, here is the beginning of the answers:
Q1: Yes. For example, take BitTorrent's very successful 10 million+ node network. Aside from the bootstrapping process, the protocol is entirely decentralized and asynchronous. See here for more info.
Q2: Yes! Go to www.whatismyip.com on your mobile telephone, and you will see your assigned IP. However, you are likely to be very filtered (e.g: incoming traffic on port 80 is likely to be blocked).
Q3: It has elements of P2P and clever tricks to get around NAT issues - see here for more info.
Q4: I don't know.
I have a questions regarding WebSocket communications in mobile connections.
I was wondering how the long-lived TCP connections can be handled for a long time in mobility networks when the user migrate among different networks. What happens to already established TCP connections when handover (hand-off) occurs?
Do different technologies (3G, 4G or etc) behave differently in this case?
I will appreciate if you could leave some online sources or articles as well that I can read more in this regard?
Thank you in advance :)
The hand-off is always transparent to the user — all TCP and voice connections are always kept active when transitioning between the towers on a commercial mobile network like LTE, UMTS etc. You might experience some periods of time where the data stops flowing, but that's about it.
I've had several opportunities to verify this myself through an interesting experiment on a T-Mobile USA's HSPA+ nationwide network. Take a 12-hour-plus drive from one major city to another one, without turning your phone off. Take a look at the area where the external IPv4-address terminates (by using traceroute). You might as well notice that it's still at the same area where you've started your trip. Now reboot the phone, and see where the external IPv4 address is routed to now. You'll notice that now it's likely terminated in a major metro area closer to where you are. I.e., your connection within the core network of the operator follows you along not just within a given city, metro or state, but also between the states and the timezones.
The reason for this is that the carrier has a Core Network, and all external connections are handled by the Packet Gateway of the Core Network, which keeps track of all the connections. More on this is documented in Chapter 7 of the book called High Performance Browser Networking (HPBN.co).
This is not really a SO but more a programmers question and I don't see what you have researched for yourself, but you certainly can't rely on a connection to stay alive, mobile or not.
In fact mobile operators kill long-living connections by resetting them after a certain amount of time or data. So you should be ready to reconnect upon a socket exception anyway.
I have to devise a push notification plan for some mobile devices where push is not yet officially supported by C2DM / APNS / BES ,etc. MQTT was also a good option but I could not get an MQTT Client for my required platform. (Blackberry Playbook, ActionScript only)
Going back to basics, I had the options of Long Polling and streaming. Streaming using a high keep-alive value seems good to me, as the development of the mobile app (it is not a browser) and the server code are both in my control. To maintain the connection open, I just need to ping the server occasionally.
What issues I may be facing in this approach ? I understand this would constantly open an HTTP connection from the device to the server. Would the battery life of mobile device drain quickly just by persistently opening a server connection ? ...Afterall the data is not being transferred, just by keeping the connection open...right ?
You might want to consider Urban Airship or take a look at this realtime technologies guide.
If realtime really matters, I'm talking about seconds or milliseconds, then a realtime web technology using WebSockets or HTTP Streaming is your best option as they deliver updates the instant they are available. This sort of thing is best suited to the sort of application that a user is actively using during a reasonable period of time where they need instant notifications or are collaborating/chatting with others.
WebSockets are probably a better standard to be using with longer term technology considerations in mind since they are the first standard for full duplex bi-directional communication between a client (mobile, web, desktop) and server.
What platform could you not find an MQTT client for? http://mqtt.org/software lists clients in just about every language I can think of and more than I'd like to bother using. The mosquitto C client library is very portable.
For battery life, it really depends what you're doing between the keepalives...
If you app runs in a country where the network isn't in the best shape, then having all your clients keep a connection open can put strain on their infrastructure (assuming you are that popular).
Battery life is the other consideration, yes, a lot of devices aren't that efficient in keeping an idle radio/3g connection open and it drains the battery a lot faster than not having a connection open.
So when I am debugging my web applications and such, I've used the Charles web proxy and debugger and love it. It's so nice to see what's being sent and received via port 80 and 443. I can see all the resources loading, not just from the "browser" per say, but also flash applications. I can also see how the calls are being made, and it pretty easy to reconstruct them. It's a great debugging tool and I love it.
So I'm wondering two things:
First, I'm wondering is if there is something similar I can use to watch traffic that might be coming though on other ports. I guess some desktop applications will use the internet, but not necessarily via http / https requests. I remember looking at some security tools a few years ago - there are a lot of security tools out there, like kismet / etherCap, ethershark, etc - is there one that does what I'm describing in an easy and intuitive way?
Also, I'm wondering if I am using my iPhone / iPad / Android device, how can I set up a proxy through my computer so I can watch the http/https requests that the device makes?
Found the answer to that one here: http://www.ravelrumba.com/blog/ipad-http-debugging/
I'm mostly on a Mac so anything that is Mac friendly would be extra helpful.
Thanks!
I believe you are looking for Wireshark. It allows you to monitor the network interface on your machine and be able to tell you sent/receive packets as well as their protocols. It also has a protocol decoder that can be used to get Layer 7 information about a IP stream. You can also do a "Follow TCP stream" which allows you to view the entire conversation of that connection. It's based on libpcap (Packet capture) which the built in tcpdump also uses.
The only downside for you web developers is that if you're using SSL encrypted sessions, you can't decode it. The endpoints of the SSL session are "above" (using OSI model) the layer at which wireshark (and similar tools) operate.
Here's a good list http://sectools.org/sniffers.html. I used Wireshark back when it was Ethereal. At that time it ran under X11, It looks like that has changed.
I'm developing a multi-player game and I know nothing about how to connect from one client to another via a server. Where do I start? Are there any whizzy open source projects which provide the communication framework into which I can drop my message data or do I have to write a load of complicated multi-threaded sockety code? Does the picture change at all if teh clients are running on phones?
I am language agnostic, although ideally I would have a Flash or Qt front end and a Java server, but that may be being a bit greedy.
I have spent a few hours googling, but the whole topic is new to me and I'm a bit lost. I'd appreciate help of any kind - including how to tag this question.
If latency isn't a huge issue, you could just implement a few web services to do message passing. This would not be a slow as you might think, and is easy to implement across languages. The downside is the client has to poll the server to get updates. so you could be looking at a few hundred ms to get from one client to another.
You can also use the built in flex messaging interface. There are provisions there to allow client to client interactions.
Typically game engines send UDP packets because of latency. The fact is that TCP is just not fast enough and reliability is less of a concern than speed is.
Web services would compound the latency issues inherent in TCP due to additional overhead. Further, they would eat up memory depending on number of expected players. Finally, they have a large amount of payload overhead that you just don't need (xml anyone?).
There are several ways to go about this. One way is centralized messaging (client/server). This means that you would have a java server listening for udp packets from the clients. It would then rebroadcast them to any of the relevant users.
A second way is decentralized (peer to peer). A client registers with the server to state what game / world it's in. From that it gets a list of other clients in that world. The server maintains that list and notifies the other clients of people who join / drop out.
From that point forward clients broadcast udp packets directly to the other users.
If you look for communication framework with high performance try look at ACE C++ framework (it has Java bindings).
Official web-site is: http://www.cs.wustl.edu/~schmidt/ACE-overview.html
You could also look into Flash Media Interactive Server, or if you want a Java implementation, Wowsa or Red5. Those use AMF and provide native functionality for ShareObjects including synching of the ShareObjects among connected clients.
Those aren't peer to peer though (yet, it's coming soon I hear). They use centralized messaging managed by the server.
Good luck