I'm currently working on an app that will need some realtime communication between two clients. Not necessarily text chat. I was wondering if I can utilize free IRC services like Freenode to act as the backend of sorts for my app's communication?
I skimmed through their TOS and I couldn't find anything against it. But I want to know if there are some gotchas that I need to be aware of.
It sounds like what you're really asking is something like the following...
How can I communicate between two clients over the Internet, even if both of these clients are behind some kind of firewall that prevents a direct TCP or UDP connection?
Answer: The usual solution is to use a server on the Internet that is reachable by both clients as an intermediary, to relay their traffic. Up until recently this was accomplished in a very application specific manner and required managing a dedicated server on the Internet. But what if there was a way to offload the burden to someone else...
I was wondering if I can utilize free IRC services like Freenode to
act as the backend of sorts for my app's communication?
Answer: Probably not. Or if it works for your test app, you will quickly get banned in production when you try to send a significant amount of traffic through the IRC servers. Fortunately this kind of relay service is actually available for developers and production use cases. The WebRTC was designed specifically to make these kinds of real-time applications possible. The firewall-busting buzzwords you should be Googling for are STUN and TURN.
I'm currently looking into Twilio's hosted service for my own apps, however it's also possible to host your own TURN relay on Amazon's EC2. Unfortunately there's no such thing as a free lunch, so you'll have to pay some amount for each of these services but you'll be able to bask in the warm glow of writing robust, standards compliant code.
I want two users of my web application to agree to communicate with each other, and then my server would no longer be relied on for the communication. What are my options to have two computers communicate over the internet without relying on my server? How would I set this up--specifically, what API or protocol could I use?
I want to allow the users to run various secure computations, and I don't want my server to act as a trusted third party.
This previous question discusses the problem, but the answer is very vague. I know very little about doing web anything, and I'm having trouble finding where to look. Someone pointed me to the xmlhttprequest API, but I don't see how that is helpful.
It is possible with pretty new WebRTC technology. You should look at
data channels between clients. RTCDataChannel API provides secure (DTLS based) peer-to-peer communication. Not all browsers have support of the WebRTC and among them not all supports RTCDataChannel API. So beware browser compatibility problems.
For example we have chat client application executable (and we can change server Endpoint).
How I can analyze sockets, packets send to server and back? So I can write server emulator for that client?
This is just example. I know this is very general question. But I need general answer. What techniques you can suggest? What tools? Any tutorials or books?
This is for educational purpose and I have no intention to violate any law.
Edited: Basically I want to get protocol that client/server communicate.
If you consider about writing a server emulator, you must know what to emulate, and therefore you must know the details of the protocol. So I doubt that approach can help to discover the unkown features.
I think that the Wireshark protocol analyser can help you to see all the network dialog between the server and the client. You do not have to write a custom server, just spy the actual exchanges :o)
I'm developing open source cross-platform platform for non-realtime multiplayer chat/game system. Like card games, boardgames, turn-based, etc. Servers are spawned server-side (not from users computer). Client has these game modules or downloads game module and then runs them.
I've come to conclusion that best protocol is "IRC-like". So that there is channels for chats and games.
For example there's channels
Lobby
Lobby.English
Lobby.Finnish
Lobby.Mygame - chat for one particular game
Lobby.Mygame.English
Game.Server.Announcements - When new server is created server shouts it here. All clients are listening to this. You can get list of all game servers running.
Game.3636534.chat - Spesific game chat
Game.3636534.game - Users send commands like Buy X here and server responds to them
Some stuff i've looked into
XMLRPC is no good because you have to poll for new data every few seconds
ICE - Pretty good but should be a bit more RPC friendly
ActiveMQ/RabbitMQ and it's publisher/consumer system is something i'm looking for. Problem is that good user ACL is missing. So basically everyone can "go" everywhere and send any garbage to the main AMQ server.
The supported programming languages for this should be at least C#, Java and Python. Encrypting support should be also there. And of course free.
You should look into XMPP which is an open standard for asynchronous message passing. It has support for encryption, firewall traversal and much else. Servers and libraries are available for many, many platforms including the ones you mention.
Links to implementations and information: http://xmpp.org/
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
If I have a separate system with its own concept of users and presence, what is the most appropriate architecture for creating a bridge to an XMPP server network? As far as I can tell there are three primary ways:
Act as a server. This creates one touchpoint, but I fear it has implications for compatibility, and potentially creates complexity in my system for emulating a server.
Act as a clients. This seems to imply that I need one connection per user in my system, which just isn't going to scale well.
I've heard of an XMPP gateway protocol, but it's unclear if this is any better than the client solution. I also can't tell if this is standard or not.
Any suggestions or tradeoffs would be appreciated. For example, would any of these solutions require running code inside the target XMPP server (not likely something I can do).
The XMPP gateway protocol you've heard of is most likely to do with transports. A transport is a server that connects to both a XMPP server and a non-XMPP server. By running a transport, I can use my Jabber client to talk to someone using, say, MSN Messenger.
A transport typically connects once to the remote network for each JID that it sees as online. That is, it's your option 2 in reverse. This is because there is no special relationship between the transport and the non-XMPP network; the transport is simply acting as a bunch of regular clients. For this to work, XMPP clients must first register with the transport, giving login credentials for the remote network, and allowing the transport to view their presence.
The only reason this has a chance of scaling better is that there can be many transports for the same remote network. For example, my Jabber server could run a transport to MSN, another Jabber server could run another one, and so on, each one providing connections for a different subset of XMPP users. While this spreads out the load on the Jabber side, and load balancing on your system may spread out the load as well, it still requires many connections between the two systems.
In your case, because (I assume) the non-XMPP side of things is cooperating, putting a XMPP server interface on the non-XMPP server is likely your best bet. That server interface is best suited for managing the mapping between XMPP JIDs and how that JID will appear on its own network, rather than forcing XMPP users to register and so on.
In case you haven't seen these, you might find them useful:
http://www.jabber.org/jabber-for-geeks/technology-overview
http://www.xmpp.org/protocols/
http://www.xmpp.org/extensions/
Hope that helps.
I too am working on a similar system.
I am going with the gateway/component route. I have looked at several options and settled with this one.
The gateway is basically a component with the specific purpose of bridging Jabber/XMPP with another network. You will have to build most of the things you take for granted when using XMPP as a client. Stuff like roster control.
There is very little help online on the actual design and building of a component. Like the above answer I found that the xmpp protocols/extensions to be of help. The main ones being:
Basic Client 2008
Basic Server 2008
Intermediate Client 2008
Intermediate Server 2008
Reading through these will show you what XEPs you will be expected to be able to handle. Ignore the stuff that will be handled by the server that your component will be attched to.
It's a shame that Djabberd has such poor documentation as their system of "everything is a module" gave the possibility of backend of the server could interface directly to the other network. I made no headway on this.
There are basically two types of server to server (s2s) connections. The first is either called a gateway or a transport, but they're the same thing. This is probably the kind you're looking for. I couldn't find specific documentation for the non-XMPP side, but how XMPP thinks about doing translations to legacy servers is at http://xmpp.org/extensions/xep-0100.html. The second kind really isn't explained in any additional XEPs -- it's regular XMPP s2s connections. Look for "Server-to-Server Communication" in RFC 3920 or RFC 3920bis for the latest draft update.
Since you have your own users and presence on your server, and it's not XMPP, the concepts aren't going to map completely to the XMPP model. This is where the work of the transport comes in. You have to do the translation from your model to the XMPP model. While this is some work, you do get to make all the decisions.
Which brings us right to one of the key design choices -- you need to really decide which things you are going to map to XMPP from your service and what you aren't. These feature and use case descriptions will drive the overall structure. For example, is this like a transport to talk to AOL or MSN chat services? Then you'll need a way to map their equivalent of rosters, presence, and keep session information along with logins and passwords from your local users to the remote server. This is because your transport will need to pretend to be those users and will need to login for them.
Or, maybe you're just an s2s bridge to someone else's XMPP based chess game, so you don't need a login on the remote server, and can just act similarly to an email server and pass the information back and forth. (With normal s2s connections the only session that would be stored would be SASL authentication used with the remote server, but at the user level s2s just maintains the connection, and not the login session.)
Other factors are scalability and modularity on your end. You nailed some of the scalability concerns. Take a look at putting in multiple transports to balance the load. For modularity, see where you want to make decisions about what to do with each packet or action. For example, how do you handle and keep track of subscription data? You can put it on your transport, but then that makes using multiple transports harder. Or if you make that decision closer to your core server you can have simpler transports and use some common code if you need to talk to services other than XMPP. The trade off is a more complex core server with more vulnerability potential.
What architecture you should use depends on the non-XMPP system.
Do you operate the non-XMPP system? If yes, you should find a way to add an XMPP-S2S interface to that system, in other words, make it act as an XMPP server. AOL is using this approach for AIM. Unfortunately, they have restricted their gateway to GoogleTalk.
You don't operate the non-XMPP system but it has a federation interface that you can use - i. e. your gateway can talk to the other system as a server and has a namespace of its own. In this case, you can build a gateway that acts as a federated server on both sides. For I don't know of any example of a gateway that uses this approach but you could use it if you want to build a public XMPP-to-SIP bridge.
If the non-XMPP system doesn't give you a federation interface, then you have no other option but acting as a bunch of clients. In the XMPP world, this is called a "transport". The differences between a transport and a normal server are basically:
the JIDs of the transport are mapped from another system (e.g. john.doe\40example.net#msngateway.example.org - really ugly!)
XMPP users who want to use the transport need to create an account on the non-XMPP system and give the login credentials of that account to the transport service. The XMPP protocol even has a protocol extension that allows XMPP users to do transport registrations in-band.
One other approach is to work with your XMPP server vendor. Most have internal APIs that make injecting presence possible from third party applications. For example, Jabber XCP provides an API for this that's really easy to use.
(Disclosure: I work for Jabber, Inc, the company behind Jabber XCP)