Computer Networks & Distributed Systems - networking

I am hoping to develop a chat program that will allow two people to communicate over a TCP or a UDP connection, basically with a client and a server.
Could anyone please suggest some books I could use that would cover such topics and perhaps also contains tutorials also.
Thank you for your time :)

I suggest you read this book (http://professionalxmpp.com/), it describes the whole chatting stack, frontend and backend along with XMPP protocol explanation, and how to use it to make your chatting application.
let me know if that what you want .. Good luck!
Edit
Also, a good list of books that might help you on the same topic: http://xmpp.org/resources/books/

Related

a webRtc Clearing View

i'm a newbie to webRTC and their is some stuff that i did not get if it was possible i would like an answer to those question and i quiet think that it will be a good reference to all the other guys over the web .
webRTC server code witch left to be handle by the developer what is it job ? i mean their is a lot of signaling method using websocket and socket.io but what did they send to the server ? .
i see some github sources in may learning path that provide these "id" i'm wondering does the server code provide these id and what is it job ?.
i did not get how i can share video conf in real base scenario .. any concret example explanation ?.
i'm wondering if i can use a combination of signalR and webRTC . is is possible thus signlaR provide real time communication and data delivering and the webRTC provide many many services like video conf .. audio .. data exchange .. etc . and is it a valid server code ? .
1) The server-side differs depending on the method used for signalling. For WebRTC specifically, because any browser that supports WebRTC will also support WebSocket, WebSocket is the likely candidate to be used for the signalling method.
Now, the server-side for WebSocket can be somewhat complex, as you have to first handle the handshake to elevate the protocol to either ws or wss, and after that, you have to handle the encrypting and decrypting of all messages sent over the line via WebSocket. This is not trivial at all, but if you do some searching around SO and the web in general for information about how to code the server-side for WebSocket, you should be able to find what you're looking for.
2) I can't understand what you're asking in this question. Could you please provide an example/link? Thanks.
3) You use WebRTC to establish a peer-to-peer connection between two clients to quickly transfer data back and forth. One benefit of this peer-to-peer connection (and the speed at which you can transfer data) is the ability to establish video connections. Also, you can establish video links between more than two clients at a time, although with too many connections, there can be bandwidth issues.
What specifically do you want to know about how to use this technology for video conferencing?
4) I'm not too familiar with SignalR, but looking at the home page, SignalR is used to push data from the server. WebRTC doesn't use a server at all (once the peer-to-peer connection has been established). By that rationale, WebRTC will likely always provide a better, faster connection than SignalR.
Please clarify some of your questions as noted above, and I will help in any way I can. Thanks.
I can answer number 4...
You can of course use SignalR to do the signaling between clients to get WebRTC running, but SignalR has no built-in functionality for the WebRTC signaling so you are in for a pretty nasty job if you are planning on doing it your self.
Since you are asking about SignalR I am jumping to conclusions here and guess that you are a .NET developer? If so there are .NET libraries out there that already have taken care of the signaling for you. One of them is XSockets.NET.
Just install the sample package from XSockets and you will have a multi video chat up and running in a minute.
Sorry for not answering 1,2 and 3... But I hope that the package from XSockets will solve these quesitons :)

Is google chat server based or p2p like skype?

For quite some time, I've been confused about whether Google chat is server based or is peer to peer like skype or as someone told me is it mix of both server and p2p communication?
Since Google chat history is stored on Google servers (unless you go off the record ), to me it indicates that it should be server based (but ofcourse if needed that's possible with p2p as well).
Now, the person who told me that Google chat is p2p (or that mix option if that makes sense) has quite a lot of experience in networking and I'd generally trust his words, but this time I'm just unable to convince myself that Google chat could be p2p and I'm looking for some citation or something which could help clarify the doubt. Is there someone who knows whether it's server based or p2p?
PS: Sorry if the question doesn't really belong to SO, please help me if another SE site is better place for this question. I just thought that since it's about networking and so it could possibly be fine here.
Google ditched XMPP recently and is now shifting to it's own closed sourced proprietary protocols. Which are still Server Based since google will now record all chat activity and going off record is not an option.
"They phasing out 'chat; with 'hangouts', which reneges on the server-to-server aspects of XMPP. More information on the implications (not directly related to the question) here" - Barney
Previos answer
They use XMPP ( jabber )
Extensible Messaging and Presence Protocol (XMPP)
As answered here
I guess reading Jabber would explain things a bit, it's an open protocol and much of the stuff would be available..
For instance Wikipedia explains a bit

I want to build a decentralized, reddit-like system using P2P. What existing p2p library should I base it on?

I want to build a decentralized, reddit-like system using P2P. Basically, I want to retain the basic capabilities of reddit, but make it decentralized, to make it more robust and immune to censorship. This will also allow people to develop different clients to match the way they want to browse it.
Could you recommend good p2p libraries to base my work on? They should be open-source, cross-platform, robust and easy to use. I don't care much about the language, I can adapt.
Disclaimer: warning, self-promotion here !!!
Have you considered JXTA's latest release? It is probably sufficient for what you want to do. Else, we are working on a new P2P framework called Chaupal, but it is not operational yet.
EDIT
There is also what I call the quick-and-dirty UDP solution (which is not so dirty after all, I should call it minimal).
Just implement one server with a public address and start listening for UPD.
Peers located behind NATs contact the server which can read how their private IP address has been translated into a public IP address from the received datagrams.
You send that information back to the peer who can forward it to other peers. The server can also help exchanging this information between peers.
Then peers can communicate directly (one-to-one) by sending datagrams to these translated addresses.
Simple, easy to implement, but does not cover for lost datagrams, replays, out-of-order etc... (i.e., the typical stuff that TCP solves for you at the IP stack level).
I haven't had a chance to use it, but Telehash seems to have been made for this kind of application. Peer2Peer apps have a particular challenge dealing with the restrictions of firewalls... since Telehash is based on UDP, it's well suited for hole-punching through firewalls.
EDIT for static_rtti's comment:
If code velocity is a requirement libjingle has a lot of effort going into it, but is primarily geared towards XMPP. You can port off parts of the ICE code and at least get hole-punching. See the libjingle architecture overview for details about their implementation.
Check out CouchDB. It's a decentralized web app platform that uses an HTTP API. People have used it to create "CouchApps" which are decentralized CouchDB-based applications that can spread in a viral nature to other CouchDB servers. All you need to know to write CouchApps is Javascript and learn the CouchDB API. You can read this free online book to learn more: http://guide.couchdb.org
The secret sauce to CouchDB is a Master-to-Master replication protocol that lets information spread like a virus. When I attended the first CouchConf, they demonstrated how efficient this is by throwing a "Couch Party" (which is where you have a room full of people replicating to the person next to them simulating an ad hoc network).
Also, all the code that makes a CouchApp work is public by default in special entities known as Design Documents.
P.S. I've been thinking of doing a similar project, but I don't have a lot of time to devote to it at the moment. GOD SPEED MY BOY!

VB.Net - Networking method for client/server game

My first question so go easy on me :)
I've been developing for years and have written WAY too many apps (mostly web apps) using web services - I'm happy with SOAP/WSDL/etc... I also used to write TCP/IP client-server apps back in the day using good old winsock.
I'm a bit bored and looking for a new project to expand my skills so decided to have a go at doing either a game or some sort of server monitoring and remote control application
I haven't decided which and the answer to this question will hopefully inform my decision.
What I'd like is some advice as to which methods I should be looking to handle the communication.
Let's assume I'm doing thew game for the moment - I want 2-way communication with low latency and the ability to handle as many simultaneous connections as possible.
I've considered web services but it seems like a lot of overhead - especially as I'd need the client to expose one as well.
TCP/IP would do the job but seems like it's a little low-level and I lsoe a lot of the advantages like definitions. Presumably I'd need to formulate a new protocol for the communications etc... I'm also unsure how I'd have one client use multiple channels for concurrent information - eg a chat and updating location information. I could attempt to multiplex this in some way but my initial ideas re: the queuing seem quite messy
.Net remoting - I've not really touched this much at all. Seems to have low overhead and more flexibility than webservices but I don't know enough to evaluate properly.
I'd really appreciate any input you can provide (and a link to a tutorial would be fantastic)
Thanks in advance for your help
EDIT: I've had an answer which points me at a UDP library. Is UDP appropriate for this? For location information/similar which requires no history, I can see how this is advantageous but for a chat, a lost packet could be an issue - or do I manually send back an acknowledgment of receipt? If so, aren't I duplicating TCP/IP functionality for limited advantage?
Apologies if this is an incorrect way to expand on the question - guidance for that appreciated too :)
If you're up to date on .NET 3.5 SP1, then you should use WCF. You say you don't want to use web services, and I assume from that you mean you don't want to use SOAP over HTTP. WCF does a lot more than SOAP over HTTP. In particular, it can do binary over TCP/IP using the same infrastructure. It also has support for peer-to-peer.
Take a look at something like Lidgren and see how that work's. Its written in c# so its able to be used with VB.Net
Lidgren is a socket wrapper, Ive used it in a few small scale multiplayer games, ( mainly by using a header stating packet type. ie first byte represents packet type,
Lidgren

How can I learn _really_ low-level network programming?

So I want to learn all about networks. Well below the socket, down to raw sockets and stuff. And I want to understand hubs, routers, access points, etc. For example, I'd like to be able to write my own software to do this kind of stuff.* Is there a great source for this kind of information?
I know that I'm asking a LOT here, and that to fully explain it all requires from high level down to low level. I guess I'm looking for a source similar in scope and depth to Applied Cryptography, but about networks.
Thanks to anyone who can help to point me (and others like me?) in the right direction.
* Yes, I realize using any of my hand-crafted network stack code would be a huge security issue, and am only looking to do it to learn :)
Similar Question: here. However I'm looking for more than just 'what's below TCP/UDP sockets?'.
Edited for Clarification: The depth I'm talking about is above the driver level. So assuming that the bits can make it to and from the other end of the wire, what next?
I learned IP networking from TCP/IP Illustrated. Highly recommended.
This may not help you learn it, but a packet sniffer like Wireshark will give you some insight into what the data looks like at a pretty low-level protocol (TCP/IP).
As you have obviously recognised, the universe does not start and end with the IP Protocol. Take a look at the OSI 7 Layer Model where IP is a Layer 3 (Network) protocol. Common IP Routers will operate at this level, but there is more complexity you probably should understand in the Data Link and Physical layers before you start coding your own network stacks.
Start with the fundamentals of data communications in all its myriad forms and work your way up the stack until you get to where you need to stop. Data Communications, Computer Networking and Open Systems is a good foundation text, and then look for more detail on each area you need to focus on. Previous answers include good links for IP and TCP/IP, and as mentioned Wireshark will let you look down through some of the layers
CISCO CCNA materials contain a great network fundamentals, but does not affect programming aspect. I'm not sure that there is an official free link, but you can try to find them.
You should equip yourself with a c compiler and the necessary libs and headers for your OS and play around. You may want to read for example:
http://snap.nlc.dcccd.edu/learn/fuller3/chap13/chap13.html
I had some more links in my delicious account, but they all went down the digital drain ;-)
Have you any embedded programming experience ? If so I recommend you buy one of these development boards. They are cheap and allow you work on every part of the networking stack plus all the software tools required are free.
Note that getting going on it isn't easy and I ended up reading the CS8900 IC datasheet to learn how to make it communicate with the ARM7 based processor. But if you enjoy that sort of thing (as I do) then they are great fun.
Hmmm ... have you looked into Computer Networks by Tanenbaum ?
The TCP/IP Guide
I have found the networking chapter in "understanding the linux kernel" and "understanding linux network internals" from oreilly to be very helpful.
The TCP/IP stack is a very good start but there is a lot more and a good understanding of how ethernet works and how ethernet != IP != the-interweb will go a long way.
books on network security often do a decent if not goos job explaining how networks work in a concise context.
what really did the trick for me was taking a job implementing NAT :)
This course worked for me: COS 461 at Princeton. Note that it assumes system-level programming experience with C.
Pretty much all the readings and lectures are available online under "Syllabus". And you can try the assignments too (unfortunately, you won't have access to the Virtual Network System).
Check this.. it is a good collection of information:
http://www.tcpipguide.com/free/t_toc.htm

Resources