I want to learn how to write a software using a peer to peer networking architecture but i don't know where to start, knowing that I use as a programming languages : c/c++ , lisp, a little of python. any pointer to documentation or tutorials is appreciated.
Why not just start with a Bittorrent API, such as https://github.com/JosephSalisbury/python-bittorrent, rather than writing your own.
After you have gained experience with a well-used P2P network, then you may start to see what you could do better, then start to write your own.
here are some guidelines which i found with google
Related
I am hoping to develop a chat program that will allow two people to communicate over a TCP or a UDP connection, basically with a client and a server.
Could anyone please suggest some books I could use that would cover such topics and perhaps also contains tutorials also.
Thank you for your time :)
I suggest you read this book (http://professionalxmpp.com/), it describes the whole chatting stack, frontend and backend along with XMPP protocol explanation, and how to use it to make your chatting application.
let me know if that what you want .. Good luck!
Edit
Also, a good list of books that might help you on the same topic: http://xmpp.org/resources/books/
I'm about to start writing a "server-side" (as in not running in a browser) software that will do a lot of network communication with UDP.
I'm currently leaning towards running the server on Node.js, mostly with the superficial motivation that it's "new and shiny".
As I'm fairly new to Javascript, I'm looking into using a some type of JS-framework, to help me avoid a few of the most obvious mistakes one often makes as a beginner. I'm looking for experiences with networking and Javascript. Does any particular framework make UDP-networking really easy/convenient?
Both Mootools and Dojo look interesting as they are both well maintained, while Prototype/script.aculo.us seems to have become stagnant, and aren't very interesting because of that.
MooTools Server could be a nice add-on to any JS software for its utility methods and OOP layer. However, it would not help in any way to manage specifically UDP.
Actually, all the frameworks you mentioned take their origin from the client-side, and therefore don't have any reason to manage networking at the transport layer (or beneath) in any way.
It is not a very common use-case since JS, even with Node, is very oriented towards webapp that almost always use HTTP over TCP. I don't have any references to give regarding such usage, beyond Node's dgram API. It seems to offer enough abstraction over the datagram to be usable, and I don't really know what you would like to have to make UDP “easier”: if you're messing with the transport layer, you're anyway asking for your hands to get dirty ;)
I want to build a decentralized, reddit-like system using P2P. Basically, I want to retain the basic capabilities of reddit, but make it decentralized, to make it more robust and immune to censorship. This will also allow people to develop different clients to match the way they want to browse it.
Could you recommend good p2p libraries to base my work on? They should be open-source, cross-platform, robust and easy to use. I don't care much about the language, I can adapt.
Disclaimer: warning, self-promotion here !!!
Have you considered JXTA's latest release? It is probably sufficient for what you want to do. Else, we are working on a new P2P framework called Chaupal, but it is not operational yet.
EDIT
There is also what I call the quick-and-dirty UDP solution (which is not so dirty after all, I should call it minimal).
Just implement one server with a public address and start listening for UPD.
Peers located behind NATs contact the server which can read how their private IP address has been translated into a public IP address from the received datagrams.
You send that information back to the peer who can forward it to other peers. The server can also help exchanging this information between peers.
Then peers can communicate directly (one-to-one) by sending datagrams to these translated addresses.
Simple, easy to implement, but does not cover for lost datagrams, replays, out-of-order etc... (i.e., the typical stuff that TCP solves for you at the IP stack level).
I haven't had a chance to use it, but Telehash seems to have been made for this kind of application. Peer2Peer apps have a particular challenge dealing with the restrictions of firewalls... since Telehash is based on UDP, it's well suited for hole-punching through firewalls.
EDIT for static_rtti's comment:
If code velocity is a requirement libjingle has a lot of effort going into it, but is primarily geared towards XMPP. You can port off parts of the ICE code and at least get hole-punching. See the libjingle architecture overview for details about their implementation.
Check out CouchDB. It's a decentralized web app platform that uses an HTTP API. People have used it to create "CouchApps" which are decentralized CouchDB-based applications that can spread in a viral nature to other CouchDB servers. All you need to know to write CouchApps is Javascript and learn the CouchDB API. You can read this free online book to learn more: http://guide.couchdb.org
The secret sauce to CouchDB is a Master-to-Master replication protocol that lets information spread like a virus. When I attended the first CouchConf, they demonstrated how efficient this is by throwing a "Couch Party" (which is where you have a room full of people replicating to the person next to them simulating an ad hoc network).
Also, all the code that makes a CouchApp work is public by default in special entities known as Design Documents.
P.S. I've been thinking of doing a similar project, but I don't have a lot of time to devote to it at the moment. GOD SPEED MY BOY!
I want to experiment with network programming in Haskell. The problem I have is that the documentation for the network package is pretty scarce, especially the one for Network.Socket which I want to use.
Do you know of some other references or clearly written projects where I can see how to use it? Are there any good alternatives to network?
Network.Socket is just bindings to the Berkeley socket API. You should read Beej's Guide to network programming.
EDIT: If you're on *nix then see the man pages for socket, bind, listen, accept, connect, recv, send and family. No matter your OS, there are also some higher level packages on Hackage (ex: network-fancy, network-server) you should look at if all you want to do is communicate (and not get involved in the gritty details).
Chapter 27 of ``Real World Haskell'' introduces networking in Haskell.
So I want to learn all about networks. Well below the socket, down to raw sockets and stuff. And I want to understand hubs, routers, access points, etc. For example, I'd like to be able to write my own software to do this kind of stuff.* Is there a great source for this kind of information?
I know that I'm asking a LOT here, and that to fully explain it all requires from high level down to low level. I guess I'm looking for a source similar in scope and depth to Applied Cryptography, but about networks.
Thanks to anyone who can help to point me (and others like me?) in the right direction.
* Yes, I realize using any of my hand-crafted network stack code would be a huge security issue, and am only looking to do it to learn :)
Similar Question: here. However I'm looking for more than just 'what's below TCP/UDP sockets?'.
Edited for Clarification: The depth I'm talking about is above the driver level. So assuming that the bits can make it to and from the other end of the wire, what next?
I learned IP networking from TCP/IP Illustrated. Highly recommended.
This may not help you learn it, but a packet sniffer like Wireshark will give you some insight into what the data looks like at a pretty low-level protocol (TCP/IP).
As you have obviously recognised, the universe does not start and end with the IP Protocol. Take a look at the OSI 7 Layer Model where IP is a Layer 3 (Network) protocol. Common IP Routers will operate at this level, but there is more complexity you probably should understand in the Data Link and Physical layers before you start coding your own network stacks.
Start with the fundamentals of data communications in all its myriad forms and work your way up the stack until you get to where you need to stop. Data Communications, Computer Networking and Open Systems is a good foundation text, and then look for more detail on each area you need to focus on. Previous answers include good links for IP and TCP/IP, and as mentioned Wireshark will let you look down through some of the layers
CISCO CCNA materials contain a great network fundamentals, but does not affect programming aspect. I'm not sure that there is an official free link, but you can try to find them.
You should equip yourself with a c compiler and the necessary libs and headers for your OS and play around. You may want to read for example:
http://snap.nlc.dcccd.edu/learn/fuller3/chap13/chap13.html
I had some more links in my delicious account, but they all went down the digital drain ;-)
Have you any embedded programming experience ? If so I recommend you buy one of these development boards. They are cheap and allow you work on every part of the networking stack plus all the software tools required are free.
Note that getting going on it isn't easy and I ended up reading the CS8900 IC datasheet to learn how to make it communicate with the ARM7 based processor. But if you enjoy that sort of thing (as I do) then they are great fun.
Hmmm ... have you looked into Computer Networks by Tanenbaum ?
The TCP/IP Guide
I have found the networking chapter in "understanding the linux kernel" and "understanding linux network internals" from oreilly to be very helpful.
The TCP/IP stack is a very good start but there is a lot more and a good understanding of how ethernet works and how ethernet != IP != the-interweb will go a long way.
books on network security often do a decent if not goos job explaining how networks work in a concise context.
what really did the trick for me was taking a job implementing NAT :)
This course worked for me: COS 461 at Princeton. Note that it assumes system-level programming experience with C.
Pretty much all the readings and lectures are available online under "Syllabus". And you can try the assignments too (unfortunately, you won't have access to the Virtual Network System).
Check this.. it is a good collection of information:
http://www.tcpipguide.com/free/t_toc.htm