If you would advise people to start reading RFC's, which would you recommend people start with?
It should be something more complex than RFC 863 (Discard Protocol), but something simpler than rocket science :)
How about the RFC about Simple Mail Transfer Protocol. It concerns everyone of us and is not that hard to read/understand.
And for fun, there is always IP Over Avian Carriers ;)
Related
My apologies in advance if my question seems a little unprofessional... I am not much of a developer after all.
As part of my university project I need to develop a program that will listen on the network and tell whether the traffic is encrypted or not? What's important is only to know if the traffic coming to a network end is encrypted and it's not really important to know the type and the algorithm of the encryption.
I really have no clue where to begin. I'd appreciate it if anyone could give me some clues on what to look for.
Thanks in advance...
Although the answer of GregS is of course correct, the best way to distinquish encrypted text is to look at the protocol used by the application. For instance, if browsers encrypt anything, it's probably done by using SSL. Remote terminals use SSH.
Although the cipher text is mostly looking as random bytes, you can probably distinquish the encryption protocols. If this is enough depends on your use case. As a first step, many protocols use a specific server port by default; this can be used as a heuristic so you can check the most common protocols for that port first.
We're all familiar with popular protocols like IMAP and POP, used for email messaging.
I have a plan for a new protocol, but I'm not sure to go about implementing it.
Is the protocol a collection of C source code, for example, that accepts and sends data through ports? Or is a protocol just a thorough description of how data should be sent, which clients then implement?
I'm lost where to start here, and I'm not very familiar with how the protocol system works.
Edit:
Also, if I write a protocol and it isn't made official by the standards group, can people/clients still implement it?
The official way is to write an RFC - a Request for Comments. People will respond to that (that's why it's an RFC) and probably try to implement your protocol.
As soon as two independent implementations exist that completely support the protocol, it's a new standard.
Of course, people aren't going to implement a new protocol for someone just for fun. So you should first find a group who is interested in listening to you. Maybe there already is a protocol which does what you want (or can easily be extended).
But you probably don't want to invent a new standard. Standards are a lot of work and - for some - overrated.
So you should describe how it works and create a library that can read and write the protocol, so developers can use it even though it's not an official standard.
As you are interested in the Replace Email section of the Paul Graham article you linked, then IMHO you will need to both develop a protocol definition, and also provide an example implementation. The protocol definition does not need to be published as an internet protocol standard in order to be useful.
You will need an implementation to so that you can test, refine and improve the ideas. It is extremely unlikely the protocol will be right at the first attempt, and you'll need something to support the initial users.
You don't need a protocol definition to implement an improved email, but you will need one if you expect others to work with you and adopt it, though it very much depends on your 'business model'. I strongly recommend you have a protocol definition from the start, even if only to keep yourself sane when you try to produce the second implementation.
I recommend having a look at some examples of sneaky approaches to protocols and implementation. My favourite is described in the Viewpoints Research 2008 Progress report on a super-compact approach to TCP/IP.
They did not follow the traditional approach to developing the implementation of a protocol (the protocol stack). Instead they wrote code which parsed the human-readable TCP/IP protocol specification, and generated the code of a TCP/IP stack from that protocol document. The usual TCP/IP stack is about 40,000 lines of code, or more. Their program, which read the protocol specification, and generated the code for a TCP/IP stack 'automatically' was only 160 lines of code. They use extremly powerful programming tools.
If you had an approach like that, you could keep the protocol implementation synchronised with the specification, and potentially make it straightforward for others to adopt your protocol.
HTH
You are confusing a protocol standard with the implementation.
These 2 are unrelated.
A protocol is described in a high level but has enough information for someone to undestand how it should be implemented.
The idea is that someone reading the document can understand how/what to implement in any language of preference
To give an example: SIP protocol in the RFC describes the various flows and also has the various messages and how they are supposed to b processed i.e. the semantics well defined.
You can implement a SIP UA or Server in C++ or Java. This is irrelevant to the SIP protocol
For this you don't need to provide any source code (you could though if you think it helps clarify some obscurity of the description).
The most important part is that your protocol is actually reviewed by stakeholders i.e. people that expect it to solve their problems.
This part is the most important not only because it could solve problems in your protocol but because they can actually verify that the concept is solid i.e. can be technically implemented
The only case that one could specify something concrete or imply something is if for example the protocol described something demanding some specific constraints e.g. hard-real time constraint which could serve as "hint" on which implementation/languages to avoid
Also, if I write a protocol and it isn't made official by the
standards group, can people/clients still implement it?
Strange question.What do you mean?How will someone know your protocol exists?
If it is official he can get it from the standards group to implement it.
Otherwise it is obvious that you have some sort of "proprietary" protocol (which is not uncommon e.g. a company can have an internal protocol for its own software) and people have to get the spec from you.
So I want to learn all about networks. Well below the socket, down to raw sockets and stuff. And I want to understand hubs, routers, access points, etc. For example, I'd like to be able to write my own software to do this kind of stuff.* Is there a great source for this kind of information?
I know that I'm asking a LOT here, and that to fully explain it all requires from high level down to low level. I guess I'm looking for a source similar in scope and depth to Applied Cryptography, but about networks.
Thanks to anyone who can help to point me (and others like me?) in the right direction.
* Yes, I realize using any of my hand-crafted network stack code would be a huge security issue, and am only looking to do it to learn :)
Similar Question: here. However I'm looking for more than just 'what's below TCP/UDP sockets?'.
Edited for Clarification: The depth I'm talking about is above the driver level. So assuming that the bits can make it to and from the other end of the wire, what next?
I learned IP networking from TCP/IP Illustrated. Highly recommended.
This may not help you learn it, but a packet sniffer like Wireshark will give you some insight into what the data looks like at a pretty low-level protocol (TCP/IP).
As you have obviously recognised, the universe does not start and end with the IP Protocol. Take a look at the OSI 7 Layer Model where IP is a Layer 3 (Network) protocol. Common IP Routers will operate at this level, but there is more complexity you probably should understand in the Data Link and Physical layers before you start coding your own network stacks.
Start with the fundamentals of data communications in all its myriad forms and work your way up the stack until you get to where you need to stop. Data Communications, Computer Networking and Open Systems is a good foundation text, and then look for more detail on each area you need to focus on. Previous answers include good links for IP and TCP/IP, and as mentioned Wireshark will let you look down through some of the layers
CISCO CCNA materials contain a great network fundamentals, but does not affect programming aspect. I'm not sure that there is an official free link, but you can try to find them.
You should equip yourself with a c compiler and the necessary libs and headers for your OS and play around. You may want to read for example:
http://snap.nlc.dcccd.edu/learn/fuller3/chap13/chap13.html
I had some more links in my delicious account, but they all went down the digital drain ;-)
Have you any embedded programming experience ? If so I recommend you buy one of these development boards. They are cheap and allow you work on every part of the networking stack plus all the software tools required are free.
Note that getting going on it isn't easy and I ended up reading the CS8900 IC datasheet to learn how to make it communicate with the ARM7 based processor. But if you enjoy that sort of thing (as I do) then they are great fun.
Hmmm ... have you looked into Computer Networks by Tanenbaum ?
The TCP/IP Guide
I have found the networking chapter in "understanding the linux kernel" and "understanding linux network internals" from oreilly to be very helpful.
The TCP/IP stack is a very good start but there is a lot more and a good understanding of how ethernet works and how ethernet != IP != the-interweb will go a long way.
books on network security often do a decent if not goos job explaining how networks work in a concise context.
what really did the trick for me was taking a job implementing NAT :)
This course worked for me: COS 461 at Princeton. Note that it assumes system-level programming experience with C.
Pretty much all the readings and lectures are available online under "Syllabus". And you can try the assignments too (unfortunately, you won't have access to the Virtual Network System).
Check this.. it is a good collection of information:
http://www.tcpipguide.com/free/t_toc.htm
I am learning Computer Networking this semester, on which I find it quite interesting in learning why Internet is designed like today. And I also enjoy reading paper referred to in the teaching slides, like End-to-End Argument in System Design and The Design Philosophy of the DARPA Internet Protocols.
Could you recommend some other interesting paper/articles to me, especially those related to higher layers like TCP/IP protocols?
Many thanks for your reply.
Roy Thomas Fielding wrote his dissertation at UCI about Architectural Styles and
the Design of Network-based Software Architectures (2000), which has more focus on applications in a networked environment; but I would recommend it, since it's related to your request about learning why Internet is designed like today.
Link: Architectural Styles and the Design of Network-based Software Architectures
Generally, the books by Andrew Tanenbaum are worth reading. They are normally used as standard lecture in networking courses.
On the other hand, they are rather basics.
One classic book is "Interconnections 2nd edition" by Radia Perlman.
Another very good one is "Routing in the Internet" by Christian Huitema.
If you are serious about learning how the Internet works, you also need to know about BGP. A decent introduction is "BGP4 Inter-Domain Routing in the Internet" by John Stewart.
A more advanced topic is MPLS. A very good book on this is "MPLS-Enabled Applications" by Ina Minei and Julian Lucek.
Another more advanced topic is multicast. I could recommend "Interdomain Multicast Routing" by Brian Edwards and others.
The books published by Cisco press tend to be good (although obviously vendor-specific) if you are interested in more practical details of how to configure network equipment.
Finally, "Unix Network Programming, Volume 1" by Richard Stevens is a must read if you want to do network programming.
http://www.cs.ucsb.edu/~almeroth/classes/F05.276/papers/vegas.pdf
I'm not sure if this is exactly what you were looking for, but there are some really interesting papers out there on peer-to-peer networking.
Some examples:
Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications
Incentives Build Robustness in BitTorrent
Protecting Free Expression Online with Freenet
If you want an introduction to what (not why) the TCP/IP protocols are, a classic book is TCP/IP Illustrated, Volume 1: The Protocols.
What do you think of Quagga compared to XORP as a dynamic software routing engine? What are the technical merits of each engine comparatively? Additionally, what do most people think of them from a programming view. Who has manipulated networks using these enginers? I was wondering from an OSPF, routing, BGP protocol user's perpspective.
The following does not answer your question completely, but the Vyatta open source routers and the OpenSolaris customer gateway software for Amazon VPC both use quagga to implement BGP support.
From the wikipedia entry for XORP,
"The software suite was selected
commercially as the routing platform
for the Vyatta line of products in its
early releases, but later has been
replaced with quagga.
What do you think of Quagga compared to XORP as a dynamic software routing engine?
It is one of many options, but not particularly of very much use to you based upon your questions/information that you posted here. Have you tried looking into some of the alternatives such as (nothing comes to mind)?
What are the technical merits of each engine comparatively?
Small, fast, oddly placed, optimized, super-heroic and more filler for a resume.
Additionally, what do most people think of them from a programming view.
I can't speak for most people, but I for myself do not give it much credit or merit, or well... you know what I mean.
Who has manipulated networks using these enginers?
I could not find specific references, but I do remember reading that both Disney and the 'famous' YUV corporation of South Africa both played with this notion before. I believe Disney abandoned it with the fall of Michael Eisner.
I was wondering from an OSPF, routing, BGP protocol user's perpspective.
I am a BGP protocol user's prospective. Hopefully we hear from OSPF and routing user's perspectives shortly.
Good question.