Is there a full specification of the gRPC protocol, like RFC, that could be followed by an implementer? I am looking for document(s) that would allow me to implement gRPC that can interoperate with the official implementations (and currently supported transports).
Different aspects are described conceptually in 1, 2, 3, 4, 5, 6, but I am wondering if it is enough to just replicate that behavior and have a code that will work with the reference implementation, or there is something undocumented that is going on under the hood?
Related
Lots of websites (e.g. twitter, stackexchange) provide RESTful OPEN APIs based on the HTTP protocol. Can I design a RESTful service based on some other protocol (such as raw TCP)?
The short answer is that a RESTful service does generally imply HTTP, but it's not strictly necessary. The wikipedia entry includes a section on implementations outside the web, though it's pretty brief and really only talks about Common Management Information Protocol (CMIP).
Realistically, to most developers, RESTful services operate over HTTP.
You could surely take inspiration from RESTful protocols on the web and build your own similar protocol over raw TCP, but you may well finding yourself implementing it in the language of HTTP. At that point you may want to ask yourself why you didn't just use HTTP in the first place.
If you look at Roy Fielding's PhD thesis, you'll see that REST is defined in chapter 5, while it's applied to HTTP in chapter 6.
"Representational state transfer" is indeed quite abstract. There's no reason you couldn't apply it to your own adhoc protocol. The aim is to make it stateless, to have safe read methods (that are cacheable), and if possible idempotent write methods.
If you adhere to the true tenants of the architecture where every operation is ignorant of historical operations you could probably drum up something a bit different. Currently the easy Put, get, post and delete operations lend well to http based service calls.
We're all familiar with popular protocols like IMAP and POP, used for email messaging.
I have a plan for a new protocol, but I'm not sure to go about implementing it.
Is the protocol a collection of C source code, for example, that accepts and sends data through ports? Or is a protocol just a thorough description of how data should be sent, which clients then implement?
I'm lost where to start here, and I'm not very familiar with how the protocol system works.
Edit:
Also, if I write a protocol and it isn't made official by the standards group, can people/clients still implement it?
The official way is to write an RFC - a Request for Comments. People will respond to that (that's why it's an RFC) and probably try to implement your protocol.
As soon as two independent implementations exist that completely support the protocol, it's a new standard.
Of course, people aren't going to implement a new protocol for someone just for fun. So you should first find a group who is interested in listening to you. Maybe there already is a protocol which does what you want (or can easily be extended).
But you probably don't want to invent a new standard. Standards are a lot of work and - for some - overrated.
So you should describe how it works and create a library that can read and write the protocol, so developers can use it even though it's not an official standard.
As you are interested in the Replace Email section of the Paul Graham article you linked, then IMHO you will need to both develop a protocol definition, and also provide an example implementation. The protocol definition does not need to be published as an internet protocol standard in order to be useful.
You will need an implementation to so that you can test, refine and improve the ideas. It is extremely unlikely the protocol will be right at the first attempt, and you'll need something to support the initial users.
You don't need a protocol definition to implement an improved email, but you will need one if you expect others to work with you and adopt it, though it very much depends on your 'business model'. I strongly recommend you have a protocol definition from the start, even if only to keep yourself sane when you try to produce the second implementation.
I recommend having a look at some examples of sneaky approaches to protocols and implementation. My favourite is described in the Viewpoints Research 2008 Progress report on a super-compact approach to TCP/IP.
They did not follow the traditional approach to developing the implementation of a protocol (the protocol stack). Instead they wrote code which parsed the human-readable TCP/IP protocol specification, and generated the code of a TCP/IP stack from that protocol document. The usual TCP/IP stack is about 40,000 lines of code, or more. Their program, which read the protocol specification, and generated the code for a TCP/IP stack 'automatically' was only 160 lines of code. They use extremly powerful programming tools.
If you had an approach like that, you could keep the protocol implementation synchronised with the specification, and potentially make it straightforward for others to adopt your protocol.
HTH
You are confusing a protocol standard with the implementation.
These 2 are unrelated.
A protocol is described in a high level but has enough information for someone to undestand how it should be implemented.
The idea is that someone reading the document can understand how/what to implement in any language of preference
To give an example: SIP protocol in the RFC describes the various flows and also has the various messages and how they are supposed to b processed i.e. the semantics well defined.
You can implement a SIP UA or Server in C++ or Java. This is irrelevant to the SIP protocol
For this you don't need to provide any source code (you could though if you think it helps clarify some obscurity of the description).
The most important part is that your protocol is actually reviewed by stakeholders i.e. people that expect it to solve their problems.
This part is the most important not only because it could solve problems in your protocol but because they can actually verify that the concept is solid i.e. can be technically implemented
The only case that one could specify something concrete or imply something is if for example the protocol described something demanding some specific constraints e.g. hard-real time constraint which could serve as "hint" on which implementation/languages to avoid
Also, if I write a protocol and it isn't made official by the
standards group, can people/clients still implement it?
Strange question.What do you mean?How will someone know your protocol exists?
If it is official he can get it from the standards group to implement it.
Otherwise it is obvious that you have some sort of "proprietary" protocol (which is not uncommon e.g. a company can have an internal protocol for its own software) and people have to get the spec from you.
Web services is a Service Oriented Architecture implementation.
But, can we say that CORBA, RMI and the Java EE platform are also an implementation of SOA?
If you say that SOA means WS-* standards, then the answer is "no".
But if SOA means distributed components communicating using an agreed-upon protocol, then the answer is yes, you can think of CORBA, RMI, and Java EE as SOA. (My advice is to drop the "2" - it's been out of the picture for a long time. Please refer to it as Java EE unless you want to appear out-of-date.)
Even web services have at least two flavors: SOAP and REST. You might also include XML-RCP.
The difference in all cases is the choice of protocol.
SOAP uses its own XML request/response idiom over HTTP.
REST is straight HTTP - GET, POST, URL for every request.
CORBA uses an OMG standard protocol. Interoperability between ORBs, and different bindings for each language, used to be a big problem. It's been so long since I've used CORBA that I have no idea of the current state of the art. I don't know anyone who uses CORBA anymore. From my vantage point, it's a failed, dead technology.
RMI is the Java-only answer to CORBA. All endpoints must be implemented in Java and speak RMI.
Java EE uses RMI as the underpinnings of EJBs.
Simple and open win: that's why HTTP-based protocols are growing in popularity.
yes. WS-* is one of the mostly used ways in which SOA is implemented.
Earlier on I've implemented a custom C++ server from scratch, something which is nice and fun to do but is very time-consuming.
So I've heard about "servlet" of Java which sounds like something I'm looking for for my next project.
I'm looking at ASP.NET to be used as a servlet because it's pretty much mature in terms of networking capabilities and programming features and I can write code for it in C# which is a language I already know and feels good with.
Anyway before getting into it I'd like to get concise answers regarding some of the capabilities that ASP.NET may provide me with.
My client application is Flash-based and has nothing to do with XML or HTTP protocols (raw sockets).
1) Can I accept raw (in terms of protocol) socket connections and keep them alive as long as needed?
2) How hard/complicated is it to apply AES encryption over a connected socket?
3) Can I have a "shared", in-memory, objects, like lists, over the application's instance?
4) Can I get one message from client X and send a copy/reply/report (on the fly) to client Y, without using a database or anything like that? This question suggests question #5.
5) Can I identify a connection by its memory address or object address, so I can refer to it directly?
6) Can my application be distributed over several servers?
7) How hard/complicated is it to scale such an application? For 10 concurrent connections there's no problem but for 50,000 I can surely expect load issues.
8) Finally, the most important question which probably has the most complex answer: How flexible is the environment from a programmer perspective? I mean, can I control events such as "on_connect", "on_disconnect" and stuff like that? How hard would it be to access connection Y from connection X's event (such as "on_data_received")?
ASP.Net is more like JSP than Java Servlets as it requires a web server to run. Typically IIS is used since Microsoft has built in ASP.Net support there, but it's possible to run ASP.Net on other web servers. As a web server product it's all HTTP and therefor connectionless. Based on your questions 1, 2, 4, 5 and 8 this may be a deal-killer for you.
If you can code your Flash application to manage itself with HTTP/HTTPS connections then using ASP.Net to build out web services that your app can use is a decent option. IIS is mature and can scale out nicely (q's 6 & 7). It also has an application-level store that you can use (q3). Of course each server in a farm would have its own copy.
It's possible to implement your own server in C# with a lot less effort than in C++, but it's still a time consuming endeavor. The TcpListener class in System.Net.Sockets would be the starting point if you chose to go that route.
I would go with Client/Server .NET application, not ASP.NET as it's out of its "scope".
Unless one of the demands is to have public website?
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I have an existing standalone application which is going to be extended by a 3rd-party, using a network protocol. The capabilities are already implemented, all I need is to expose them to the outside.
Assuming the transport protocol is already chosen (UDP), are there any resources that will help me to design my application protocol?
There seems to be a lot of information about software design, but not on protocol design.
I've already looked at Application Protocol Design.
See Jabber protocols design guidelines and RFC 4101. Although it is aimed at making RFCs more easy to understand to reviewers, this RFC provides some interesting advices.
Have you looked at Google Protocol Buffer? It seems like a good way to resolve this issue.
You can create an endpoint that communicates with your existing app and then responds from 'outside' using the protobuffer protocol. It's binary, so it's tiny and fast and you don't have to write your own protocol manager, 'cause you can use the Google ones. The downside is that it has to be implemented on both sides of the system (on your 'server' side and on the consumer/client side).
Another recommendation for protocol buffers - nice tight binary with little effort. Note, however, that while the binary protocol is well defined, there isn't yet an agreed RPC standard (several are in progress, tending to lean towards TCP or HTTP).
The spec makes it very easy to have the client and server in different architectures, which is good - plus it is extensible.
Caveat: I'm the author of one of the .NET versions, so I may well be biased ;-p
First off, UDP is primarily a one-way broadcast transport method. Also, it is potentially lossy, so you need to be able handle missing packets and out-of-order packets. If you need any level of reliability from UDP, or require two-way connections, you will end up needing just about everything from TCP, so you might as well go with that to start with and let the network stack take care of it.
Next up, if your data is potentially larger than a single IP packet then you will need some way of identifying the start and end of each packet, and a means of handling illegal or corrupt packets. I would recommend some kind of header with packet length, some kind of footer, and maybe a checksum.
Then you need some way of encoding the messages and responses. There are many RPC protocols around. You could look at SOAP, or design a custom XML-based protocol, or a binary one.
You should really think hard about whether you really want to design, document and maintain your own protocol or use something that is already existing. It is probable there is already a documented protocol that matches your needs. Depending on what you are doing it will probably look overkill at first and implementing all the spec will look tedious and a lot less fun than writing your own but if you intend for your application to still be actively developed in a few years it should save you a lot of time and money to use something that already exist and is known by third parties. Besides, if you can use an existing library for that protocol, the implementation part should be a lot faster.
Designing new protocol is more fun than implementing one but less than maintaining one as you have to live with all the defects. No protocol is perfect but if you have never designed one you can be assured you will make more mistake designing it than the people who designed the existing well known protocol you could use instead.
In short, leverage what already exists whenever possible.
If you're choosing XML keep in mind that you will have a giant overhead of markup.
A simple binary protocol will also be need not so much ressources to parse compared to xml.
If you do not want to build your protocol from ground up, you should take a look at SOAP. Support varies for different programming languages, but cross language communication is explicitly encouraged.
Unfortunately UDP and SOAP seem to have stuck in its infancy, HTTP is most commonly used.
I have an existing standalone application which is going to be extended by a 3rd-party, using a network protocol.
It would help to know a little more about what your program does and what the nature of these 3rd party extensions are. Maybe some rationale for using UDP?