Does the Corda Enterprise have an Information Broadcast solution?
If it does not currently exist, will it be possible in the future?
The quick answer is, of course you can perform "information broadcast" and it can do so in exactly the same way that other DLT platforms do. In fact, it makes no sense that you can't broadcast with Corda!
This question comes up a lot, probably because there is some marketing material which says that Corda messaging happens on a peer to peer basis and that "there is no broadcast". What this actually means is that there is no gossiping of transactions with Corda. This is a good thing because it means peers have fine-grained control over which other peers can see their transactions.
To send a message to a peer on a network, you must know where the recipient can be reached. As such DLT/blockchain platforms maintain a list of peers. Platforms like Bitcoin, have a list of peers bundled with the software to bootstrap the network. This list can grow as more peers are discovered. With Corda, this is currently done through the network map service. Corda nodes can query their local cache of the network map to get a list of peers on the network.
If you want to broadcast a message to all peers on the network or a sub-set of peers on the network then it follows that you can iterate through the set of peers that you which to send a message to and send them the message. Easy. Note there is no gossiping here. It's simply just a bunch of unicast messages. You can do this asynchronously, as well.
It is also possible to facilitate gossiping of messages with Corda. In section 12 of the technical white paper, there is a concept mentioned called data distribution groups or clubs. You can think of a club as a directed minimum spanning tree of nodes on a network, which might look something like this:
As such, a node can start a club, then invite others to it, and so on. Members of the club can send a message to the club and it will be forwarded on to all the others. Referring to the image above, if node one publishes a message to the club, then all the other nodes will receive it.
I've implemented a prototype of this here. It's a feature that we plan to roll out in the near future.
Probably worth noting that most networks don't default to broadcasting or multicasting because it makes them a lot slower (if you look into the original history of the Internet, for example, you'll see that multicast didn't exist).
Broadcast platforms have a lot of problems in that senders typically don't know if the recipients have received those messages or not, so it's not at all unusual to find that when some systems talk about "broadcasting" they actually do multiple unicasts instead.
The Corda approach means that there's guaranteed delivery of the messages sent to all the relevant parties. As a point of comparison, even though Wi-Fi networks support multicast messages at L2, most access points will prefer to convert L3 (IP level) multicasts into a series of point-to-point L2 messages as these will be delivered reliably (the receiver ACKs the messages).
It's not that hard to build a gossipy sort of design on top of Corda's messaging. We did this for project Ubin phase 2a in 2017.
Related
I was wondering if DDS could be used over the internet, and if it would be a good choice for online gaming.
I have seen on the RTI website that they support WAN, but does that mean I can subscribe to a topic from another participant that is on the other side of the world?
What would happen to the QoS guarantees if this was the case?
Thanks.
Disclaimer: I work on OpenDDS full time, but have no experience in networked games programming.
A internet-enabled DDS could be used for connecting game clients. Whether or not it's a good idea is something I can't answer at the moment with no specific information, but the QoS part is a good question. In OpenDDS, as far as I'm aware, we try to adhere to the QoS defined by the user as if it was a normal RTPS connection. This means using it over the Internet might require some tuning of the QoS depending on what QoS you want to use. For example if deadline QoS was being used on a local network, the time period might have to be relaxed given the greater latency of the Internet.
For OpenDDS, internet-enabled RTPS is described in Chapter 15 of the OpenDDS's Developer's Guide: http://download.objectcomputing.com/OpenDDS/OpenDDS-latest.pdf. In addition to using ICE to overcome NATs, we also have a feature called the RTPS Relay to enable connections when a client can't use ICE.
I'm not familiar with what specific capabilities RTI Connext here has but as far as I'm aware they are similar, in that they use ICE as well. Also it should be noted that internet-enabled RTPS is not standardized, so the Connext and OpenDDS wouldn't be able to be talk over WAN.
OpenDDS would only be appropriate for games in very constrained environments because of the bandwidth requirements. If all users are on the same LAN then the UDP multicast approach that RTPS uses would be effective for a peer-to-peer game architecture. However, if remote users are added, then the requirement of every peer having to send every update directly to every other peer will very quickly explode the bandwidth requirements.
Given that the RTPS relay is already another application that needs to be run, a game server that collates updates from peers and sends world state would be far more effective for cases where users are not all on a single LAN segment.
We are working on a project to create a V2V ad-hoc network between vehicles. Basically we are deploying Raspberry Pis in vehicles and we are using Xbee modules and Zigbee protocol in order to exchange data between vehicles if they are near from each other.
Our ad-hoc network is completely dynamic and decentralized (We cannot have any static nodes in the network). Our problem is that the topology of the mesh network created by the Zigbee protocol requires a coordinator to be always in the network and the network will fail in case this coordinator failed down.
It seems that using Zigbee protocol requires knowing the topology of the network beforehand.
We do not know whether it is feasible to create our dynamic network using the Zigbee protocol without knowing the topology of the network beforehand and without the requirement of the coordinator. Can we have more than 1 coordinator in the network to overcome this problem? Thanks in advance.
Is there a particular reason you are looking to use Zigbee? if you are after a truely decentralised network you would be better off using either a different protocol (one that has no need for a coordinator) or by defining your own using basic RF modems (which is a lot more complicated)
however XBee modules are configurable by AT commands, with a bit of work you could probably set them up to periodically change mode (when it is running as coordinator) to check for other coordinators and if it finds one then stay as a standard node, and if the coordinator drops out (stops replying) then steps up to be coordinator.
this approach would require you to solve a few issues though:
how do the remaining modules in the network decide which becomes coordinator
how often would the coordinator scan for other coordinators in order to effect a reasonable response time, yet not disrupt data flow
i would suggest that you provide a bit more information relevant to the question such as:
how much data is being transferred?
how often is this data being transferred?
how are you planning to define where the data gets sent (addresses? or multicast to everywhere?)
hopefully this helps you in the right direction, but in the mean time i'd suggest you look at the XBee datasheets for the AT commands and what is possible (has been a while since i used them)
James
Digi sells XBee modules that implement protocols other than ZigBee. Both the plain 802.15.4 module and their proprietary DigiMesh module would be possible candidates for your project if you don't need to be ZigBee-compliant.
I think that you could send broadcast messages with 802.15.4.
With DigiMesh, all nodes are of the same node type. But I do not know how well it will handle having networks come together and fragment on a regular basis. You could contact Digi's technical support or sales support teams to see if they can provide any guidance.
Are there any libraries which put a reliability layer on top of UDP broadcast?
I need to broadcast large amounts of data to a large number of machines as quickly as possible, and generally it seems like such a problem must have already been solved many times over, but I wasn't able to find anything except for the Spread toolkit, which has a somewhat viral license (you have to mention it in all materials advertising the end product, which I'm not sure our customer will be willing to do).
I was already going to write such a thing myself (because it would be extremely fun to do!) but decided to ask first.
I looked also at UDT (http://udt.sourceforge.net) but it does not seem to provide a broadcast operation.
PS I'm looking at something as lightweight as a library - no infrastructure changes.
How about UDP multicast? Have a look at the PGM protocol for which there are several commercial and open source implementations.
Disclaimer: I'm the author of OpenPGM, an open source implementation of said protocol.
Though some research has been done on reliable UDP multicasting, I haven't yet used anything like that. You should take into consideration that this might not be as trivial as it first sounds.
If you don't have a list of nodes in the target network you have no idea when and to whom to resend, even if active nodes receiving your messages can acknowledge them. Sending to a large number of nodes, expecting acks from all of them might also cause congestion problems in the network.
I'd suggest to rethink the network architecture of your application, e.g. using some kind of centralized solution, where you submit updates to a server, and it sends this message to all connected clients. Or, if the original sender node's address is known a priori, then just let clients connect to it, and let the sender push updates via these connections.
Have a look around the IETF site for RFCs on Reliable Multicast. There is an entire working group on this. Several protocols have been developed for different purposes. Also have a look around Oracle/Sun for the Java Reliable Multicast Service project (JRMS). It was a research project of Sun, never supported, but it did contain Java bindings for the TRAM and LRMS protocols.
I've seen and read a lot of similar questions, and the corresponding Wikipedia articles (NAT traversal, STUN, TURN, TCP hole punching), but the overwhelming amount of information doesn't really help me with my very simple problem:
I'm writing a P2P application, and I want two users of my application behind NAT to be able to connect to each other. The connection must be reliable (comparable to TCP's reliability) so I can't just switch to UDP. The solution should work on today's common systems without reconfiguration. If it helps, the solution may involve a connectible 3rd-party, as long as it doesn't have to proxy the entire data (for example, to get the peers' external (WAN) IP addresses).
As far as I know, my only option is to use a "reliable UDP" library + UDP hole punching. Is there a (C/C++) library for this? I found enet in a related question, but it only takes care of the first half of the solution.
Anything else? Things I've looked at:
Teredo tunnelling - requires support from the operating system and/or user configuration
UPnP port forwarding - UPnP isn't present/enabled everywhere
TCP hole punching seems to be experimental and only work in certain circumstances
SCTP is even less supported than IPv6. SCTP over UDP is just fancy reliable UDP (see above)
RUDP - nearly no mainstream support
From what I could understand of STUN, STUNT, TURN and ICE, none of them would help me here.
ICE collects a list of candidate IP/port targets to which to connect. Each peer collects these, and then each runs a connectivity check on each of the candidates in order, until either a check passes or a check fails.
When Alice tries to connect to Bob, she somehow gets a list of possible ways - determined by Bob - she may connect to Bob. ICE calls these candidates. Bob might say, for example: "my local socket's 192.168.1.1:1024/udp, my external NAT binding (found through STUN) is 196.25.1.1:4454/udp, and you can invoke a media relay (a middlebox) at 1.2.3.4:6675/udp". Bob puts that in an SDP packet (a description of these various candidates), and sends that to Alice in some way. (In SIP, the original use case for ICE, the SDP's carried in a SIP INVITE/200/ACK exchange, setting up a SIP session.)
ICE is pluggable, and you can configure the precise nature/number of candidates. You could try a direct link, followed by asking a STUN server for a binding (this punches a hole in your NAT, and tells you the external IP/port of that hole, which you put into your session description), and falling back on asking a TURN server to relay your data.
One downside to ICE is that your peers exchange SDP descriptions, which you may or may not like. Another is that TCP support's still in draft form, which may or may not be a problem for you. [UPDATE: ICE is now officially RFC 6544.]
Games often use UDP, because old data is useless. (This is why RTP usually runs over UDP.) Some P2P applications often use middleboxes or networks of middleboxes.
IRC uses a network of middleboxes: IRC servers form networks, and clients connect to a near server. Messages from one client to another may travel through the network of servers.
Failing all that, you could take a look at BitTorrent's architecture and see how they handle the NAT problem. As CodeShadow points out in the comments below, BitTorrent relies on reachable peers in the network: in a sense some peers form a network of middleboxes. If those middleboxes could act as relays, you'd have an IRC-like architecture, but one that's set up dynamically.
I recommend libjingle as it is used by some major video game companies which heavily relies on P2P network communication. (Have you heard about Steam? Vavle also uses libjingle , see the "Peer-to-peer networking" session in the page: https://partner.steamgames.com/documentation/api)
However, the always-work-solution would be using a relay server. Since there is no "standard" way to go through NAT, you should have this relay server option as a fall-back strategy if a connection has to be always established between any peers.
I want to develop simple Serverless LAN Chat program just for fun. How can I do this ? What type Architecture should I use?
Last year I have worked on TCP,UDP Client/ Server application Project.It was simple (Server listens to certain port/socket and Client connect to server's port etc..) But I have no idea about how to develop "Serverless" LAN Chat program. How can I do this? UDP,TCP,Multicast,Broadcast? or Should program behave like both server and client?
The simplest way would be to use UDP and simply broadcast your messages all over the network.
A little bit more advanced version would be to only use the broadcast to discover other nodes in the network.
Every node maintains a list of known peers.
Messages are sent with TCP to all known peers.
When a node starts up, it sends out an UDP broadcast to discover other nodes.
When a node receives a discovery broadcast, it sends "itself" to the source of the broadcast, in order to make it self known. The receiving node adds the broadcaster to it's own list of known peers.
When a node drops out of the network, it sends another broadcast in order to inform the remaining nodes that they should remove the dropped client from their list.
You would also have to consider handling the dropping out of nodes without them informing the rest of the network.
The spread toolkit may be a bit overkill for what you want, but an interesting starting point.
From the blurb:
Spread is an open source toolkit that provides a high performance messaging service that is resilient to faults across local and wide area networks. Spread functions as a unified message bus for distributed applications, and provides highly tuned application-level multicast, group communication, and point to point support. Spread services range from reliable messaging to fully ordered messages with delivery guarantees.
Spread can be used in many distributed applications that require high reliability, high performance, and robust communication among various subsets of members. The toolkit is designed to encapsulate the challenging aspects of asynchronous networks and enable the construction of reliable and scalable distributed applications.
Spread consists of a library that user applications are linked with, a binary daemon which runs on each computer that is part of the processor group, and various utility and demonstration programs.
Some of the services and benefits provided by Spread:
Reliable and scalable messaging and group communication.
A very powerful but simple API simplifies the construction of distributed architectures.
Easy to use, deploy and maintain.
Highly scalable from one local area network to complex wide area networks.
Supports thousands of groups with different sets of members.
Enables message reliability in the presence of machine failures, process crashes and recoveries, and network partitions and merges.
Provides a range of reliability, ordering and stability guarantees for messages.
Emphasis on robustness and high performance.
Completely distributed algorithms with no central point of failure.
Apples iChat is an example of the very product you are envisioning. It uses Bonjour (apple's zero-conf networking protocol) to identify peers on a LAN. You can then chat or audio/video chat with them.
I'm not entirely sure how Bonjour works inside, but I know it uses multicast. Clients "register" services on the LAN, and the Bonjour protocol allows for each host to pull up a directory of hosts for a given service (all without central management).