I have been playing with Contiki for some time now and have tried out various examples and wrote my own for both the simulation environment and the real hardware. I have only been experimenting with self contained networks that, for example, measure the temperature difference between two nodes and then communicate that data with other devices (PCs) over plain text RS232 link, blink LEDs, and such simple stuff.
Now I want to make a more complicated system where instead of just forwarding the data in plain text to be read on a console I would forward it to an application that would in turn post it to some sort of web service and, vice versa, receive data from web service to be delivered to nodes on the network. There is quite a lot of examples and tutorials describing this kind of setup but all of them (as far as I am aware) are focusing on the IP(v6) stack and SLIP to achieve this. The problem with this is that I have a really lousy programmer and uploading of a 50 kB image takes about 1.5 mins so the development cycle is pure hell. I am also out of luck with simulation since my platform is not really supported at the moment.
That's why I decided to try out the Rime stack, image size is 1/3 of the IPv6 and the development cycle is somewhat acceptable now (I really should get a decent JTAG programmer...) Meanwhile, I am having a bit of trouble wrapping my head around this new setup with a different network stack on which there is very little information around. Although it is pretty easy to understand by itself, I am not sure how I would go about connecting a Rime network to an IP network and if it was even possible or advised/intended by it's designers.
I have some ideas in my head, ranging from ad hoc communication over a serial link between a server application running on the PC and the collector node, to a real Rime border router that is certainly outside of my league, for now.
How would you go about it? It would certainly work for my simple experimental case to just have a collector node that gathers the data from the Rime network and sends the aggregated data over a serial connection to the custom application that does the rest of the magic, but, I wouldn't want to be the guy that reinvented the wheel and I am quite sure that Rime wasn't designed to be used in a vacuum so there must be at least an advised way of doing this?
Rime is a really simple stack (By simple I mean few functionnality). But it's quite quicker for simple task.
You need to program the Rime stack on your gateway. Thus, your board and the gateway can communicate with the same stack. So now you have the data send on your gateway. The gateway now can send the data with IP to whoever you want.
If you want more technical detail, then edit your question with more specific technical context.
Btw JTAG is a must have. (for industrial application)
Edit : An other solution is to simply send your data from your board to your gateway in broadcast. Then the gateway take the data and interpret it. The cons of this method is you have to somehow be sure that your gateway interpret only the data of your board (not of others board)
Related
My knowledge about network programming is limited, so, all the comments are more than welcome. Essentially my question boils down to the following question:
Q1. Is there really such a thing as decentralized asynchronous cross-platform peer-to-peer communication?
Let me explain myself.
If we have two http servers running on computers with actual IP addresses, then clearly the answer is yes, assuming one writes a protocol for the interaction.
To go one step further, if one of them (or both) is (are) behind a router, then, with port forwarding the communication can still be established. However, here the problems start because if someone wants to run such a server on the background, say in a mobile phone, the app that is relying on this server really works when one is at home (we can not really expect to request port forwarding everywhere we go).
But even beyond that,
Q2. do mobile phones obtain an actual IP address from telecommunication companies when someone is not using a wi-fi?
If this is true, then clearly one can have cross-platform asynchronous peer-to-peer communication at the expense of not using wi-fi by running an http server on a smartphone. (I understand that this is not convenient, but it is certainly doable.)
Concluding, the two (perhaps there are more) relevant questions that I can think of are:
Q3. How does Skype really work?
Q4. How does Viber really work?
Based on the answer for Skype, it says: If one of the callee or both of them do not have a public IP, then they send voice traffic to another online Skype node over UDP or TCP.
So, it appears that there is no direct communication in Skype, because they have to use a man-in-the-middle for such a scenario.
Regarding Viber, I could not find a good-thorough answer to this particular question. Do people talk to each other through a Viber centralized server, or, do they establish a direct connection? Of course if they do establish a direct connection, then I really want to know how they manage such a thing since a mobile phone may or may not have a physical address. How is a Viber message routed to my cell phone from a friend of mine even when Viber is not running and I am behind a router?
I guess the answer to Viber is really push notifications, but as far as I can understand, all the variations of push notifications rely on open connections, and then the servers of the applications send the notifications to the clients through such connection(s). So, this approach gives us the feeling that it is asynchronous, but essentially it is not. We are cheating, in the sense that there is a constantly open connection to a server, and moreover, as far as I can understand, the application server has to push the notification through that server. Schematically:
A > Central App Server > Central Server w/ open connection to my cellphone > me
So, this seems to be once again a centralized approach.
Honestly, the only approach that I can think of that is both decentralized and asynchronous (on mobile phones as well) is to run an http server on every platform/device, but this comes at the expense of not using Wi-Fi and assuming that a telecommunication company really assigns a physical IP address to every mobile phone (which I do not know if it is true, do you?).
What about WASTE, darknets, F2Fs, etc? Do they offer advantages in the sense of a more direct asynchronous communication between some interested parties? Are there real-world applications (also including mobile phones) using such approaches for communication.
Really, this is not the actual problem that I would like to work on, but I would like to know what the state of the art is so that I can figure out how I can proceed from there. So, all comments are really more than welcome. If you have references for the state of the art I would like to know about them as well, but a brief description would also be nice.
I appreciate all your time and effort in advance.
You asked many questions, here is the beginning of the answers:
Q1: Yes. For example, take BitTorrent's very successful 10 million+ node network. Aside from the bootstrapping process, the protocol is entirely decentralized and asynchronous. See here for more info.
Q2: Yes! Go to www.whatismyip.com on your mobile telephone, and you will see your assigned IP. However, you are likely to be very filtered (e.g: incoming traffic on port 80 is likely to be blocked).
Q3: It has elements of P2P and clever tricks to get around NAT issues - see here for more info.
Q4: I don't know.
I'm putting together a small 4 node cluster on which I'm going to be running storm. I have a few questions about the networking side of things. First off all the computers are equipped with gigabit ethernet however the hub that I currently have only goes up to 100 megabits. Should I upgrade my hub? Or will the performance gain be negligible? Second I read on a few sites that a hub is not the best piece of hardware to use that a switch would be better for my purposes. I'm trying to use Storm to have one machine pull data down from the internet and then pass it off to the others for processing. Would a switch or hub be more useful? Thanks for all your help folks.
A Router can allow for serious networking capabilities, it's also oftentimes overkill. With only 4 machines you're probably much more likely to want a Gigabit Switch instead: sold in stores oftentimes under the name Gigabit Router -- which is technically a lie as it's usually a Bridge (Hub or Switch, Networking has a lot of overloaded names). Router are many times more expensive than Switches if you have difficulty identifying between the two from just marketing names. A hub on the other hand is oftentimes a dumb Switch with less capabilities (and sometimes speed penalties in high data flow situations).
The question as to if you need to upgrade is dependent on where you bottleneck is. Is the data you're sending large? Do your cluster computer spend a lot of time computing instead of receiving data? First determine if your networking speed will be your bottleneck, then decide if you should upgrade that bottleneck. If you're worried about network speed but aren't 100% sure it will be a bottleneck, a cheap 1 Gigabit Switch won't cost you much and will almost certainly meet you're needs.
Also note that if you're data needs to first come over the internet (isn't generated on your side of the network) you're bottleneck will almost certainly be your internet connection before your local network.
So essentially, profile your problem before making a choice.
I am sorry if my question is obvious, but i need the expert suggestion/views, i want to test my Client/Server game for which currently, i am using localhost same machine for both Client and Server.
that's why i not getting any fluctuation in data and measure idea of performance, and in other parameters, what i wanted to ask to have a real world scenario:
if i create a little network with two computers or
if check that on LAN on which i am or
ARE THESE THREE CASES (localhost included) ARE EQUIVALENT? or
I really need to test that on different LANs to have reliable testing data and realistic data,
How these different network setups will influence the testing process?
Can somebody please suggest, which could be the ideal way or enough for testing?
which above setup will give me more up and downs in number with LEAST setup/implementing efforts.
Note: The game is suppose to play on the LAN but it is capable of more.
Thanks,
Jibbylala
P.S: i m newbie in network stuff so if u used the wrong terms, vocabulary pardon me
emphasized text
You will want to test your application under different situations. For example, test it using a small LAN where you only have one switch between the two computers. That will ensure that you can, in fact, connect and play over a simple LAN. Then, test different LAN connections such as a slow link (turn a network card down to 10 Mbps), on a wireless LAN, and if possible even a larger or corporate-type LAN. The more testing you can do about different situations, the better. Testing on just your localhost will not be enough.
Are there any libraries which put a reliability layer on top of UDP broadcast?
I need to broadcast large amounts of data to a large number of machines as quickly as possible, and generally it seems like such a problem must have already been solved many times over, but I wasn't able to find anything except for the Spread toolkit, which has a somewhat viral license (you have to mention it in all materials advertising the end product, which I'm not sure our customer will be willing to do).
I was already going to write such a thing myself (because it would be extremely fun to do!) but decided to ask first.
I looked also at UDT (http://udt.sourceforge.net) but it does not seem to provide a broadcast operation.
PS I'm looking at something as lightweight as a library - no infrastructure changes.
How about UDP multicast? Have a look at the PGM protocol for which there are several commercial and open source implementations.
Disclaimer: I'm the author of OpenPGM, an open source implementation of said protocol.
Though some research has been done on reliable UDP multicasting, I haven't yet used anything like that. You should take into consideration that this might not be as trivial as it first sounds.
If you don't have a list of nodes in the target network you have no idea when and to whom to resend, even if active nodes receiving your messages can acknowledge them. Sending to a large number of nodes, expecting acks from all of them might also cause congestion problems in the network.
I'd suggest to rethink the network architecture of your application, e.g. using some kind of centralized solution, where you submit updates to a server, and it sends this message to all connected clients. Or, if the original sender node's address is known a priori, then just let clients connect to it, and let the sender push updates via these connections.
Have a look around the IETF site for RFCs on Reliable Multicast. There is an entire working group on this. Several protocols have been developed for different purposes. Also have a look around Oracle/Sun for the Java Reliable Multicast Service project (JRMS). It was a research project of Sun, never supported, but it did contain Java bindings for the TRAM and LRMS protocols.
We're about to design an inhouse industry network consisting basically of the following: 1 server connected via wire to up to 100 proprietary RF access points (basically embedded devices), which each can be connected via radio to up to 100 endpoint embedded devices. Something like this:
Now, I'm wondering about some design decisions that we need to take and I'm sure there are plenty of similar designs out there and lots of folks with experiences of them, both good and bad. Maybe you can chime in?
All endpoint devices are independent and will communicate their own unique data to the server, and the other way around. The server therefore needs to be able to target each endpoint device individually. Each endpoint device pairs itself with 1 access point and then talks a proprietary RF protocol to it, TCP/IP is not an option there.
The server will know which endpoint device is paired with which access point, so when the server needs to talk to an individual endpoint device, the communication must go through the paired access point. Hence, the server needs to directly address the access point.
Question: Considering the limited resources available in the proprietary access point, is TCP/IP between server and access point recommended for this scenario? Or would you suggest something entirely different?
I find the diagram confusing:
If this isn't its own network and the server to AP link is running on your internal company network, there isn't really an option, there must be a TCP/IP stack on the AP.
If this is its own isolated network then what is the router for?
If this is, in fact, its own isolated network then you are right, there really isn't a need for the Ethernet connectivity at all. The overhead you will see on the wireless is huge, your no overhead ideal data rate is 250kbit/sec, running ZigBee on 802.15.4 # 2.4ghz point to point your real data throughout is usually around 20kbit/sec. A custom protocol should be able to obtain lower overhead but this would need to be defined.
If I were designing this I would choose a SoC for the AP that had on board 802.15.4 and CAN (Controller Area Network). Depending on size and data rate just get a PCI CAN card for the server and connect it up, use something like DeviceNet as your protocol layer for server to AP communications. This can be expanded by using CAN switches and repeaters. CAN is used all the time in industrial automation, a little googling can find you example of tens of thousands of nodes used in some manufacturing plants.
There are small TCP/IP stacks, for example LwIP.
You didn't mention the amount of data to be communicated, or bandwidth considerations?
A 3rd party TCPIP stack targeted at the 8051 would simplify all the networking issues with connecting 100 units. You probably will still end up with a proprietary protocol that sits on top of the tcpip stack but then it is just simple point-to-point communication between the server and each end point.