Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
Given two computers attached to the Internet that know nothing about each other before hand, is it possible for one computer to be able to broadcast a message so that the second computer could receive it and respond?
I know UDP broadcast exsits, but I believe that those are generally filtered by the ISP before it reaches the true Internet. Is this true?
The current best way to achieve a multinode network without centralized coordination is through the use of Distributed Hash Tables. That link explains a bit and links to various implementations you can leverage.
That said, you still need each machine to coordinate with at least some peers. It's just that you don't need it to coordinate with a central server. A solution using a central server that know both (all) participating machines will also work, but imposes further restrictions on anonymity and scalability, just remember what happened to Napster.
You need an intermediate third party that they both know, that could distribute messages directed towards it in a broadcast-like fashion.
A solution for this problem (where none of your peers know the final address of the other) could be relying on IM protocols.
In particular, the XMPP protocol is extensible, open and used by many providers such as Google Talk. Libraries exist for most languages and it has the plus of being able to work (slowly and going through a 3rd party server) even if both hosts are behind a NAT-box.
If communication must use another channel, you can use XMPP to exchange IP address and then proceed with the standard socket route (but if you encrypt your messages, there should be no problem even going through a 3rd party server - to be true all packets go through untrusted 3rd party routers so you should encrypt anyway if you have sensitive data..).
Hope this helps.
No, you can't broadcast like that over the internet. You need to know which address you want your packets to go to.
A possible solution for you is to use a dynamic DNS service.
Your application would need to know in advance which hostname the other host will be using, but this service would at least get around the fact that you don't know exactly which IP address the other computer is on.
Note that this won't solve the potential issue of firewalls between the two hosts blocking your packets. The only practical way around that is for both hosts to open an outbound connection to a central host which can then relay data between them.
Look at the chord or pastry algorithm. It is an overlay network (DHT based) which has a discovery mechanism involved. It's a P2P (Peer 2 Peer) routing algorithm.
UDP is a dead end - its just a protocol where the order the packets are received is less important and there are issues routing over WANS. You said that you want to connect two computer on the "internet" presumably with the end points moving around etc. The only way is to use a central server as a register/directory. If each end point allso a web service or something and registeres its current IP address and name periodically then the other end point can look up the IP address of the other using this service. (could host your own DNS server and code your end point to register on this DNS?)
One of the problems is that even if you have the IP address what is one or more nodes are behind a firewall or NAT router ? You will need to host a server to proxy traffic. The best example is SKYPE - look into how it works it is documented, very interesting.
The simplist answer might be to jump on the back on an existing service such as messanger, skype, bit torrent, etc.
Simon
If the computers are running Windows, I'd look at using PNRP.
Multicasting is also a possible solution. It's certainly feasible in a corporate network
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Firstly, thank you for taking the time out to read this post.
I'm looking to develope a TCP/IP enabled device using the Microchip PIC18 or PIC32 family of embedded microcontrollers with Microchip's TCP/IP Stack. However, my knowledge of networking is pretty basic at the moment, thus the reason for this post.
Can anyone recommend the best protocol to use for my TPC/IP embedded device so that it can communicate with a server in a data centre? My intention is to have the embedded device located at a remote location somewhere over the internet, where the server can communicate with the device and download data such as thermometer probe readings to be stored in a database. I would also like the ability for the server in the data centre to be able to reconfigure settings and variables on the remote device should I need to.
My research on protocols so far has lead me to the following options:
SNMP v3 (version 3 due to encryption and authentication)
UDP (though I read this can be unreliable but is fast)
TCP (I'm not too clued up on this yet)
Can anyone offer me advice on the best route to go down? I'm not expecting a detailed answer from you, but I would really like an idea of what topics/protocols to look into and research.
My intent is to deploy many of these embedded devices over the internet where they all send their data back to the server.
I assume that the remote embedded device will have to connect to the server rather than vice versa as the server will have a static IP address or DNS name, whereas the remote device addresses will be unknown.
Any advice on this would be greatly appreciate. Please don't hesitate to ask if I've missed out any key information in this post.
Many thanks.
Rob
* UPDATE *
It was pointed out that I'm probably misusing the term Web Server, so I've amended my post to mention Server in a Data Centre instead. Thank you for pointing this out to me.
If the target is a Web server you don't have any choice. You have to use HTTP, which runs over TCP.
Or else you are misusing the term 'Web server'.
In many ways this depends on your specific requirements. TCP/IP is able to provide quite reliable connections because it provides a means to determine if the client is connected, when they connected and when they disconnected. UDP is connectionless, so the server opens a port and listens for data, but has no automatic connection management, so clients need to explicitly 'tell' the server when they have arrived or are going (this also means you will need to make your own timeout facility).
Also, if you have very limited memory/processing resources, it is worth bearing in mind that UDP is a less 'costly' protocol as it avoids a lot of the overheads TCP incurs due to its inbuilt connection management.
While these are all protocols, they really just handle the connections themselves. You will probably still need to create your own protocol for the management of the data itself. For instance, when you send data over either TCP or UDP, the bytes you send may not all arrive to the server at the same time. This means you need a way of validating each packet you receive to ensure you have it all. This is often achieved with a combination of a checksum and a byte representing the total size of the data sent.
You might also consider mqtt (http://mqtt.org). It is a lightweight messaging protocol. For encoding your messages, you might consider protobuf (https://code.google.com/p/protobuf/)
My knowledge about network programming is limited, so, all the comments are more than welcome. Essentially my question boils down to the following question:
Q1. Is there really such a thing as decentralized asynchronous cross-platform peer-to-peer communication?
Let me explain myself.
If we have two http servers running on computers with actual IP addresses, then clearly the answer is yes, assuming one writes a protocol for the interaction.
To go one step further, if one of them (or both) is (are) behind a router, then, with port forwarding the communication can still be established. However, here the problems start because if someone wants to run such a server on the background, say in a mobile phone, the app that is relying on this server really works when one is at home (we can not really expect to request port forwarding everywhere we go).
But even beyond that,
Q2. do mobile phones obtain an actual IP address from telecommunication companies when someone is not using a wi-fi?
If this is true, then clearly one can have cross-platform asynchronous peer-to-peer communication at the expense of not using wi-fi by running an http server on a smartphone. (I understand that this is not convenient, but it is certainly doable.)
Concluding, the two (perhaps there are more) relevant questions that I can think of are:
Q3. How does Skype really work?
Q4. How does Viber really work?
Based on the answer for Skype, it says: If one of the callee or both of them do not have a public IP, then they send voice traffic to another online Skype node over UDP or TCP.
So, it appears that there is no direct communication in Skype, because they have to use a man-in-the-middle for such a scenario.
Regarding Viber, I could not find a good-thorough answer to this particular question. Do people talk to each other through a Viber centralized server, or, do they establish a direct connection? Of course if they do establish a direct connection, then I really want to know how they manage such a thing since a mobile phone may or may not have a physical address. How is a Viber message routed to my cell phone from a friend of mine even when Viber is not running and I am behind a router?
I guess the answer to Viber is really push notifications, but as far as I can understand, all the variations of push notifications rely on open connections, and then the servers of the applications send the notifications to the clients through such connection(s). So, this approach gives us the feeling that it is asynchronous, but essentially it is not. We are cheating, in the sense that there is a constantly open connection to a server, and moreover, as far as I can understand, the application server has to push the notification through that server. Schematically:
A > Central App Server > Central Server w/ open connection to my cellphone > me
So, this seems to be once again a centralized approach.
Honestly, the only approach that I can think of that is both decentralized and asynchronous (on mobile phones as well) is to run an http server on every platform/device, but this comes at the expense of not using Wi-Fi and assuming that a telecommunication company really assigns a physical IP address to every mobile phone (which I do not know if it is true, do you?).
What about WASTE, darknets, F2Fs, etc? Do they offer advantages in the sense of a more direct asynchronous communication between some interested parties? Are there real-world applications (also including mobile phones) using such approaches for communication.
Really, this is not the actual problem that I would like to work on, but I would like to know what the state of the art is so that I can figure out how I can proceed from there. So, all comments are really more than welcome. If you have references for the state of the art I would like to know about them as well, but a brief description would also be nice.
I appreciate all your time and effort in advance.
You asked many questions, here is the beginning of the answers:
Q1: Yes. For example, take BitTorrent's very successful 10 million+ node network. Aside from the bootstrapping process, the protocol is entirely decentralized and asynchronous. See here for more info.
Q2: Yes! Go to www.whatismyip.com on your mobile telephone, and you will see your assigned IP. However, you are likely to be very filtered (e.g: incoming traffic on port 80 is likely to be blocked).
Q3: It has elements of P2P and clever tricks to get around NAT issues - see here for more info.
Q4: I don't know.
I have a somewhat basic understanding of network programming (and networking concepts in general) from taking a networking course in university a few years ago.
I remember being able to create a simple chat application, where the chat server is used as a central directory aware of which clients are currently online, but once a client knows another client it wants to chat with, the actual messages between them don't need to go through the server. I remember we could only test this over a bunch of LAN machines.
This C# chat program also has several comments mentioning that the program does not work over the internet: http://www.geekpedia.com/tutorial239_Csharp-Chat-Part-1---Building-the-Chat-Client.html
My question is why do these applications not work over the internet when "commercial" chat applications can. Surely, there is some way to make my computer accessible to the outer network even if its IP address is not valid outside the network of the ISP.
I see no problem with the linked-to code. The server doesn't even bind to a local address, which means it will listen for connections on all ip-addresses on the computer. There is however a comment for in the server article where the user changed the TcpListener object creation to bind to a specific address, which means clients only can connect to that specific address.
In the original server design, with using TcpListenet with only a port number, there should be nothing preventing its use on an Internet connected computer, unless there is a firewall blocking access.
Were you aware of networkComms.net and in particular the short chat example demonstrating the functionality here (It's less than 15 lines of code)? This was written specifically for people writing server-client apps in c# and given most of the problems you might come across will already have been solved and it might save you some time. This library is completely plug & play and has no issues working over the internet (as long as you can setup the necessary port forwarding where necessary).
Generally if both of your targets are behind NAT (so no true external ip addresses) and you are unable to configure port forwarding you need to look at 'TCP / UDP hole punching', quite an advanced technique.
First-timer on Stack Overflow here. I'm surprised nobody seems to have asked this question, and I hope this is the right place to ask this. I'm trying to determine if I should expect regular network switches (just simple switches, not routers) to have the capability to isolate local network traffic (i.e. targeted traffic that is directed to another local port in the ame switch) within the switch?
For example, if I have 2 machines connected to ports on the same switch (say, ports 2 and 3) and conversing using a directed, non-broadcast protocol (e.g. TCP), I wanted to make sure the traffic between these 2 machines are not forwarded the the rest of the network outside of the switched subnet.
I'm building a home network and I wanted to build private network "subnets" or "zones" using switches where local subnet traffic does not get forwarded to the "backbone" or the rest of the network. Note that I am NOT trying to block any inbound or outbound traffic to/from/between these "zones", but I just wanted to implement a "need to know" basis for these zones to limit network-wide exposure for localized traffic destined within the same switch. Specifically, I wanted the backbone to have as little unnecessary traffic as possible.
So back to the original question: is it fair to expect any network switch out there to be smart enough not to forward local traffic to the rest of the network? I would expect this to be the case, but I wanted to make sure.
PS: You can assume I have a DHCP/WINS server somewhere on the network that will be assigning IP addresses and the such.
I hope the question makes sense, and any help will be appreciated!
- K.
Short answer: yes, the switch is smart enough (otherwise it would be a hub).
And if you need fancy stuff you might have a look a VLANs.
And I believe this question belongs to serverfault or maybe superuser. That's probably why nobody asked it here :)
I wrote an server application in erlang and a client in C#. They communicate through 3 TCP ports. Port numbers are hardcoded. Now I'd like to do this dynamically. This is my first time doing network programming, so please pardon my inability to use proper terminology :-D
What I would like to do is make a supervisor which would accept a TCP connection from a client on a previously known port (say, 10000, or whatever), then find 3 free ports, start a server application on those 3 ports and tell the client those port numbers so client can connect to the server.
My particular problem is: how do I find 3 ports which are not in use? (clarification: which module:fun() to use to find a free port?)
My general problem is: I'm sure this kind of stuff (one server allocating ports and redirecting clients) is quite common network programming problem and there should be a bunch of (erlang-specific or general) resources about this, but I just don't have the terminology to google it out.
According to the Erlang documentation here, if the Port argument to the gen_tcp:listen/2 function is 0, then the OS will assign any available port to the socket. The latter can then be retreived using inet:port/1 .
You can therefore do something like this :
{ok, Listen} = gen_tcp:listen(0, [Options]),
Port = inet:port(Listen).
just in case you didn't know that - you dont have to allocate new ports for each client, it's perfectly fine to have all clients to connect to same ports
UPDATE:
if there is a reason to allocate new ports for incoming clients then it's far beyond your first "introduction to network programming" program.
separate ports could mean you want to completely isolate environments of different groups of clients. it's comparable to providing completely different IP addresses to connect to. if you want to write a simple ping-pong program - you don't need it. and i honestly believe you will never need to use such solution in your whole life - that's how incredibly rarely it is.
regarding cpu/ports overhead - allocating ports and starting a server that listens to that port is already far bigger overhead than accepting clients on same port.
You need to avoid commonly known ports, ftp, http, smtp etc, But I don't think there is any master list of which ports other software uses that you should avoid. I think your best bet is to come up with a range of ports you want to use. Check at runtime if anybody else answers ( is using the ports ) on the numbers you choose dynamically, if not issue it to the client.