I have a legacy setup that utilizes a Zigbee Controller within a LAN. We have an application that communicates with that controller over UDP to send commands to Zigbee devices.
We'd like to be able to send commands from a cloud-based server to a controller running within a LAN without forwarding a port on the firewall.
So ultimately: Our app in cloud => (something) => Local Firewall => LAN => (something) => Zigbee Controller
I'm wondering what the best (somethings) are to accomplish this. In the end, I want to take our current UDP messages that are being sent on (some) port, translate them to packets that can be sent to a device inside a LAN on port 80 which then re-sends them over the proper UDP port. I'm assuming this is somehow possible, but I'm not sure of the right approach / device.
Any suggestions?
You may try "hole punching" over UDP. The basic idea is to let the client (your ZigBee controller) send a periodic message to the cloud over UDP to maintain an active connection. Your cloud server will be able to send messages to the controller, without port forwarding and, because of UDP, without having to maintain an active session like it does over TCP.
Related
if when i try to connect with webRTC from outside the local network, it uses a STUN server to exchange info like ip address, port..., and then once the two clients know each other they will directly connect, but the thing is, if any router by default blocks any port that's opened exept for 80, then how do the two clients connect, on what port?
WebRTC is an open framework and does not dictate how the connections are established between the communicating peers. Establishing the connection itself involves the following.
1-The peer discovery is made via Signaling for doing SDP exchanges and exchanging remote host identities. The Signaling can be done over any transport – UDP/TCP, any protocol and via any standard (SIP/XMPP) or custom application level protocol over HTTP/WebSockets etc. The choice of Ports for signalling traffic is left open to the application developer.
2-Once the peer discovery is complete, and the PeerConnections are established at each endpoints, the media data can be sent. Depending on the network topology of the Peers (whether they are in the same host network, or they are in different network (behind Nated IP address), they can choose to connect such that the media traffic is sent over UDP or over TCP. For a more protected environment, the peers may even need a TURN server to relay the media traffic on Peers behalf. Here again, the choice of Media Ports is left open and can be decided via the end application.
I found this on web I hope its helpful
I'm trying to build a peer-to-peer game, where each player is both a server and a client with tcp sockets. The sockets can connect fine when I'm using local ip:s, but of course fails when I'm trying to use external ip:s.
But I'm thinking that the players should be able to connect to each other if they just knew the external address + port that the router assigns to them.
Setting up port forwarding is out if the question since I don't have access to the players routers.
I'm thinking of having a server in between, just to be able to read the external address and port of the players, and tell the other player about it so that it can connect.
But I haven't found any info anywhere if that's how port forwarding works. If computer A makes a request from a local address and a port to the server, and the router assigns this address + port to an external address + port, and the server tell computer B which address + port to use. Can computer B use that external address + port to connect to the computer A and start a tcp socket with it? Is there any way to know that this external address + port stays the same when another computer makes a request against them?
The problem is that most people don't expose their PC directly to the internet. They have a router that has an external address. When you send a packet to their IP address, it is going to their router. The router doesn't know where to forward it to and what port to use without port forwarding.
So, getting everyone to enable port forwarding is out of the question, as it should be. A simpler mechanism is to have a server on the internet that you control. It has a firewall with port forwarding setup. The clients are just clients, they connect to the server on a port and send and receive info about the current status of the game. That way, everyone has real-time updates on their local game engine. Plus, this way is much easier to program and implement.
No, TCP doesn't work like that.
The source port that has been used to talk to the rendezvous server will be transient and specific to that particular initial TCP socket connection and can only be used as a destination for return traffic on the same connection from the rendezvous server, and can't be used by a third party to make new inbound connections.
The typical (only?) practical solution when NATs are involved and port forwarding is not available is to have that central server relay all messages bidirectionally between the peers.
Hole punching is what I was looking for.
https://en.wikipedia.org/wiki/Hole_punching_(networking)
I have been looking for an answer to this problem, but I cannot find what I am looking for. I think, perhaps, it is because I lack the knowledge to ask the question in meaningful way.
I have been learning a lot about remote access to devices at home. I know that ISP's change public IP addresses regularly (dynamic IP address). I know that to get around this, one could use a service like "no-ip", etc. Or one could get a static IP address.
What I do not understand is how some of the latest home automation devices are able to be controlled remotely without use of a static IP, or a service like "no-ip". For example, a wifi enabled thermostat, or lighting system.
If the device had a built in server, or client, then I assume that the device could connect to an outside server in a remote location. The user could then also log into that server and send commands to the device. What I don't understand is how commands sent to the device from a cell phone, for example, can reach the home device. Presumably the off site location of the server would have to know the public IP address where the devise is located, and then port-forwarding would have to be set up to allow access to the device.
What am I missing here? Is it possible to create a homemade wifi enabled thermostat, webcam, or other device without using port-forwarding, no-ip, or a static IP?
Well, there are several ways to bypass the inbound connection constraint of NAT protocol. Such as:
A virtual adapter on the device configured to a VPN server that has an inbound port open ready to transfer data. Various open source solutions such as openVPN are considered as great examples for this service over IOT boards like Raspberry Pi, Beagle Bone, etc. These are used as gateways often. Further, they communicate with the microcontrollers over popular IOT protocols such as MQTT, COAP, etc.
Another solution is to create a port forwarding tunnel, since the router won't block the outbound connection. There are various tunneling services that are availble such as localtunnel, ngrok, etc. You could also use a cloud server that has a public IP such as AWS, DigitalOcean, etc. Again as above mentioned point, they can be implemented in the gateways.
Some devices "phone home" to a server so that there are ports open between them and the servers, and the mobile apps just contact the servers. This is the same way your web browser can receive web pages from a web server. If you have a NAT router, the router must open a port from the inside device to the outside server. This is maintained in a NAT table with expiration timers for UDP and session monitoring for TCP.
I am new to network programming and try to understand managing traffic via TUN/TAP interface.
Since I have almost nonexistent system programming skills, and feel confident on Java; I use OpenVPN tun/tap driver and ready made Java binding for it. It works on TAP mode.
As an example application I am trying to imitiate no encryption, no authentication client server VPN application.
I can catch Ethernet Frame packets, but for the routing part, I failed miserably. (I can modify route/arp tables.)
Do anybody know how OpenVPN send packets from client to server, and from server to target. Opening sockets from Java looks like an alternative; but I was hoping that modifying packets(change IPs and/or MAC addresses) and writing back to the virtual tap interface would be enough. Is it so?
Can I inject packets to send other locations, or by default received packet moves towards application layer?
-- Edit:
Scneario
Client Tap0 _____ Server Tap0 ______ Target
Eth0 Eth0
Target: Ping from client, move through tap interfaces, target see only server ip (anonymization)
What I achived so far.
Catch traffic at client tap0 interface.
I coulnt forward traffic at server Tap so to fasten things I used Java socket programming between client-server.
Now I read packets from socket at server, and try to OpenVPN Tap driver's write method to move forward but I am not sure where do I fail. I see packets with tcpdump at server tap0, but they do not pass to server eth0.
My most important question is if I modify packet(ip, mac address) and call write method, is it possible that packet moves forward. (Or does it move to application layer whatever you change??)
Any help would be appreciated.
1. Routing is a Layer 3 (IP) problem and handled by the OS. As for the Ethernet frames on Layer 2, you have multiple options. In any case, you'll have to parse the incoming packets' headers and extract the MAC address, and decide based on the MAC where to pass the packet: To a specific client, all clients (broadcasts) or the local tap interface.
Option 1: On each client, use a tun device, and let the server use a tap device. Assign pseudo MAC addresses to each client, respond accordingly to ARP requests from the server's OS and let the OS on the server take care of the rest. Applicationwise, you'll only have to forward all incoming packets to the tap device and all outgoing packets to the client to which you assigned this MAC.
Option 2: Let the clients choose their own MAC address and forward ARP-requests through the network. The server application has to decide for incoming packets from a client whether to forward the packet to a client, or send it to the local tap device if the address matches the local device's MAC.
In both cases, clients pass all packets from their local tun/tap device to the server and vice versa.
2. You can do almost anything. A packet is only "received" when you decide to write it to the tap device, and you can of course temper with any packets, or inject new ones, ...
As a final comment, I've found that toying with tun devices is conceptually simpler, because they work on Layer 3. You'll have to open a tun device on the server for each client, but within your application you'll have to do nothing but to forward anything coming from the device to the single client, and vice versa.
I was wondering that how application like skype ( a popular chat client ) works in local network with one router, How it can listen on particular port?
for example:=
In one network A and B are two machines running skype , gateway of both is G1,
now how A and B will have same IP on internet that is of G1, but how can they ensure that they are listening on different ports? How can they ask to router G1 for unique port.
I want to make a simple text chat server on linux. How can I have connections between two different computers in two different networks?
Solution to your problem is to have a forwarding server somewhere in the net.
Different programs use different means to connect to each other. But every chat server, including Skype, has a server, which forwards data or information about subnet IP/port availability.
There are two types of clients: "listening" clients and "passive" ones. Listening clients have direct access to Internet via router port forwarding, and "passive" ones have to use additional tricks to get their hands on external data, line external servers or additional ports to listen.
The point is, not clients connect to each other, but they connect to a server, which then connects back to them to verify they are available, and, if at least one of them is not firewalled, direct another on to connect to the first one, excludint itself from further communication. And if both are firewalled, then is has to forward their messages through itself.
Host Discovery
Manual discovery, client A knowns who client B is
Discovery through broadcast UDP which is used by lot of games for LAN play. A client sends out a packet to the broadcast address for their subnet. The peers can choose to pick up this broadcast and respond. The downside is that this is limited to the current subnet. The more general INADDR_BROADCAST (255.255.255.255) works for all subnets on the local-link, but it cannot be routed, so won't work over internet (this is what DHCP auto-configuration uses).
Discovery through a central (Rendezvous) server. Each individual client knows the address of the server, and the latter informs them about each other. This technique is used by IRC, Voip, IMs and by most 'peer-to-peer' networks.
Communication
After the initial discovery is done you want to be able to talk to eachother. On the internet this can get tricky. Most people nowadays have their own router and sit behind a NAT, so direct connections are impossible.
Using a Rendezvous server, you can possibly talk to each other using the server itself. client A tells the server what to say, and it in turn tells client B, since both clients have an outbound connection to the server.
It is possible for the clients to talk to each other without the server proxying. This requires either DMZ, port forwarding or UPnP. DMZ will basically forward all incoming connections on all the ports to a given local IP. Port forwarding only forwards certain ports to local IPs. UPnP is a bit more advanced, the client requests that the router temporarily forwards a port to it, and you tell the other client via the rendezvous server where to connect.
Chatting app implementation
The easiest solution to your problem is most likely to use a central server, which is known by all the clients, that proxies host discovery and possibly the communication between the clients. If you want the clients to communicate directly, you can just proxy host discovery, and then let either DMz, manual port forwarding or UPnP do the rest.
Another solution would be to just have direct communication through NAT traversal techniques discussed above, and do manual host discovery.
Yet another solution would be to use a public webserver and 'abuse' its ability to insert content to chat with each other.
You need a central UDP Rendezvous Server.
After the initial connection from the client to the server the UDP clients can be redirected to talk to eachother directly even if firewalled.
The trick is to open an UDP connection from the inside.
Check out Real-Time Media Flow Protocol and how they use it.
Check out UDP Hole Punching
alt text http://labs.adobe.com/technologies/stratus/images/p2pvideo_250x215.jpg
Traditional NAT servers replace the source address and port with the address and a random port number of the external interface of the NAT server. This works well for simple protocols such as HTTP and SMTP, but it can create problems for more complex protocols that require multiple response ports on the external interface of the NAT server. NAT servers also aren’t aware of information stored in the data portion of the application layer header without the help of NAT editors and similar software fixes.
Windows XP’s answer to these problems is NAT Traversal, which can automatically allow the UPnP-enabled NAT client application to communicate with a UPnP NAT device. NAT Traversal provides methods to allow the UPnP client to learn the public IP address of the NAT server and to negotiate dynamically assigned port mappings for UPnP NAT client applications.
NAT Traversal features can be built into any hardware device or software application. Applications that commonly cause troubles for NAT devices but work well when UPnP-enabled include the following:
Multiplayer Internet games
Audio and video communications
Terminal Services clients and servers
Peer-to-peer file sharing applications
When these applications are UPnP-enabled, access through the Windows XP ICS allows them to work seamlessly.
Unless A and B are actually "listening" to the responses to outgoing requests, your router will need to be cofigured to forward the relevant port numbers to the relevant hosts. This isn't something that you can request in the code, it's something you need to configure on the router itself.