I am trying to create a small academic project in the area of home automation. I explored various protocols and standards in IOT domain and find COAP close to my requirements.
The intent is pretty simple: To have a peripheral at home (like bulb, temperature sensor etc) which act as COAP server and expose certain resources and services. Since COAP uses an addressing scheme similar to web, any client having the address of this server shall be able to talk to this COAP server (i.e. peripheral) and control its attributes.
I have certain doubts about how a client not present in the same LAN as the server, can discover this server and the resources exposed by it. I understand that COAP facilitates resource discovery using CoRE link format but it seems that it primarily uses multicast which can only work within a LAN. I don't want to put a resource directory server at home and various COAP servers registering with it and then this directory server talking to a server sitting in the cloud. I want to have the ability to just install a peripheral at the home which can communicate to a remote client (like COPPER) using wifi home network. But how to discover the COAP server sitting at home from a remote location and then talk to it considering that COAP server will be on a private IP address.
Is this really possible using just COAP?
As you're sitting behind a NAT, that will allow neither direct discovery nor connections; CoAP will not try to magically pierce that.
Popular ways out are:
Use IPv6 (obliviating the use of a NAT) and (at least selectively) disable the firewall. You will still only be able to do discovery locally, but once discovered, you can connect to your devices from anywhere.
Use a VPN to make your mobile device act as if it were in the NAT.
Run a CoAP proxy at the router with a port open to the outside, and tell your mobile device to connect through that proxy. This does not give you local discovery either.
For the proxy and IPv6 cases, you can then still run a resource directory in your local network. That does neither need to connect to nor accept registrations from any cloud service, but it can help you do the very discovery that you'd like to do with multicast but can't because your mobile device is too far away. Both the resource directory and the proxy should be configurable to not talk to devices outside the local network unless they are authenticated.
Make your router accept HTTP requests , and run a HTTP to CoAP proxy in your system, access your CoAP devices via your IP thought HTTP-CoAP proxy.
Related
I am new to openthread I have some interrogations about Thread device connectivity to cloud server.
Cloud server <------------> local internet <-----> Thread network
ipv4 Router(firewall) ipv4 OTBR ipv6
Our products (Thread network) will be built in clients networks which have various internet routeur/firewall and network configurations.
Using UDP(DTLS) to PUT/GET/POST... (CoAP) data on cloud server did you experience any issues with UDP timeout parameters ? Will I need to do hole punching to ensure cloud server can contact end device ?
As i understand it, from cloud server point of view, he can only Rest method on the Border Router CoAP server, as he don't know end device ipv6 and i don't plan to do port forwarding ?
Would allowing cloud server to contact specific thread end device require ipv6 tunnel ?
To finish, if im spouting nonsense, please enlight me about how you build your connection with cloud server :) !
Thank you for reading this post, I hope I was clear.
Best,
Let me try to sort some things.
There are two general approaches:
the clients from your local network starts the communication and the cloud-server answer. The router acts as NAT. In that scenario there are usually timeouts on which the "NAT" rules expires and the traffic from the cloud-server will not be forwarded to a client in the local network.
the cloud server starts the communication. That traffic is sent to your router, and the router and forwards the message to a local network node. This approach requires usually configuration of the router (there are some protocols to do that from your client devices, but even that requires to enable that function). You configure a port on the router to forward the traffic to a specific address+port of your clients. Though this requires either configuration of a lot of ports (for each client one port) or one coap-node, which acts as coap-proxy and configuration for that.
The first approach will end up in a lot of traffic just to keep the NAT open.
The seconds requires either a lot of configuration or a "coap-proxy", where I'm not sure, if you can find a proper implementation.
(By the way, the router may have only a temporary fixed ip-address, e.g. one change peer day. So the second approach requires rare updates of the router's address in your cloud server. And sure, there are some Internet provider, which doesn't offer that your router is reachable, because they add a extra NAT.)
I am making a game in Libgdx where a user(client) will be a host of the server himself
and only one other client can connect to him and play with him(likewise there might be multiple independent servers and pair of clients playing with each other like miniclip games)
but how will the other client locate host-client (or servers) IP?
I am using kryonet
Some guy suggested me to do NAT punching and stuff but I am totally confused.
Please help
(I am complete beginner to networking)
If you want your clients to discover servers just in LAN, you can do broadcasting on the servers and sniffing on the clients. As they are in the same LAN, there should be no need for NAT.
Otherwise, you have to set up managers(servers for managing) on the Internet. Servers have to register themselves to a manager, clients ask a manager for servers information. As managers know both servers and clients public IP addresses, NAT should be easy once you think your managers as STUN servers.
The most simple manager can be just a simple wraper over database/Redis with a server name/server ip/server metadata table. Details about how to implement it relies on your skill stack. To enable STUN, there are open source STUN server and client you can just pick up, e.g. coturn. No matter you put the STUN server into your manager or not, both your managers and STUN servers have to be published on the Internet (sure include AWS) where they have their own public IP.
Or you can let the managers to relay messages for servers and clients. But this way may be too complicated compares to previous. I would not recommend.
I have been looking for an answer to this problem, but I cannot find what I am looking for. I think, perhaps, it is because I lack the knowledge to ask the question in meaningful way.
I have been learning a lot about remote access to devices at home. I know that ISP's change public IP addresses regularly (dynamic IP address). I know that to get around this, one could use a service like "no-ip", etc. Or one could get a static IP address.
What I do not understand is how some of the latest home automation devices are able to be controlled remotely without use of a static IP, or a service like "no-ip". For example, a wifi enabled thermostat, or lighting system.
If the device had a built in server, or client, then I assume that the device could connect to an outside server in a remote location. The user could then also log into that server and send commands to the device. What I don't understand is how commands sent to the device from a cell phone, for example, can reach the home device. Presumably the off site location of the server would have to know the public IP address where the devise is located, and then port-forwarding would have to be set up to allow access to the device.
What am I missing here? Is it possible to create a homemade wifi enabled thermostat, webcam, or other device without using port-forwarding, no-ip, or a static IP?
Well, there are several ways to bypass the inbound connection constraint of NAT protocol. Such as:
A virtual adapter on the device configured to a VPN server that has an inbound port open ready to transfer data. Various open source solutions such as openVPN are considered as great examples for this service over IOT boards like Raspberry Pi, Beagle Bone, etc. These are used as gateways often. Further, they communicate with the microcontrollers over popular IOT protocols such as MQTT, COAP, etc.
Another solution is to create a port forwarding tunnel, since the router won't block the outbound connection. There are various tunneling services that are availble such as localtunnel, ngrok, etc. You could also use a cloud server that has a public IP such as AWS, DigitalOcean, etc. Again as above mentioned point, they can be implemented in the gateways.
Some devices "phone home" to a server so that there are ports open between them and the servers, and the mobile apps just contact the servers. This is the same way your web browser can receive web pages from a web server. If you have a NAT router, the router must open a port from the inside device to the outside server. This is maintained in a NAT table with expiration timers for UDP and session monitoring for TCP.
I use to develop my project on my localhost, on apache in ubuntu machine.
Sometimes i need to show progress to my costumer.
Is it possible to access to localhost from remote machine?
You can use a service that provides a tunnel to your local service, such as localtunnel, pagekite or ngrok. These services simplify setting up remote demos, mobile testing and some provide request inspection as well.
I find ngrok useful because it provides a https address, which is needed to test things like webcam access.
Terms used in this answer:
Host = machine with site on it
Client = machine you are trying to access the host from
If the host and client are on the same network, you can access the host from the client by entering
http://(hostname or ip address)
in your client's browser. If the site is not running on port 80 (for http) or port 443 (for https), add the post as so (this example is for if your server is on 8080, a common alternate port):
http://(hostname or ip address):8080
If the host and client are not on the same network, and you need to reach across the internet from the client to see the host, you will need to make your host available on the internet for the client to access.
This can be extremely dangerous for your information security if you're not sure what you're doing and I'd recommend getting a cheap-o hosting account (can get them for like $10/month at places like 1:1 hosting).
There are many methods to do this - the difference is security, easiness of the configuration and cost of the solution.
Following I am typing some methods with some analyses
Port Forwarding (with Dynamic DNS and SSL encryption)
This requires router configuration (to forward your routers public port to loclhoat port), however this requires you to have fixed ip address. In case your ip address is not fixed (in most cases) you need to use Dynamic DNS services to be able to use domain name instead ip address (there are lot of available free services). Here we still have security question open. To solve security question i.e. setup ssl certificate we can use Let’s Encrypt service ( https://letsencrypt.org/ ) to get free certificate, however we should configure local server to use the certificate or we should setup reverse proxy (in most cases nginx or apache) and configure proxy to use certificate.
Conclusion – Hard to setup if we want to have secure connection (can be done for free)
VPN
For this scenario we should use VPN services. We should connect our local machine to VPN then in other side we should connect our client's machine to VPN that will allow us to access to localhost by local IP address. We can set up our own VPN server however this requires knowledge to do it right.
Conclusion – Easy, Paid, Secure, Bad User Experience (connecting to VPN every time you need to connect to localhost)
Tunneling
For this scenario we can use free tunneling services (i.e. https://tunnelin.com/). The process is very straight forward i.e. Register a User, Connect your device to service (by running one line command on device), use Web interface to open/close secure tunnels to the device.
Conclusion – Free, Secure, Easy
Yes, if you have a public and static IP. Usually, ISPs offer static ips during a session (i.e. until you disconnect and connect again)
I was wondering that how application like skype ( a popular chat client ) works in local network with one router, How it can listen on particular port?
for example:=
In one network A and B are two machines running skype , gateway of both is G1,
now how A and B will have same IP on internet that is of G1, but how can they ensure that they are listening on different ports? How can they ask to router G1 for unique port.
I want to make a simple text chat server on linux. How can I have connections between two different computers in two different networks?
Solution to your problem is to have a forwarding server somewhere in the net.
Different programs use different means to connect to each other. But every chat server, including Skype, has a server, which forwards data or information about subnet IP/port availability.
There are two types of clients: "listening" clients and "passive" ones. Listening clients have direct access to Internet via router port forwarding, and "passive" ones have to use additional tricks to get their hands on external data, line external servers or additional ports to listen.
The point is, not clients connect to each other, but they connect to a server, which then connects back to them to verify they are available, and, if at least one of them is not firewalled, direct another on to connect to the first one, excludint itself from further communication. And if both are firewalled, then is has to forward their messages through itself.
Host Discovery
Manual discovery, client A knowns who client B is
Discovery through broadcast UDP which is used by lot of games for LAN play. A client sends out a packet to the broadcast address for their subnet. The peers can choose to pick up this broadcast and respond. The downside is that this is limited to the current subnet. The more general INADDR_BROADCAST (255.255.255.255) works for all subnets on the local-link, but it cannot be routed, so won't work over internet (this is what DHCP auto-configuration uses).
Discovery through a central (Rendezvous) server. Each individual client knows the address of the server, and the latter informs them about each other. This technique is used by IRC, Voip, IMs and by most 'peer-to-peer' networks.
Communication
After the initial discovery is done you want to be able to talk to eachother. On the internet this can get tricky. Most people nowadays have their own router and sit behind a NAT, so direct connections are impossible.
Using a Rendezvous server, you can possibly talk to each other using the server itself. client A tells the server what to say, and it in turn tells client B, since both clients have an outbound connection to the server.
It is possible for the clients to talk to each other without the server proxying. This requires either DMZ, port forwarding or UPnP. DMZ will basically forward all incoming connections on all the ports to a given local IP. Port forwarding only forwards certain ports to local IPs. UPnP is a bit more advanced, the client requests that the router temporarily forwards a port to it, and you tell the other client via the rendezvous server where to connect.
Chatting app implementation
The easiest solution to your problem is most likely to use a central server, which is known by all the clients, that proxies host discovery and possibly the communication between the clients. If you want the clients to communicate directly, you can just proxy host discovery, and then let either DMz, manual port forwarding or UPnP do the rest.
Another solution would be to just have direct communication through NAT traversal techniques discussed above, and do manual host discovery.
Yet another solution would be to use a public webserver and 'abuse' its ability to insert content to chat with each other.
You need a central UDP Rendezvous Server.
After the initial connection from the client to the server the UDP clients can be redirected to talk to eachother directly even if firewalled.
The trick is to open an UDP connection from the inside.
Check out Real-Time Media Flow Protocol and how they use it.
Check out UDP Hole Punching
alt text http://labs.adobe.com/technologies/stratus/images/p2pvideo_250x215.jpg
Traditional NAT servers replace the source address and port with the address and a random port number of the external interface of the NAT server. This works well for simple protocols such as HTTP and SMTP, but it can create problems for more complex protocols that require multiple response ports on the external interface of the NAT server. NAT servers also aren’t aware of information stored in the data portion of the application layer header without the help of NAT editors and similar software fixes.
Windows XP’s answer to these problems is NAT Traversal, which can automatically allow the UPnP-enabled NAT client application to communicate with a UPnP NAT device. NAT Traversal provides methods to allow the UPnP client to learn the public IP address of the NAT server and to negotiate dynamically assigned port mappings for UPnP NAT client applications.
NAT Traversal features can be built into any hardware device or software application. Applications that commonly cause troubles for NAT devices but work well when UPnP-enabled include the following:
Multiplayer Internet games
Audio and video communications
Terminal Services clients and servers
Peer-to-peer file sharing applications
When these applications are UPnP-enabled, access through the Windows XP ICS allows them to work seamlessly.
Unless A and B are actually "listening" to the responses to outgoing requests, your router will need to be cofigured to forward the relevant port numbers to the relevant hosts. This isn't something that you can request in the code, it's something you need to configure on the router itself.