Forwarding data from server to router to Arduino - networking

Basically I have an Arduino UNO that I have connected via Ethernet to my router. I have logged the IP with my server and I would like to push data to my arduino. I guess the flow of data would be to send data (not sure the best way? Should I just open up a port and stream the data?) to the router that then forwards it to my Arduino at a local address assigned by my router.
How can I tell the router to forward this information on without logging in and changing anything with the router. The point of this exercise is to do everything with code and not do any other activities like logging into the router and setting things up. I want to do this as I have my Arduino recording temperature and I want to push data to state if it should switch sending the data from degrees or Fahrenheit or visa versa.

I would use a socket server with a publish/subscribe pattern. Probably several out there but I know XSockets.NET best.
A few hours ago I actually connected my Arduino to XSockets and communicated between webpages and my arduino both ways.
I will put the code on github when I am done, but you can look at a video showing the concept here: The Stuff For You Starts At 3:20

You dont need to open any ports as long as your arduino and server is on the same local network. Since IP adresses maps all devices on a network including your arduino.
The reason you want to open ports on your router is to make people outside your local network access a server on your network. Reasons why port forwarding exists is that when someone sends a packet of data to your public IP (which is your router) it doesent know which local IP to send the packet to. It then discards it, unless you port forward a port to a local IP, which means your router sends all packets with that port to that local IP adress.
You also need to make your own server program, for example a java program to act as a server.
I made something similar to what you are looking for, but i use a wifi shield instead. I've had some problems with maintaining connections with it, but so far it seems that it's a problem with the wifi libary.
You still might learn something from it though:
Maintaining communication between Arduino and Java program
If you want to know more about how the java server works in general, here is a great tutorial which explains a lot of the basics:
http://www.thenewboston.org/watch.php?cat=25&number=38

Related

How to send data between two ESP32's over same LAN?

I want to send data(I think I can achieve it sending http requests) between two ESP's that are connected to the same LAN. I don't want to use one as an access point! I have one esp that is acting as a homekit device, and I want to send data(for example, sensor readings) to it from another esp. I've seen many tutorials where one device is acting as an access point, but that's not what I need. I also don't really need to access the esp data directly from my laptop, so I believe I don't need the html page. I would appreciate any help, code, or any existing tutorials, because I didn't find a single one.
Configuring one ESP as an AP is only done for convenience. You can configure both of them in STA (station) mode and they will connect to, e.g. your phone hotspot or your wifi router.
Connected to the same hotspot, they are on the same network, and they will be able to communicate with each other (unless the hotspot is configured to not allow this ... ). The tricky part will most likely be for the "client" to determine the IP address of the "server" in order to create the connection.
If you have admin access to the hotspot, you can cheat and hard code the IP addresses (assuming that the hotspot will assign the same IP address to the device the next time it sees it.)
It's also not difficult to use mdns (aka bonjour), which allows a network device to assign itself a local hostname. So you can name one ESP ("myserver") which allows the ESP running the client code to connect to "http://myserver.local"

Failed to make requests to ESP8266 Webserver from an android app

So I have an ESP device that is connected to my WiFi network. It runs a web server and now I want to make requests to this webserver from my android app. So I tried a couple of things and none got satisfied.
Things I tried and why not suitable for me.
Setting up mDNS:
Can't use it since android doesn't really do mDNS it fails in the lookup.
Setting a Static IP:
This approach will fail if the routers' gateway is changed. (at least that's what I believe).
(Can anyone elaborate on this ^. AFAIK if you want to make a static IP you need to match the given gateway with your router's gateway, So if I configured esp board for my router, it might not work with someone else's router.)
Setting a WebSocket connection:
To do this also need the hostname which is IP again.
Asking IP from user:
Since this is going to be a product I can't ask the user to provide the IP address in the mobile application. It leads to a lack of user experience.
I still hope that there should be another way of handling this case which I'm not aware of. I'm actually dying to get this done and I need you guys to help me with this.
It sounds like you are over complicating stuff here.
Step 1. Connect esp board to your wifi (i guess you have figured this out)
step 2. check what lan address your board has connected to, for this is usually just use
Serial.print("IP address: ");
Serial.println(WiFi.localIP());
step 3. start your wifiserver at port 80
WiFiServer(80);
step 4. in your android app, or any other platform(i recommend using browser in computer while simply testing if it works or not) type url: local_ip_address_that_you_got_above/mygetrequest?value1=1
step 5. Listen for incoming clients and intercept the get value that you will recieve in the header in fashion of "GET /mygetrequest?value1=1"
step 6. done :)
If you want to skip the process of implementing all of it from scratch you can get full source code of my example server described above at: https://spacerival.com/lounge/2614/arduino-server but i do highly suggest you to implement it on your own and not just copy paste that source code, since you wont learn much at all to be fair
So after trying all day long, I came with a solution. This is a bit of a workaround as I was lost. So What I did was I created a Soft AP with a small web server that returns the LocalIP. So my esp is working on both AP and STA modes right now.
How it works is simple
ESP runs with a small webserver with /getLocalIP endpoint.
ESP opens a SoftAP for the user to connect. The user connects with the
SoftAP.
The user will be asked to enter credentials for the desired
WiFi.
After connecting to the desired WiFi mobile app will request a
for 192.168.4.1/getLocalIP (AP's IP Address).
Since ESP is already connected to the WiFi this endpoint will send the WiFi.localIP() to the app.
So now the app has the LocalIP.

How to find nodemcu (esp8266 + arduino) with dynamic IP (DHCP)?

I would like to connect the nodemcu to some LEDs, running a HTTP server (RESTful presumably) and turn on/off accordingly. However, how can the devices in the same network find the device? I guess I could try static IP for esp8266, but that is not guarantee to work every time everywhere. So how can I scan the network to find that, or is there any better solution?
I think most people use multicast DNS (mdns) for this, a quick Google should give you some examples for the environment you use :)
I like using MQTT, which is a different protocol that uses a central broker, but it's probably not quite what you want.
Also, why would a fixed IP not work? Most routers allow to assign one based on MAC address, which should work consistently.
And finally, you should be able to set a hostname for the ESP (wifi_station_set_hostname in the ESP SDK, which I think you can call from Arduino also), which should make it known with your router under that name.
I set my DHCP server to always give the same IP address to my NodeMCUs based on their MAC addresses. They are also configured with mDNS but I don't have mDNS on my Android phone so it didn't help me.

Programming Arduino's from a centralized location

I have 16 Arduinos that are in very tight spaces and hard to get to when I need to reprogram them with my FTDI cable. I would like to have or create some sort of centralized place where I can connect my FTDI cable, make some sort of selection (switch of some sort), which will then connect the pins my cable is on to the selected arduino.
Does anyone have any thoughts on how this can be accomplished? I've toyed with transistors, but that takes a lot of them and didn't quite work.
Are there any premade solutions that are out there that I have yet to find?
Thanks very much!
Here is a thought! TCP to Serial.
Sound complicated.
Not really.
Note from AVRDUDE's manual the following
For programmers that attach to a serial port using some kind of higher
level protocol (as opposed to bit-bang style programmers), port can be
specified as net:host:port. In this case, instead of trying to open a
local device, a TCP network connection to (TCP) port on host is
established. The remote endpoint is assumed to be a terminal or
console server that connects the network stream to a local serial port
where the actual programmer has been attached to. The port is assumed
to be properly configured, for example using a transparent 8-bit data
connection without parity at 115200 Baud for a STK500.
With this, One could place your Arduino's behind your choice of TCP to Serial Server. Which is available in several forms. Cisco has a gang TS (but that is expensive, unless used). lantronix (and others have single end point devices. But then for Linux, there is "Net2Ser" which can serve up all your ttyS (aka Serial/COM ports).
With the later you could use a raspberry or TP-Link TL-WR703N (
In the latest IDE 1.5.6r2 add one entry for each TCP-to-Serial Port to ./Arduino/hardware/arduino/avr/programmers.txt file. While replacing the IP and Port with corresponding values.
TCP2001.name=TCP 2001
TCP2001.communication=serial
TCP2001.protocol=stk500v1
TCP2001.program.protocol=stk500v1
TCP2001.program.tool=avrdude
TCP2001.program.extra_params=-Pnet:192.168.1.100:2001
TCP2002.name=TCP 2002
TCP2002.communication=serial
TCP2002.protocol=stk500v1
TCP2002.program.protocol=stk500v1
TCP2002.program.tool=avrdude
TCP2002.program.extra_params=-Pnet:192.168.1.100:2002
...
Additionally change "protocol" to "upload.protocol" the following line in ./Arduino/hardware/arduino/avr/platform.txt
tools.avrdude.program.pattern="{cmd.path}" "-C{config.path}" {program.verbose} -p{build.mcu} -c{protocol} {program.extra_params} "-Uflash:w:{build.path}/{build.project_name}.hex:i"
to
tools.avrdude.program.pattern="{cmd.path}" "-C{config.path}" {program.verbose} -p{build.mcu} -c{upload.protocol} {program.extra_params} "-Uflash:w:{build.path}/{build.project_name}.hex:i"
With this you can use "Upload Using Programmer" with the selected programmer to send over TCP, rather then use the Upload.
That all said, it will only work on Linux.
avrdude: ser_open(): network connects are currently notimplemented for Win32 environments

Good commuication practices via Serial Port (RS232)

I want to know some general communication strategies, which should be followed for a communication via COM (serial) Port.
I can read/write at the port, also can list all the available ports on the machine. I'm using Qt C++ with QSerialDevice. For testing on a windows machine with no physical serial port, I'm using VPSE (virtual port)
Typical Flow (assumed):
List all the available serial ports of the computer.
Find out at which port my device (micro-controller) is connected - how to do it?
Device found.
Start a thread for reading from the port, another thread for writing it.
Safely close connection
Finding out at which port the device is connected sounds challenging to me. How to achieve it? I guess, I will send a HELLO message to each ports, if my device replies a specific per-defined reply message I can be sure my device is connected at that port. Can I?
Also I think, to seperate threads are necessary because at any time I may receive message from the Micro-controller device.
Thanks for helping :)

Resources