Hi there,
I will search and tried lots of solution for our problem but none of works. This will be a long post so be ready.
Our System:
We have an arduino uno r3 clone and ethernet shield wiznet w5100,
This Arduino makes a http post request to a windows web service.
According to response arduino will make something that is not revelant to question.
So in our test environment, we will install a windows webservice to our windows machine and plug this machine to our local network. Then plug arduino to our local network too. than with our server ip ,arduino made an dhcp request get ip from our modem-router and can call webservices from our server.
From now on everything works fine.
Here is some sample code from our arduino.(I only use this extra library for arduino:"RestClient.h"
#include <Ethernet.h>
#include <SPI.h>
#include "RestClient.h"
RestClient client = RestClient("192.168.100.17",51200);
String response;
String PostData;
void setup()
{
Serial.begin(9600);
byte mac[] = { 0x04, 0xD6, 0x2E, 0x81, 0x01, 0xB0 };
if ( Ethernet.begin(mac) == 0) {
Serial.println("Failed to configure Ethernet using DHCP");
}
delay(1000);
Serial.println(Ethernet.localIP());
}
void postDataToServer(long rfidnumber,int rfidsource){
String postedRFIDNumber=String(rfidnumber);
postedRFIDNumber="000"+postedRFIDNumber;
response = "";
PostData="RFID="+postedRFIDNumber+"&SOURCE="+rfidsource;
const char * myPost = PostData.c_str();
int statusCode =
client.post("/sqlpublish/TTSWebService.asmx/INSERT_INDEXRFID",myPost,&response);
}
We need to install our system to a company. This Company has its own local network. they have very restricted local network. You can connect their network but can not go to the "www" without their permissions. But that is not problem. We will use only local connection for our web services because we will also use an windows server which will inside local network.
This is a picture of their network system schema:
In this picture switches is missing but you can simply guess they uses lots of swtiches .Because company is very wide and have lots of device.
So if I connect any device to their local network , this device first call dhcp protocals get ip from Windows DHCP Server and then can communicate in the local network with other devices. But can not get through the internet modem because of the firewall in the router.
Then We will setup our system like this:
In this setup switch models are : "AVAYA".
VSP7000 XLS
ERS4826 GTS-PWR+
ERS3549 GTS-PWR+
Firewall is: Watchguard Firebox M300
When we setup the system something weird is going on. First of all when I connect my PC(My Device) to local network I can call webservices in the windows server. But Arduino cannot get IP from DHCP Server, and naturaly cannot connect to web services. So we think that "ok we can give IP static". Than we will give IP to our Arduino manually. After that weird things started to begin. When Arduino try to conenct our Windows webservices it only get response some times. Roughly 1 of 20 has got response from server anything else get time out connection.And also succeded response time also too long.
If we ping our static arduino IP From another Device(For example Device_1), It get response sometimes again. (Same Amount)
So then we try to narrow down our problem.
First Of All We Change the setup like this for once to make sure problem in local network.
And Normaly all systems works perfectly. So our Modem's DHCP server make it works perfectly.(Also in this setup If I give Ip static it worked too).
So there is these possibilties for connection error.
1-Firewall
2-Switch problem
3-Arduino Clon Problem.
1-Firewall
When We talked about the problem with system admin, He told ust every local network connection and port is open in the firewall. He is probably right because any pc connect to local network can call the web sevice.
2-Switch problem
This Question, talks about it as a solution it says use static ip, bu in our case it did not solve the problem.
In this Question jdr5ca answer make sense but ı hve no idea how can test the problem or solve the problem
In this Question answer tried but not working.
Lastly this post , but it is so general. and also what ı should use replace for "arping" in windows.
3-Arduino Clon Problem.
next week we try it with with original arduino ,
I will inform everybody.
SO, any suggestion,tool, or some diagnostic tool for the problem I'm open all the suggestion.
Apparently Our Problem is a specific switch model!
Here is the model: https://www.zyxel.com/tr/tr/products_services/es_108a.shtml?t=p (zyxel ES-108E)
When we directly connect arduino to zyxel switch, and ping arduino from another device in the network only some of pings successful.(Roughly %15 succeeded).
But if we use another switch model or don't directly connect to zyxel switch, it works fine.
I dont know why it is not working with zyxel but problem is the switch!
Related
I need to write a program, using ESP-IDF, which is able to update the firmware of an ESP32-S3-based device wirelessly, without having on-field access to Internet.
I googled and found the example project about OTA:
OTA: https://github.com/espressif/esp-idf/tr ... system/ota
But this project seems to use the ESP32 as an HTTP client, connecting to a server running on a computer, and both the ESP32 and the computer being connected to the same local network created by a router acting as AP. For the firmware update on-field, I won't have access to Internet or a local network via a router, hence I will need to set up a direct connection between the ESP32 and my computer. I thought that I could do this by using the ESP32 as AP and then connect with my PC to the ESP32-created network.
For using ESP32 as AP, I found the following example project:
WiFI as AP: https://github.com/espressif/esp-idf/tr ... ted/softAP
and for using ESP32 as HTTP server I found the following example project:
HTTP server: https://github.com/espressif/esp-idf/tr ... ver/simple
My idea was to try to combine code from the above 3 example projects in order to use the ESP32 as AP instead of as STA, run an HTTP server on ESP32, use a computer as HTTP client to connect to the ESP32 server, and from the computer browser to upload the new firmware file to the ESP32 server, and then copy the new firmware from a buffer to the ota flash partition.
As I don't have experience with OTA at all and less experience with ESP32, I wanted to first ask if the above idea is possible at all or there exists a simpler way to accomplish the OTA. If there is no simpler method, how can I use the OTA code with ESP32 being an HTTP server instead of HTTP client (as the OTA example seems to be coupled to an HTTP client) ?
P.S. Using the ESP32 as HTTP client seems not applicable to me because I will need to know in advance a certain permanent IP address for the computer acting as an HTTP server which I think I cannot have, but I am not sure if I am thinking correctly ?
I have to send a file to the local server so I use the httpClient and try to use the post with the host: http://127.0.0.1:5000/show-version, In the serial monitor, but it doesn't work, again the ESP is in access point mode
void SendVersion (){
client.begin(HOST);
client.addHeader("Content-Type", "text/plain");
int response = client.POST(version);
if(response>0){
String response = client.getString(); //Get the response to the request
Serial.println(response); //Print return code
Serial.println(response); //Print request answer
}else{
Serial.print("Error on sending POST: ");
Serial.println(response);
}
client.end();
}
127.0.0.1 is a special IP address which means "this computer or device". When you use it on the ESP32 it means the ESP32, not the server you're trying to connect to. It's also known as localhost - again, shorthand for the computer or device the software is running on. It does not identify an external computer.
You need to use the actual IP address of the server you're trying to connect to. How you'll find that depends on the OS the server is running - if you don't know how to do it, use Google to find out.
And of course, if the ESP32 is in AP mode then the server it's trying to talk to needs to be connected to the ESP32's wifi network in order for the ESP32 to be able to talk to it.
So I have an ESP device that is connected to my WiFi network. It runs a web server and now I want to make requests to this webserver from my android app. So I tried a couple of things and none got satisfied.
Things I tried and why not suitable for me.
Setting up mDNS:
Can't use it since android doesn't really do mDNS it fails in the lookup.
Setting a Static IP:
This approach will fail if the routers' gateway is changed. (at least that's what I believe).
(Can anyone elaborate on this ^. AFAIK if you want to make a static IP you need to match the given gateway with your router's gateway, So if I configured esp board for my router, it might not work with someone else's router.)
Setting a WebSocket connection:
To do this also need the hostname which is IP again.
Asking IP from user:
Since this is going to be a product I can't ask the user to provide the IP address in the mobile application. It leads to a lack of user experience.
I still hope that there should be another way of handling this case which I'm not aware of. I'm actually dying to get this done and I need you guys to help me with this.
It sounds like you are over complicating stuff here.
Step 1. Connect esp board to your wifi (i guess you have figured this out)
step 2. check what lan address your board has connected to, for this is usually just use
Serial.print("IP address: ");
Serial.println(WiFi.localIP());
step 3. start your wifiserver at port 80
WiFiServer(80);
step 4. in your android app, or any other platform(i recommend using browser in computer while simply testing if it works or not) type url: local_ip_address_that_you_got_above/mygetrequest?value1=1
step 5. Listen for incoming clients and intercept the get value that you will recieve in the header in fashion of "GET /mygetrequest?value1=1"
step 6. done :)
If you want to skip the process of implementing all of it from scratch you can get full source code of my example server described above at: https://spacerival.com/lounge/2614/arduino-server but i do highly suggest you to implement it on your own and not just copy paste that source code, since you wont learn much at all to be fair
So after trying all day long, I came with a solution. This is a bit of a workaround as I was lost. So What I did was I created a Soft AP with a small web server that returns the LocalIP. So my esp is working on both AP and STA modes right now.
How it works is simple
ESP runs with a small webserver with /getLocalIP endpoint.
ESP opens a SoftAP for the user to connect. The user connects with the
SoftAP.
The user will be asked to enter credentials for the desired
WiFi.
After connecting to the desired WiFi mobile app will request a
for 192.168.4.1/getLocalIP (AP's IP Address).
Since ESP is already connected to the WiFi this endpoint will send the WiFi.localIP() to the app.
So now the app has the LocalIP.
We can update the firmware wirelessly using the OTA updates. But for updating the firmware, the esp32 device needs to be on a local network.
Is there any way or source to update the firmware of the device which is not on the local network?
There is also a way to update ESP32 firmware over the internet without any additional update server (with public and static IP address) and without port forwarding. It is based on Husarnet p2p VPN service.
The instruction is here: https://docs.husarnet.com/blog/internet-ota-esp32
Assuming the ESP32 OTA works very similar to ESP8266 which I have used. This can be accomplished in at least two ways.
This link: http://esp8266.github.io/Arduino/versions/2.1.0/doc/ota_updates/ota_updates.html
gives details on using the browser to deploy the update to the device. You would need to forward your port 80 traffic from your public IP to your internal device IP address. Then you can access your ESP through the browser and your public IP address. The better option is to publish your updated firmware to an externally available HTTP server and change your ESP OTA codde to check that external server for updates. This method is detailed in the "HTTP Server" section of the above document. This is just general guidance to the general question you asked. I believe the solutions above would work, but I have not tried them myself yet.
Yes, absolutely. OTA has no requirement of the device being on a local network. You just need to host your firmware .bin file on a web server, then have the ESP32 connect to the server and send a GET request. Espressif's example project for OTA is pretty good, but I'm assuming you're using the ESP32 Arduino libraries and not the ESP-IDF.
I'm guessing you're using this. I'd take a look at the AWS OTA example which I believe is one of the standard ESP32 Arduino examples. It's under Examples, Examples for SparkFun ESP32 Thing, Update, AWS_S3_OTA_Update. It looks like it allows you to OTA from an Amazon Web Services hosted server, which should do exactly what you want.
You can make an app that fetches the update file from the internet or elsewhere then transfer the file via BLE to the ESP32. In this case, the ESP32 will need to be connected to the phone via BLE only (no WIFI needed).
Here is the example Arduino code for the ESP32 and the Android app code
Basically I have an Arduino UNO that I have connected via Ethernet to my router. I have logged the IP with my server and I would like to push data to my arduino. I guess the flow of data would be to send data (not sure the best way? Should I just open up a port and stream the data?) to the router that then forwards it to my Arduino at a local address assigned by my router.
How can I tell the router to forward this information on without logging in and changing anything with the router. The point of this exercise is to do everything with code and not do any other activities like logging into the router and setting things up. I want to do this as I have my Arduino recording temperature and I want to push data to state if it should switch sending the data from degrees or Fahrenheit or visa versa.
I would use a socket server with a publish/subscribe pattern. Probably several out there but I know XSockets.NET best.
A few hours ago I actually connected my Arduino to XSockets and communicated between webpages and my arduino both ways.
I will put the code on github when I am done, but you can look at a video showing the concept here: The Stuff For You Starts At 3:20
You dont need to open any ports as long as your arduino and server is on the same local network. Since IP adresses maps all devices on a network including your arduino.
The reason you want to open ports on your router is to make people outside your local network access a server on your network. Reasons why port forwarding exists is that when someone sends a packet of data to your public IP (which is your router) it doesent know which local IP to send the packet to. It then discards it, unless you port forward a port to a local IP, which means your router sends all packets with that port to that local IP adress.
You also need to make your own server program, for example a java program to act as a server.
I made something similar to what you are looking for, but i use a wifi shield instead. I've had some problems with maintaining connections with it, but so far it seems that it's a problem with the wifi libary.
You still might learn something from it though:
Maintaining communication between Arduino and Java program
If you want to know more about how the java server works in general, here is a great tutorial which explains a lot of the basics:
http://www.thenewboston.org/watch.php?cat=25&number=38