I'm trying to send data from multiple ESP-8266 to feeds on my Adafruit IO account.
The problem is that when I try to send new values, I'm faced with a ban from publishing because the 2 seconds time limit is violated when two or more of my MCUs happen to send data at the same time (I can't synchronize them to avoid this).
is there any possible solution to this problem?
I suggest to consider those three options:
A sending token which is send from one ESp to the next. So basically all ESPs are mot allowed to send. If the token is received its allowed to send - waits the appropriate time limit hands the token to the next ESP. This solution has all Arduinos connected via an AP/router and would use client to client communication. It can be setup fail safe, so if the next ESP is not available (reset/out of battery etc) you take the next on the list and issue an additional warning to the server
The second solution could be (more flexible and dynamic BUT SPO - single point of failure) to set up one ESP as data collector to do the sending.
If the ESps are in different locations you have to set them up that they meet the following requirement:
If you have a free Adafruit IO Account, the rate limit is 30 data
points per minute.
If you exceed this limit, a notice will be sent to the
{username}/throttle MQTT topic. You can subscribe to the topic if you
wish to know when the Adafruit IO rate limit has been exceeded for
your user account. This limit applies to all Data record modification
actions over the HTTP and MQTT APIs, so if you have multiple devices
or clients publishing data, be sure to delay their updates enough that
the total rate is below your account limit.
so its not 2 sec limit but 30/min (60/min if pro) so you limit sending each ESP to the formula:
30 / Number of ESPs sending to I/O -> 30 / 5 = 6 ==> 5 incl. saftey margin
means each ESP is within a minute only allowed to send 5 times. Important if the 5 times send limit is up it HAS to wait a minute before the next send.
The answer is simple, just don't send that frequent.
In the IoT world
If data need frequent update (such as motor/servo, accelerometer, etc.), it is often that you'd want to keep it local and won't want/need to send it to the cloud.
If the data need to be in the cloud, it is often not necessary need to be updated so frequently (such as temperature/humidity).
Alternatively, if you still think that your data is so critical that need to be updated so frequently, dedicate one ESP as your Edge Gateway to collect the data from sensor nodes, and send it to the cloud at once, that actually the proper way of an IoT network design with multiple sensor nodes.
If that still doesn't work for you, you still have the choice of pay for the premium service to raise the rate limit, or build your own cloud service and integrate it with your Edge Gateway.
Related
I am developping an application on the STM32 SPBTLE-1S module (BLE 4.2). The module connects to a Raspberry Pi.
When the connection quality is low, a disconnection will sometimes occur with error code 0x28 (Reason: Instant Passed) before the connection timeout is reached.
Current connection settings are:
Conn_Interval_Min: 10
Conn_Interval_Max: 20
Slave_latency: 5
Timeout_Multiplier: 3200
Reading more on this type of error, it seems to happen when "an LMP PDU or LL PDU that includes an instant cannot be performed because the instant when this would have occurred has passed." These paquets are typically used for frequency hopping or for connection updates. In my case, they must be frequency hoping paquets.
Any idea on how to prevent these disconnections caused by "Instant Passed" errors? Or are they simply a consequence of the BLE technology?
Your question sounds similar to this one
In a nutshell, there's only two possible link layer requests that can result in this type of disconnect (referred to as LL_CONNECTION_UPDATE_IND & LL_CHANNEL_MAP_IND in the latest v5.2 Bluetooth Core Spec)
If you have access to the low level firmware for the bluetooth stack on the embedded device, what I've done in the past is increase the number of slots in the future the switch "Instant" is at so there's more time for the packet to go through in a noisy environment.
Otherwise the best you can do is try to limit the amount of times you change connection parameters to make that style of disconnect less likely to occur. (The disconnect could still be triggered by channel map change but I haven't seen many BLE stacks expose a lot of configuration over when those take place.)
I have seen a number of examples of paho clients reading sensor data then publishing, e.g., https://github.com/jamesmoulding/motion-sensor/blob/master/open.py. None that I have seen have started a network loop as suggested in https://eclipse.org/paho/clients/python/docs/#network-loop. I am wondering if the network loop is unnecessary for publishing? Perhaps only needed if I am subscribed to something?
To expand on what #hardillb has said a bit, his point 2 "To send the ping packets needed to keep a connection alive" is only strictly necessary if you aren't publishing at a rate sufficient to match the keepalive you set when connecting. In other words, it's entirely possible the client will never need to send a PINGREQ and hence never need to receive a PINGRESP.
However, the more important point is that it is impossible to guarantee that calling publish() will actually complete sending the message without using the network loop. It may work some of the time, but could fail to complete sending a message at any time.
The next version of the client will allow you to do this:
m = mqttc.publish("class", "bar", qos=2)
m.wait_for_publish()
But this will require that the network loop is being processed in a separate thread, as with loop_start().
The network loop is needed for a number of things:
To deal with incoming messages
To send the ping packets needed to keep a connection alive
To handle the extra packets needed for high QOS
Send messages that take up more than one network packet (e.g. bigger than local MTU)
The ping messages are only needed if you have a low message rate (less than 1 msg per keep alive period).
Given you can start the network loop in the background on a separate thread these days, I would recommend starting it regardless
I am designing a temperature and humidity logging system using microcontrollers and embedded C. The system sends the data to the server using GET method (could be POST,too) whenever it gathers a new data from the sensors. Whenever the power and/or the internet is gone, it logs to an SD card.
I want to send these logged data to the server whenever the internet comes back. When sent separately, each of my request is as follows and it takes about 5 seconds to complete even on my local network.
GET /add.php?userID=0000mac=000:000:000:000:000:000&id=0000000000sensor=000&temp=%2B000.000&hum=000.000 HTTP/1.1\r\nHost: 192.168.10.25\r\n\r\n
However, since my available RAM is very limited to only about 400 bytes in this microcontroller, I cannot buffer and send all the logged data in one request.
For an electricity/internet loss of 2 days, about 3000 of data set is logged. How do I send these to the server in a very quick way?
Well I think a simple for loop will do it. Here is the pseudocode:
foreach(loggedRequest){
loadTheRequestFromTheLogFile();
sendTheDataForThisRequest(); // Hang here until the server returns a response
clearMemoryFromPreviousRequest();
}
By waiting for the server's response, you will be sure that the data got to the server as well as that you will not have the ram full.
You can also go with an inner for loop with a fixed number of requests and then wait for all their responses. This will be faster but you have to do multiple tests to be sure you're not reaching to the ram limit.
I have tried reading about requests/sec definition on the parse.com, but I still couldn't understand if push messages are considered a "request"?
I was wondering if I could use parse.com push services for free, even at 10 million push messages a month, as long as I don't pass the 1 million unique devices threshold?
How is it calculated within the free 30 requests/second, if at all?
Thanks :)
A request to send a push does use an API request, and does count towards your burst limit.
However, that request could be to send a push to a single device, or a million devices, and it still uses just one request.
So yeah, you can get by for free with the limited scenario you described.
I can see how a datastream/channel can be used to send commands to a device (e.g. an actuator). The device can periodically poll the channel for incoming commands, but if the device has no storage of its own how can it tell which commands it has already received/processed?
This all depends on your implementation and hardware choices. And the real answer to this question lays far beyond the scope of Xively. You say that the device has no storage of its own, but I assume it has some kind of volatile memory at the very least.
The best thing to do would be to store the timestamp of the last datapoint that was received and compare it to whatever current data you have. If the timestamp is greater than the one in memory then you know it is new data.
An alternative to HTTP polling would be to use a socket with some kind of publish/subscribe interface that will allow you to received only new data from the server. Xively offers this on it's TCP, WebSockets, and MQTT socket servers.