I've started a project where I need to actively (all the time) scan for BLE Devices. I'm on Linux, using Bluez 5.49 and I use Python to communicate with dbus 1.10.20).
I' m able to start scanning, stop scanning with bluetoothctl and get the BLE Advertisement data through DBus (GetManagedObjects() of the BlueZ interface). The problem I have is when I let the scanning for many hours, dbus-deamon start to take more and more of the RAM and I'm not able to find how to "flush" what dbus has gathered from BlueZ. Eventually the RAM become full and Linux isn't happy.
So I've tried not to scan for the entire time, that would maybe let the Garbage collector do its cleanup. It didn't work.
I've edited the /etc/dbus-1/system.d/bluetooth.conf to remove any interface that I didn't need
<policy user="root">
<allow own="org.bluez"/>
<allow send_destination="org.bluez"/>
</policy>
That has slow down the RAM build-up but didn't solve the issue.
I've found a way to inspect which connection has byte waiting and confirmed that it comes from blueZ
Connection :1.74 with pid 3622 '/usr/libexec/bluetooth/bluetoothd --experimental ' (org.bluez):
IncomingBytes=1253544
PeakIncomingBytes=1313072
OutgoingBytes=0
PeakOutgoingBytes=210
and lastly, I've found that someone needs to read what is waiting in DBus in order to free the memory. So I've found this : https://stackoverflow.com/a/60665430/15325057
And I receive the data that BlueZ is sending over but the memory still built-up.
The only way I know to free up dbus is to reboot linux. which is not ideal.
I'm coming at the end of what I understand of DBus and that's why I'm here today.
If you have any insight that could help me to free dbus from BlueZ messages, it would be highly appreciated.
Thanks in advance
EDIT Adding the DBus code i use to read the discovered devices:
#!/usr/bin/python3
import dbus
BLUEZ_SERVICE_NAME = "org.bluez"
DBUS_OM_IFACE = "org.freedesktop.DBus.ObjectManager"
DEVICES_IFACE = "org.bluez.Device1"
def main_loop(subproc):
devinfo = None
objects = None
dbussys = dbus.SystemBus()
dbusconnection = dbussys.get_object(BLUEZ_SERVICE_NAME, "/")
bluezInterface = dbus.Interface(dbusconnection, DBUS_OM_IFACE)
while True:
try:
objects = bluezInterface.GetManagedObjects()
except dbus.DBusException as err:
print("dbus Error : " + str(err))
pass
all_devices = (str(path) for path, interfaces in objects.items() if DEVICES_IFACE in interfaces.keys())
for path, interfaces in objects.items():
if "org.bluez.Adapter1" not in interfaces.keys():
continue
device_list = [d for d in all_devices if d.startswith(path + "/")]
for dev_path in device_list:
properties = objects[dev_path][DEVICES_IFACE]
if "ServiceData" in properties.keys() and "Name" in properties.keys() and "RSSI" in properties.keys():
#[... Do someting...]
Indeed, Bluez flushes memory when you stop discovering. So in order to scan continuously you need start and stop the discovery all the time. I discover for 6 seconds, wait 1 second and then start discovering for 6 seconds again...and so on. If you check the logs you will see it deletes a lot of stuff when stopping discovery.
I can't really reproduce your error exactly but my system is not happy running that fast while loop repeatedly getting the data from GetManagedObjects.
Below is the code I ran based on your code with a little bit of refactoring...
import dbus
BLUEZ_SERVICE_NAME = "org.bluez"
DBUS_OM_IFACE = "org.freedesktop.DBus.ObjectManager"
ADAPTER_IFACE = "org.bluez.Adapter1"
DEVICES_IFACE = "org.bluez.Device1"
def main_loop():
devinfo = None
objects = None
dbussys = dbus.SystemBus()
dbusconnection = dbussys.get_object(BLUEZ_SERVICE_NAME, "/")
bluezInterface = dbus.Interface(dbusconnection, DBUS_OM_IFACE)
while True:
objects = bluezInterface.GetManagedObjects()
for path in objects:
name = objects[path].get(DEVICES_IFACE, {}).get('Name')
rssi = objects[path].get(DEVICES_IFACE, {}).get('RSSI')
service_data = objects[path].get(DEVICES_IFACE, {}).get('ServiceData')
if all((name, rssi, service_data)):
print(f'{name} # {rssi} = {service_data}')
#[... Do someting...]
if __name__ == '__main__':
main_loop()
I'm not sure what you are trying to do in the broader project but if I can make some recommendations...
A more typical way of scanning for service/manufacturer data is to subscribe to signals in D-Bus that trigger callbacks when something of interest happens.
Below is some code I use to look for iBeacons and Eddystone beacons. This runs using the GLib event loop which is maybe something you have ruled out but is more efficient on resources.
It does use different Python dbus bindings as I find pydbus more "pythonic".
I have left the code in processing the beacons as it might be a useful reference.
import argparse
from gi.repository import GLib
from pydbus import SystemBus
import uuid
DEVICE_INTERFACE = 'org.bluez.Device1'
remove_list = set()
def stop_scan():
"""Stop device discovery and quit event loop"""
adapter.StopDiscovery()
mainloop.quit()
def clean_beacons():
"""
BlueZ D-Bus API does not show duplicates. This is a
workaround that removes devices that have been found
during discovery
"""
not_found = set()
for rm_dev in remove_list:
try:
adapter.RemoveDevice(rm_dev)
except GLib.Error as err:
not_found.add(rm_dev)
for lost in not_found:
remove_list.remove(lost)
def process_eddystone(data):
"""Print Eddystone data in human readable format"""
_url_prefix_scheme = ['http://www.', 'https://www.',
'http://', 'https://', ]
_url_encoding = ['.com/', '.org/', '.edu/', '.net/', '.info/',
'.biz/', '.gov/', '.com', '.org', '.edu',
'.net', '.info', '.biz', '.gov']
tx_pwr = int.from_bytes([data[1]], 'big', signed=True)
# Eddystone UID Beacon format
if data[0] == 0x00:
namespace_id = int.from_bytes(data[2:12], 'big')
instance_id = int.from_bytes(data[12:18], 'big')
print(f'\t\tEddystone UID: {namespace_id} - {instance_id} \u2197 {tx_pwr}')
# Eddystone URL beacon format
elif data[0] == 0x10:
prefix = data[2]
encoded_url = data[3:]
full_url = _url_prefix_scheme[prefix]
for letter in encoded_url:
if letter < len(_url_encoding):
full_url += _url_encoding[letter]
else:
full_url += chr(letter)
print(f'\t\tEddystone URL: {full_url} \u2197 {tx_pwr}')
def process_ibeacon(data, beacon_type='iBeacon'):
"""Print iBeacon data in human readable format"""
print('DATA:', data)
beacon_uuid = uuid.UUID(bytes=bytes(data[2:18]))
major = int.from_bytes(bytearray(data[18:20]), 'big', signed=False)
minor = int.from_bytes(bytearray(data[20:22]), 'big', signed=False)
tx_pwr = int.from_bytes([data[22]], 'big', signed=True)
print(f'\t\t{beacon_type}: {beacon_uuid} - {major} - {minor} \u2197 {tx_pwr}')
def ble_16bit_match(uuid_16, srv_data):
"""Expand 16 bit UUID to full 128 bit UUID"""
uuid_128 = f'0000{uuid_16}-0000-1000-8000-00805f9b34fb'
return uuid_128 == list(srv_data.keys())[0]
def on_iface_added(owner, path, iface, signal, interfaces_and_properties):
"""
Event handler for D-Bus interface added.
Test to see if it is a new Bluetooth device
"""
iface_path, iface_props = interfaces_and_properties
if DEVICE_INTERFACE in iface_props:
on_device_found(iface_path, iface_props[DEVICE_INTERFACE])
def on_device_found(device_path, device_props):
"""
Handle new Bluetooth device being discover.
If it is a beacon of type iBeacon, Eddystone, AltBeacon
then process it
"""
address = device_props.get('Address')
address_type = device_props.get('AddressType')
name = device_props.get('Name')
alias = device_props.get('Alias')
paired = device_props.get('Paired')
trusted = device_props.get('Trusted')
rssi = device_props.get('RSSI')
service_data = device_props.get('ServiceData')
manufacturer_data = device_props.get('ManufacturerData')
if address.casefold() == '00:c3:f4:f1:58:69':
print('Found mac address of interest')
if service_data and ble_16bit_match('feaa', service_data):
process_eddystone(service_data['0000feaa-0000-1000-8000-00805f9b34fb'])
remove_list.add(device_path)
elif manufacturer_data:
for mfg_id in manufacturer_data:
# iBeacon 0x004c
if mfg_id == 0x004c and manufacturer_data[mfg_id][0] == 0x02:
process_ibeacon(manufacturer_data[mfg_id])
remove_list.add(device_path)
# AltBeacon 0xacbe
elif mfg_id == 0xffff and manufacturer_data[mfg_id][0:2] == [0xbe, 0xac]:
process_ibeacon(manufacturer_data[mfg_id], beacon_type='AltBeacon')
remove_list.add(device_path)
clean_beacons()
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('-d', '--duration', type=int, default=0,
help='Duration of scan [0 for continuous]')
args = parser.parse_args()
bus = SystemBus()
adapter = bus.get('org.bluez', '/org/bluez/hci0')
bus.subscribe(iface='org.freedesktop.DBus.ObjectManager',
signal='InterfacesAdded',
signal_fired=on_iface_added)
mainloop = GLib.MainLoop()
if args.duration > 0:
GLib.timeout_add_seconds(args.duration, stop_scan)
adapter.SetDiscoveryFilter({'DuplicateData': GLib.Variant.new_boolean(False)})
adapter.StartDiscovery()
try:
print('\n\tUse CTRL-C to stop discovery\n')
mainloop.run()
except KeyboardInterrupt:
stop_scan()
Related
I'm trying to get the data stream of a sensor transmitter that uses the modbus rtu communication protocol on my raspberry pi 3B. I'm able to get the data with the pymodbus2.5.3 library.
For this I use this code:
from pymodbus.client.sync import ModbusSerialClient # Import the pymodbus library part for syncronous master (=client)
client = ModbusSerialClient(
method='rtu', #Modbus Modus = RTU = via USB & RS485
port='/dev/ttyUSB0', #Connected over ttyUSB0, not AMA0
baudrate=19200, #Baudrate was changed from 38400 to 19200
timeout=3, #
parity='N', #Parity = None
stopbits=2, #Bites was changed from 1 to 2
bytesize=8 #
)
if client.connect(): # Trying to connect to Modbus Server/Slave
#Reading from a holding register
res = client.read_holding_registers(address=100, count=8, unit=1) #Startregister = 100, Registers to be read = 8, Answer size = 1 byte
if not res.isError(): #If Registers don't show Error
print(res.registers) #Print content of registers
else:
print(res) #Print Error Message, for meaning look at (insert git hub)
else: #If not able to connect, do this
print('Cannot connect to the Transmitter M80 SM and Sensor InPro 5000i.')
print('Please check the following things:')
print('Does the RS485-to-USB Adapter have power? Which LEDs are active?')
print('Are the cables connected correctly?')
And get the following output:
[15872, 17996, 16828, 15728, 16283, 45436, 16355, 63231]
With the help of the Modbus Poll and Slave Programms I know that those results should decoded be:
[0.125268, --, 23.53, --, 1.21094, --, 1.77344, --]
To get to the right results I tried the command that the pymodbus github suggests .decode():
res.decode(word_order = little, byte_order = little, formatters = float64)
[I know that those aren't the needed options but I copied the suggested github code to check if it works.]
After putting the code segment into the code the changed part looks like this:
if not res.isError(): #If Registers don't show Error
res.decode(word_order = little, byte_order = little, formatters = float64)
print(res.registers) #Print content of registers
else:
print(res) #Print Error Message, for meaning look at (insert git hub)
When I run this code, I get the following output, that traces to the decoding segment:
NameError: name 'little' is not defined
After this, I imported also the pymodbus part translation. But it showed the same output.
How can I decode my incoming data?
You can use BinaryPayloadDecoder to help decoding your payload, here is a simplified example, change Endian.Big and Endian.Little if needed.
if client.connect(): # Trying to connect to Modbus Server/Slave
#Reading from a holding register
res = client.read_holding_registers(address=100, count=8, unit=1) #Startregister = 100, Registers to be read = 8, Answer size = 1 byte
# ====== added code start ======
decoder = BinaryPayloadDecoder.fromRegisters(res.registers, Endian.Little, wordorder=Endian.Little)
first_reading = decoder.decode_32bit_float()
second_reading = decoder.decode_32bit_float()
# ====== added code end ======
if not res.isError(): #If Registers don't show Error
print(res.registers) #Print content of registers
else:
print(res) #Print Error Message, for meaning look at (insert git hub)
Remember to import from pymodbus.payload import BinaryPayloadDecoder at top and add necessary exception handlers in your final code.
Reference document: https://pymodbus.readthedocs.io/en/latest/source/library/pymodbus.html#pymodbus.payload.BinaryPayloadDecoder
I am developing a program to read Modbus data via serial RTU using pymodbus.
My purpose is to read data from an ABB XRCG4 Xseries flow computing PC. I know that the COM port on the device side is COM1 and the serial protocol is RS232, on my PC, using a serial converter I am trying to connect to it.
I know the configuration as follows:
PROTOCOL= 'rtu'
USB_PORT = 'COM4'
PARITY = 'None'
BAUD_RATE = 9600
TIMEOUT = 3
For test purposes I made an Arduino board into a Modbus Slave. I can read the data from the board with QModMaster, Modbus Pool and SimplyModbusMaster but I am unable to do so with my script. For the board I am unable to read anything at all, not eve with the programs mentioned above.
When I use the above mentioneds program to read data from the Arduino, everything works fine, when I try with my script, I get the AttributeError: 'ModbusIOException' object has no attribute 'registers'
I also noticed that I get this error if I unplug the cable while connection is running.
What am I missing? My guess is that I might not have a hardware dealer for the dsrdtr exchange, according to page 24 of the pymodbus documentation or maybe on the board side, the Register Format is not ok for my code.
Here you can see the details from the ABB board
Port description part1
Port description part2
My code:
from pymodbus.client.sync import ModbusSerialClient as ModbusClient
from pymodbus.client.sync import *
from pymodbus.register_read_message import *
from pymodbus.constants import Endian
from pymodbus.payload import BinaryPayloadDecoder
from pymodbus.payload import BinaryPayloadBuilder
from pymodbus.compat import iteritems
from pymodbus.exceptions import *
import time
import logging
import logging.handlers as Handlers
logging.basicConfig()
log = logging.getLogger()
log.setLevel(logging.DEBUG)
USB_PORT = 'COM4'
PARITY = 'N' #'N'
BAUD_RATE = 9600
TIMEOUT = 0.15
client = ModbusClient(method='rtu',
port=USB_PORT,
stopbits=1,
bytesize=8,
parity=PARITY,
baudrate=BAUD_RATE,
timeout=TIMEOUT, )
while True:
connection = client.connect()
#print(client.)
print(f"COM Details {client.socket}")
#client.socket.setRTS(True)
print(f"Connection status is: {connection} \nClient State: {client.state} \nTimeout: {client.timeout}")
#print(client.get_serial_settings)
try:
#################################### CITIM INPUT REGISTER (adresa,nr de registrii si unit ID
input_value = client.read_input_registers(count=0x01,address=10,unit=0x01)
#client.socket.setRTS(False)
print(input_value)
decoder = BinaryPayloadDecoder.fromRegisters(input_value.registers, byteorder=Endian.Little)
decoded = {
'string': decoder.decode_string(8),
'float': decoder.decode_32bit_float(),
'16uint': decoder.decode_16bit_uint(),
'ignored': decoder.skip_bytes(2),
#'8int': decoder.decode_8bit_int(),
'bits': decoder.decode_bits(),}
############### CITIM HOLDING REGISTER (exemplu, modelul vantilatorului)
holding_values = client.read_holding_registers(0xD1A5,6,unit=0x01)
holding_decoder = BinaryPayloadDecoder.fromRegisters(holding_values.registers, byteorder=Endian.Big)
holding_decoded = {
'string': holding_decoder.decode_string(8),
#'float': holding_decoder.decode_32bit_float(),
#'16uint': holding_decoder.decode_16bit_uint(),
'ignored': holding_decoder.skip_bytes(2),
#'8int': holding_decoder.decode_8bit_int(),
'bits': holding_decoder.decode_bits(),
}
print("-" * 60)
print("INPUT REGISTERS:")
print("-" * 60)
print(f"\nResponse: {input_value.registers} \nInput Registers Decoded values: \n")
for name, value in iteritems(decoded):
print ("%s\t" % name, value)
print(f"\n Decoded speed: {decoded['float']}")
print(f"Holding values registers:\n {holding_values.registers}")
print("-" * 60)
print("Holding Values Decoded Data")
print("-" * 60)
for name, value in iteritems(holding_decoded):
print ("%s\t" % name, value)
except ConnectionException as e:
print(f"USB Disconnected {e}")
time.sleep(2)
client.close()
This is the code I wrote to record my audio until I stop the program and save it in output.wav. The problem is that the output.wav is all silent and when I print the frames, they are all zeros. I am running it on Mac OS. I tried recording an audio using the Mac voice recorder and it works so there shouldn't be a problem with the microphone, I guess. Any suggestions why that is the case?
import pyaudio
import wave
CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 2
RATE = 44100
def record():
p = pyaudio.PyAudio()
stream = p.open(format=FORMAT,
channels=CHANNELS,
rate=RATE,
input=True,
frames_per_buffer=CHUNK)
print("Start recording")
frames = []
try:
while True:
data = stream.read(CHUNK)
frames.append(data)
except KeyboardInterrupt:
print("Done recording")
except Exception as e:
print(str(e))
sample_width = p.get_sample_size(FORMAT)
stream.stop_stream()
stream.close()
p.terminate()
return sample_width, frames
def record_to_file(file_path):
wf = wave.open(file_path, 'wb')
wf.setnchannels(CHANNELS)
sample_width, frames = record()
wf.setsampwidth(sample_width)
wf.setframerate(RATE)
wf.writeframes(b''.join(frames))
wf.close()
if __name__ == '__main__':
print('#' * 80)
print("Please speak word(s) into the microphone")
print('Press Ctrl+C to stop the recording')
record_to_file('output.wav')
print("Result written to output.wav")
print('#' * 80)
Running this code on my Windows 10 machine and python 3.7 produced an audio file. Make sure that your microphone is enabled on your mac
I'm trying to send data from my dronekit.io vehicle using flask-socket.io. Unfortunately, I got this log:
Starting copter simulator (SITL)
SITL already Downloaded and Extracted.
Ready to boot.
Connecting to vehicle on: tcp:127.0.0.1:5760
>>> APM:Copter V3.3 (d6053245)
>>> Frame: QUAD
>>> Calibrating barometer
>>> Initialising APM...
>>> barometer calibration complete
>>> GROUND START
* Restarting with stat
latitude -35.363261
>>> Exception in attribute handler for location.global_relative_frame
>>> Working outside of request context.
This typically means that you attempted to use functionality that needed
an active HTTP request. Consult the documentation on testing for
information about how to avoid this problem.
longitude 149.1652299
>>> Exception in attribute handler for location.global_relative_frame
>>> Working outside of request context.
This typically means that you attempted to use functionality that needed
an active HTTP request. Consult the documentation on testing for
information about how to avoid this problem.
Here is my code:
sample.py
from dronekit import connect, VehicleMode
from flask import Flask
from flask_socketio import SocketIO, emit
import dronekit_sitl
import time
sitl = dronekit_sitl.start_default()
connection_string = sitl.connection_string()
print("Connecting to vehicle on: %s" % (connection_string,))
vehicle = connect(connection_string, wait_ready=True)
def arm_and_takeoff(aTargetAltitude):
print "Basic pre-arm checks"
while not vehicle.is_armable:
print " Waiting for vehicle to initialise..."
time.sleep(1)
print "Arming motors"
vehicle.mode = VehicleMode("GUIDED")
vehicle.armed = True
while not vehicle.armed:
print " Waiting for arming..."
time.sleep(1)
print "Taking off!"
vehicle.simple_takeoff(aTargetAltitude)
while True:
if vehicle.location.global_relative_frame.alt>=aTargetAltitude*0.95:
print "Reached target altitude"
break
time.sleep(1)
last_latitude = 0.0
last_longitude = 0.0
last_altitude = 0.0
#vehicle.on_attribute('location.global_relative_frame')
def location_callback(self, attr_name, value):
global last_latitude
global last_longitude
global last_altitude
if round(value.lat, 6) != round(last_latitude, 6):
last_latitude = value.lat
print "latitude ", value.lat, "\n"
emit("latitude", value.lat)
if round(value.lon, 6) != round(last_longitude, 6):
last_longitude = value.lon
print "longitude ", value.lon, "\n"
emit("longitude", value.lon)
if round(value.alt) != round(last_altitude):
last_altitude = value.alt
print "altitude ", value.alt, "\n"
emit("altitude", value.alt)
app = Flask(__name__)
socketio = SocketIO(app)
if __name__ == '__main__':
socketio.run(app, host='0.0.0.0', port=5000, debug=True)
arm_and_takeoff(20)
I know because of the logs that I should not do any HTTP request inside "vehicle.on_attribute" decorator method and I should search for information on how to solve this problem but I didn't found any info about the error.
Hope you could help me.
Thank you very much,
Raniel
The emit() function by default returns an event back to the active client. If you call this function outside of a request context there is no concept of active client, so you get this error.
You have a couple of options:
indicate the recipient of the event and the namespace that you are using, so that there is no need to look them up in the context. You can do this by adding room and namespace arguments. Use '/' for the namespace if you are using the default namespace.
emit to all clients by adding broadcast=True as an argument, plus the namespace as indicated in #1.
What I am trying to ultimately achieve is to control my garage door opener with a relay connected to a WeMos D1 Mini, connected to my home WiFi. I am using the openGarageDoor() function. Everything works fine with serial connection.
I have been trying to run HTTP server on a WeMos D1 Mini with this script.
customertagsAction() -- try:
import usocket as socket
except:
import socket
CONTENT = b"""\
HTTP/1.0 200 OK
Hello #%d from MicroPython!
"""
def main(micropython_optimize=False):
s = socket.socket()
# Binding to all interfaces - server will be accessible to other hosts!
ai = socket.getaddrinfo("0.0.0.0", 8080)
print("Bind address info:", ai)
addr = ai[0][-1]
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(addr)
s.listen(5)
print("Listening, connect your browser to http://<this_host>:8080/")
counter = 0
while True:
res = s.accept()
client_sock = res[0]
client_addr = res[1]
print("Client address:", client_addr)
print("Client socket:", client_sock)
if not micropython_optimize:
# To read line-oriented protocol (like HTTP) from a socket (and
# avoid short read problem), it must be wrapped in a stream (aka
# file-like) object. That's how you do it in CPython:
client_stream = client_sock.makefile("rwb")
else:
# .. but MicroPython socket objects support stream interface
# directly, so calling .makefile() method is not required. If
# you develop application which will run only on MicroPython,
# especially on a resource-constrained embedded device, you
# may take this shortcut to save resources.
client_stream = client_sock
print("Request:")
req = client_stream.readline()
print(req)
while True:
h = client_stream.readline()
if h == b"" or h == b"\r\n":
break
print(h)
client_stream.write(CONTENT % counter)
client_stream.close()
if not micropython_optimize:
client_sock.close()
counter += 1
print()
main()
The requests are received properly and the GET variables are shown on the print(). The best i have been able to do is
req = client_stream.readline()
print(req)
while True:
h = client_stream.readline()
if h == b"" or h == b"\r\n":
break
print(h)
client_stream.write(CONTENT % counter)
//my function here:
if 'opengaragedoor=1' in req:
openGarageDoor()
client_stream.close()
I don't know how to parse the request properly. I only have come up with this dirty solution. This probably causes a timeout on the requesting system, as Postman or such needs to wait for the function to run through.