I am running a python program to listen to azure iot hub. The function is returning me a coroutine object instead of a json. I saw that if we use async function and call it as a normal function this occurs, but i created a loop to get event and then used run_until_complete function. What am i missing here?
async def main():
try:
client = IoTHubModuleClient.create_from_connection_string(connection_string)
print(client)
client.connect()
while True:
message = client.receive_message_on_input("input1") # blocking call
print("Message received on input1")
print(type(message))
print(message)
except KeyboardInterrupt:
print ( "IoTHubClient sample stopped" )
except:
print ( "Unexpected error from IoTHub" )
return
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
OUTPUT-
Message received on input1
<class 'coroutine'>
<coroutine object receive_message_on_input at 0x7f1439fe3a98>
Long story short: you just have to write await client.receive_message_on_input("input1"). Your main is a coroutine, but receive_message_on_input is a coroutine as well. You must wait for it to complete.
I could tell you the story, but it's too long really. :)
Related
I'd like to use a pub/sub StreamingPullFuture subscription with discordpy to receive instructions for removing users and sending updates to different servers.
Ideally, I would start this function when starting the discordpy server:
#bot.event
async def on_ready():
print(f'{bot.user} {bot.user.id}')
await pub_sub_function()
I looked at discord.ext.tasks but I don't think this use case fits since I'd like to handle irregularly spaced events dynamically.
I wrote this pub_sub_function() (based on the pub/sub python client docs) but it doesn't seem to be listening to pub/sub or return anything:
def pub_sub_function():
subscriber_client = pubsub_v1.SubscriberClient()
# existing subscription
subscription = subscriber_client.subscription_path(
'my-project-id', 'my-subscription')
def callback(message):
print(f"pubsub_message: {message}")
message.ack()
return message
future = subscriber_client.subscribe(subscription, callback)
try:
future.result()
except KeyboardInterrupt:
future.cancel() # Trigger the shutdown.
future.result() # Block until the shutdown is complete.
Has anyone done something like this? Is there a standard approach for sending data/messages from external services to a discordpy server and listening asynchronously?
Update: I got rid of pub_sub_function() and changed the code to this:
subscriber_client = pubsub_v1.SubscriberClient()
# existing subscription
subscription = subscriber_client.subscription_path('my-project-id', 'my-subscription')
def callback(message):
print(f"pubsub_message: {message}")
message.ack()
return message
#bot.event
async def on_ready():
print(f'{bot.user} {bot.user.id}')
await subscriber_client.subscribe(subscription, callback).result()
This works, sort of, but now the await subscriber_client.subscribe(subscription, callback).result() is blocking the discord bot, and returning this error:
WARNING discord.gateway Shard ID None heartbeat blocked for more than 10 seconds.
Loop thread traceback (most recent call last):
Ok, so this Github pr was very helpful.
In it, the user says that modifications are needed to make it work with asyncio because of Google's pseudo-future implementation:
Google implemented a custom, psuedo-future
need monkey patch for it to work with asyncio
But basically, to make the pub/sub future act like the concurrent.futures.Future, the discord.py implementation should be something like this:
async def pub_sub_function():
subscriber_client = pubsub_v1.SubscriberClient()
# existing subscription
subscription = subscriber_client.subscription_path('my-project-id', 'my-subscription')
def callback(message):
print(f"pubsub_message: {message}")
message.ack()
return message
future = subscriber_client.subscribe(subscription, callback)
# Fix the google pseduo future to behave like a concurrent Future:
future._asyncio_future_blocking = True
future.__class__._asyncio_future_blocking = True
real_pubsub_future = asyncio.wrap_future(future)
return real_pubsub_future
and then you need to await the function like this:
#bot.event
async def on_ready():
print(f'{bot.user} {bot.user.id}')
await pub_sub_function()
Let's say I am building a simple chat app using gRPC with the following .proto:
service Chat {
rpc SendChat(Message) returns (Void);
rpc SubscribeToChats(Void) returns (stream Message);
}
message Void {}
message Message {string text = 1;}
The way I often see the servicer implemented (in examples) in Python is like this:
class Servicer(ChatServicer):
def __init__(self):
self.messages = []
def SendChat(self, request, context):
self.messages.append(request.text)
return Void()
def SubscribeToChats(self, request, context):
while True:
if len(self.messages) > 0:
yield Message(text=self.messages.pop())
While this works, it seems very inefficient to spawn an infinite loop that continuously checks a condition for each connected client. It would be preferable to instead have something like this, where the send is triggered right as a message comes in and doesn't require any constant polling on a condition:
class Servicer(ChatServicer):
def __init__(self):
self.listeners = []
def SendChat(self, request, context):
for listener in self.listeners:
listener(Message(request.text))
return Void()
def SubscribeToChats(self, request, context, callback):
self.listeners.append(callback)
However, I can't seem to find a way to do something like this using gRPC.
I have the following questions:
Am I correct that an infinite loop is inefficient for a case like this? Or are there optimizations happening in the background that I'm not aware of?
Is there any efficient way to achieve something similar to my preferred solution above? It seems like a fairly common use case, so I'm sure there's something I'm missing.
Thanks in advance!
I figured out an efficient way to do it. The key is to use the AsyncIO API. Now my SubscribeToChats function can be an async generator, which makes things much easier.
Now I can use something like an asyncio Queue, which my function can await on in a while loop. Similar to this:
class Servicer(ChatServicer):
def __init__(self):
self.queue = asyncio.Queue()
async def SendChat(self, request, context):
await self.queue.put(request.text)
return Void()
async def SubscribeToChats(self, request, context):
while True:
yield Message(text=await self.queue.get())
I have a callback in Flask+dash
server = Flask(__name__, static_folder='static')
app = dash.Dash(external_stylesheets=external_stylesheets, server=server)
thus:
#server.route("/Data/<symbol>")
def Data(symbol):
ib.qualifyContracts(symbol)
This gives a warning (in actuality it is an error):
RuntimeWarning:
coroutine 'IB.qualifyContractsAsync' was never awaited
However, if I put async in front of def soo I can await the function call (but not even inserting the await yet):
#server.route("/Data/<symbol>")
async def Data(symbol):
ib.qualifyContracts(symbol)
I get an exception
TypeError
TypeError: The view function did not return a valid response. The return type must be a string, dict, tuple, Response instance, or WSGI callable, but it was a coroutine.
How does one deal with dash callbacks that need to call other functions that need to be awaitable?
I watched an asyncio Pycon video on youtube where the presenter in one of her examples said she was using asyncio.create_task to run the coroutine instead of await, as await would block the execution of the coroutine until whatever it is awaiting is complete. I thought asyncio.create_task returns a Task which needs to be await. Nevertheless, I ran test using the following code and I am somewhat surprised by its result.
async def say_hello():
print("Hello")
await asyncio.sleep(0.5)
print("World")
async def run_coro(coro):
asyncio.create_task(coro)
asyncio.run(run_coro(say_hello()))
The code above prints only Hello. I figured that asyncio.create_task in run_coro stops execution as soon as it reaches the await line in say_hello. If I rewrite say_hello as follows and the run it using run_coro I see all the lines before `await asyncio.sleep(0.5) printed on the terminal
async def say_hello():
print("Hello")
print("Hello")
print("Hello")
print("Hello")
await asyncio.sleep(0.5)
print("World")
If I rewrite run_coro as follows then all the lines get printed as expected:
async def run_coro(coro):
asyncio.create_task(coro)
await asyncio.sleep(1)
Can anyone tell why?
Because
asyncio.run actually calls loop.run_until_complete, which means the main program will exit as soon as run_coro returns. So the scheduler does not have chance to run say_hello
asyncio.create_task(coro) runs coro in the background
if you change your code to
async def run_coro(coro):
asyncio.create_task(coro)
await asyncio.sleep(0.49)
coro actually only runs for 0.49 seconds, so you still can not see print("World")
You need to suspend the current task by blocking coroutine or using await so that the next task can be done, which is why having await asyncio.sleep(1) work as expected. See more: here
I want develop a web-socket watcher in python in such a way that when I send sth then it should wait until the response is received (sort of like blocking socket programming) I know it is weird, basically I want to make a command line python 3.6 tool that can communicate with the server WHILE KEEPING THE SAME CONNECTION LIVE for all the commands coming from user.
I can see that the below snippet is pretty typical using python 3.6.
import asyncio
import websockets
import json
import traceback
async def call_api(msg):
async with websockets.connect('wss://echo.websocket.org') as websocket:
await websocket.send(msg)
while websocket.open:
response = await websocket.recv()
return (response)
print(asyncio.get_event_loop().run_until_complete(call_api("test 1")))
print(asyncio.get_event_loop().run_until_complete(call_api("test 2")))
but this will creates a new ws connection for every command which defeats the purpose. One might say, you gotta use the async handler but I don't know how to synchronize the ws response with the user input from command prompt.
I am thinking if I could make the async coroutine (call_api) work like a generator where it has yield statement instead of return then I probably could do sth like beow:
async def call_api(msg):
async with websockets.connect('wss://echo.websocket.org') as websocket:
await websocket.send(msg)
while websocket.open:
response = await websocket.recv()
msg = yield (response)
generator = call_api("cmd1")
cmd = input(">>>")
while cmd != 'exit'
result = next(generator.send(cmd))
print(result)
cmd = input(">>>")
Please let me know your valuable comments.
Thank you
This can be achieved using an asynchronous generator (PEP 525).
Here is a working example:
import random
import asyncio
async def accumulate(x=0):
while True:
x += yield x
await asyncio.sleep(1)
async def main():
# Initialize
agen = accumulate()
await agen.asend(None)
# Accumulate random values
while True:
value = random.randrange(5)
print(await agen.asend(value))
asyncio.run(main())