I have 2 py. files:
1. I have 4 workers. Each worker works 10 seconds. The total execution time of the program is 10 seconds. So all workers works simultaneously.
import asyncio
from time import sleep, time
from concurrent.futures import ProcessPoolExecutor
num_jobs = 4
queue = asyncio.Queue()
executor = ProcessPoolExecutor(max_workers=num_jobs)
loop = asyncio.get_event_loop()
def work():
sleep(10)
async def producer():
tasks = [loop.run_in_executor(executor, work) for _ in range(num_jobs)]
for f in asyncio.as_completed(tasks, loop=loop):
results = await f
await queue.put(results)
async def consumer():
completed = 0
while completed < num_jobs:
job = await queue.get()
completed += 1
if __name__ == '__main__':
s = time()
loop.run_until_complete(asyncio.gather(producer(), consumer()))
print("duration", time() - s)
2. Python Telegram Bot, that uses Webhooks for receiving messages from users. This bot receives messages from user and after 10 seconds send echo-message to user. But this not gives me that result that I give in the first case. In this second case, messages are not processed simultaneously.
import asyncio
import logging
from aiogram.dispatcher import Dispatcher
from aiogram.utils.executor import start_webhook
from aiogram import Bot, types
from bot_files.config import *
bot = Bot(token=bot_token)
dp = Dispatcher(bot)
# webhook settings
WEBHOOK_HOST = f'https://2568-176-8-60-184.ngrok.io'
WEBHOOK_PATH = f'/webhook/{bot_token}'
WEBHOOK_URL = f'{WEBHOOK_HOST}{WEBHOOK_PATH}'
# webserver settings
WEBAPP_HOST = '0.0.0.0'
WEBAPP_PORT = os.getenv('PORT', default=5000)
async def on_startup(dispatcher):
await bot.set_webhook(WEBHOOK_URL, drop_pending_updates=True)
async def on_shutdown(dispatcher):
await bot.delete_webhook()
#dp.message_handler()
async def echo(message: types.Message):
await asyncio.sleep(10)
await message.answer(message.text)
if __name__ == '__main__':
logging.basicConfig(level=logging.INFO)
start_webhook(
dispatcher=dp,
webhook_path=WEBHOOK_PATH,
skip_updates=True,
on_startup=on_startup,
on_shutdown=on_shutdown,
host=WEBAPP_HOST,
port=WEBAPP_PORT,
)
I want to put #dp.message_handler() in loop.run_until_complete(asyncio.gather(producer())) such a producer(). But I can't put decorator as a parameter. My total question is: How to process messages from many users simultaneously for telegram bot on Python, such in the first case? I know how to make it for long polling. Is it possible make it for Webhooks???
If your bot works on Webhooks, Aiogram has special decorator for simultaneous messages processing from many users - async_task: https://docs.aiogram.dev/en/latest/_modules/aiogram/dispatcher/dispatcher.html#Dispatcher.async_task.
Related
I'd like to use a pub/sub StreamingPullFuture subscription with discordpy to receive instructions for removing users and sending updates to different servers.
Ideally, I would start this function when starting the discordpy server:
#bot.event
async def on_ready():
print(f'{bot.user} {bot.user.id}')
await pub_sub_function()
I looked at discord.ext.tasks but I don't think this use case fits since I'd like to handle irregularly spaced events dynamically.
I wrote this pub_sub_function() (based on the pub/sub python client docs) but it doesn't seem to be listening to pub/sub or return anything:
def pub_sub_function():
subscriber_client = pubsub_v1.SubscriberClient()
# existing subscription
subscription = subscriber_client.subscription_path(
'my-project-id', 'my-subscription')
def callback(message):
print(f"pubsub_message: {message}")
message.ack()
return message
future = subscriber_client.subscribe(subscription, callback)
try:
future.result()
except KeyboardInterrupt:
future.cancel() # Trigger the shutdown.
future.result() # Block until the shutdown is complete.
Has anyone done something like this? Is there a standard approach for sending data/messages from external services to a discordpy server and listening asynchronously?
Update: I got rid of pub_sub_function() and changed the code to this:
subscriber_client = pubsub_v1.SubscriberClient()
# existing subscription
subscription = subscriber_client.subscription_path('my-project-id', 'my-subscription')
def callback(message):
print(f"pubsub_message: {message}")
message.ack()
return message
#bot.event
async def on_ready():
print(f'{bot.user} {bot.user.id}')
await subscriber_client.subscribe(subscription, callback).result()
This works, sort of, but now the await subscriber_client.subscribe(subscription, callback).result() is blocking the discord bot, and returning this error:
WARNING discord.gateway Shard ID None heartbeat blocked for more than 10 seconds.
Loop thread traceback (most recent call last):
Ok, so this Github pr was very helpful.
In it, the user says that modifications are needed to make it work with asyncio because of Google's pseudo-future implementation:
Google implemented a custom, psuedo-future
need monkey patch for it to work with asyncio
But basically, to make the pub/sub future act like the concurrent.futures.Future, the discord.py implementation should be something like this:
async def pub_sub_function():
subscriber_client = pubsub_v1.SubscriberClient()
# existing subscription
subscription = subscriber_client.subscription_path('my-project-id', 'my-subscription')
def callback(message):
print(f"pubsub_message: {message}")
message.ack()
return message
future = subscriber_client.subscribe(subscription, callback)
# Fix the google pseduo future to behave like a concurrent Future:
future._asyncio_future_blocking = True
future.__class__._asyncio_future_blocking = True
real_pubsub_future = asyncio.wrap_future(future)
return real_pubsub_future
and then you need to await the function like this:
#bot.event
async def on_ready():
print(f'{bot.user} {bot.user.id}')
await pub_sub_function()
I try to make a program to get the content of a Telegram Channel using Telethon (I'm begginer) :
from telethon import TelegramClient, events, sync
# Remember to use your own values from my.telegram.org!
api_id = 12345678
api_hash = "zdzhdzud"
client = TelegramClient(None, api_id, api_hash)
#client.on(events.NewMessage(chats='Channel_Test'))
async def my_event_handler(event):
print("\n\n\n",event.raw_text,"\n\n\n")
client.start()
client.run_until_disconnected()
I get "RuntimeWarning: Enable tracemalloc to get the object allocation traceback" 2 times.
Thank you
I expected to get a str for each message
So it can actually show the info I want it to in the terminal. But when I prompt it to send it as a discord message it appears to be attempting to send a blank message. It's probably something stupid, but thank you for looking. The language is Python.
import os
import discord
import requests
import json
import pprint
client = discord.Client()
def get_time():
response = requests.get("http://api.timezonedb.com/v2.1/get-time-zone?key=W9BJQ3QMGG69&format=json&by=position&lat=37.9838&lng=23.7275")
return pprint.pprint(response.json())
#client.event
async def on_ready():
print('We have logged in as {0.user}'.format(client))
#client.event
async def on_message(message):
if message.author == client.user:
return
if message.content.startswith('$petertest'):
clock = get_time()
await message.channel.send(clock)
client.run(os.environ['TOKEN'])
You are using the pprint module to print the data to the console itself. That is the issue there
Changing your code to simply return the data will fix the error.
return response.json()
If you want to send the formatted json data to discord, you can use json.dumps:
if message.content.startswith('test'):
clock = get_time()
clock = json.dumps(clock, indent=4)
await message.channel.send(clock)
How do I forward a message to another chat as soon as I receive it from a contact?
I created this example just to test routing, but it doesn't work.
#!/usr/local/bin/python3
from telethon import TelegramClient, events
api_id = 9999900
api_hash = 'xxxxxxxxxxxxxxxxxxxxxxxxxxx'
client = TelegramClient('session_name', api_id, api_hash)
client.start()
#client.on(events.NewMessage)
async def main(event):
await client.send_message('Other People', 'Hello!') #Don't work. Keeps waiting forever
with client:
client.run_until_disconnected()
#client.on(events.NewMessage)
async def main(event):
await client.forward_messages(entity, event.message)
This code will work for forwarding new messages.
You can simply put chat_id of target user in place of entity
I want develop a web-socket watcher in python in such a way that when I send sth then it should wait until the response is received (sort of like blocking socket programming) I know it is weird, basically I want to make a command line python 3.6 tool that can communicate with the server WHILE KEEPING THE SAME CONNECTION LIVE for all the commands coming from user.
I can see that the below snippet is pretty typical using python 3.6.
import asyncio
import websockets
import json
import traceback
async def call_api(msg):
async with websockets.connect('wss://echo.websocket.org') as websocket:
await websocket.send(msg)
while websocket.open:
response = await websocket.recv()
return (response)
print(asyncio.get_event_loop().run_until_complete(call_api("test 1")))
print(asyncio.get_event_loop().run_until_complete(call_api("test 2")))
but this will creates a new ws connection for every command which defeats the purpose. One might say, you gotta use the async handler but I don't know how to synchronize the ws response with the user input from command prompt.
I am thinking if I could make the async coroutine (call_api) work like a generator where it has yield statement instead of return then I probably could do sth like beow:
async def call_api(msg):
async with websockets.connect('wss://echo.websocket.org') as websocket:
await websocket.send(msg)
while websocket.open:
response = await websocket.recv()
msg = yield (response)
generator = call_api("cmd1")
cmd = input(">>>")
while cmd != 'exit'
result = next(generator.send(cmd))
print(result)
cmd = input(">>>")
Please let me know your valuable comments.
Thank you
This can be achieved using an asynchronous generator (PEP 525).
Here is a working example:
import random
import asyncio
async def accumulate(x=0):
while True:
x += yield x
await asyncio.sleep(1)
async def main():
# Initialize
agen = accumulate()
await agen.asend(None)
# Accumulate random values
while True:
value = random.randrange(5)
print(await agen.asend(value))
asyncio.run(main())