Airflow Logs in Slack - airflow

I would like to know how I can report a log in my Slack message with Airflow.
I have tried to circumvent this issue by saving my exception error in a new function which includes also the original function :
def other_fun():
logf = open("log_original_main.log", "w")
try:
original_main()
except Exception as e:
logf.write(str(e))
logf.close()
But I have no idea with a Bash Operator.
So I hope you can provide a more general solution which works for both cases.

Airflow has an operator for slack notification called "SlackAPIPostOperator"
SlackAPIPostOperator(
task_id='Your_TASK_ID',
token='YOUR_TOKEN',
text="Message you want to notify in your slack",
channel=SLACK_CHANNEL,
username=SLACK_USER)
For more information you can visit Here, it will guide you how to generate token and when and where to use this operator. You can also use SlackWebhookOperator check (for more information)
And also this answer will help you.

Related

context.job_queue.run_once not working in Python Telegram BOT API

I'm trying to setup a bot which:
Receives the keywords in /search_msgs userkey command from a TG group
Search in DB for that userkey and send back appropriate text back
I'm getting two errors
None type object has no attribute args, in callback_search_msgs(context), see code snippet
AttributeError: 'int' object has no attribute 'job_queue', in search_msgs(update, context), see code snippet.
Telegram's official documents is way too difficult for me to read and understand. Couldn't find even one place where Updater, update, Commandhandler, context are all explained together with examples.
How to fix this code?
import telegram
from telegram.ext import Updater,CommandHandler, JobQueue
token = "Token"
bot = telegram.Bot(token=token)
# Search specific msgs on user request
def search_msgs(update, context):
context.job_queue.run_once(callback_search_msgs, context=update.message.chat_id)
def callback_search_msgs(context):
print('In TG, args', context.args)
chat_id = context.job.context
search_msgs(context, chat_id)
def main():
updater = Updater(token, use_context=True)
dp = updater.dispatcher
dp.add_handler(CommandHandler("search_msgs",search_msgs, pass_job_queue=True,
pass_user_data=True))
updater.start_polling()
updater.idle()
if __name__ == '__main__':
main()
Let me first try & clear something up:
Telegram's official documents is way too difficult for me to read and understand. Couldn't find even one place where Updater, update, Commandhandler, context are all explained together with examples.
I'm guessing that by "Telegram's official documents" you mean the docs at https://core.telegram.org/bots/api. However, Updater, CommandHandler and context are concepts of python-telegram-bot, which is one (of many) python libraries that provides a wrapper for the bot api. python-telegram-bot provides a tutorial, examples, a wiki where a lots of the features are explained and documentation.
Now to your code:
In context.job_queue.run_once(callback_search_msgs, context=update.message.chat_id) you're not telling job_queue when to run the the job. You must pass an integer or a datetime.(date)time object as second argument.
in
def callback_search_msgs(context):
print('In TG, args', context.args)
chat_id = context.job.context
search_msgs(context, chat_id)
you are passing context and chat_id to search_msgs. However, that function treats context as an instance of telegram.ext.CallbackContext, while you pass an integer instead. Also, even if that worked, this would just schedule another job in an infinite loop.
Finally, I don't understand what scheduling jobs has to do with looking up a key in a database. All you have to do for that is something like
def search_msgs(update, context):
userkey = context.args[0]
result = look_up_key_in_db(userkey)
# this assumes that result is a string:
update.effective_message.reply_text(result)
To understand context.args better, have a look at this wiki page.
Disclaimer: I'm currently the maintainer of python-telegram-bot.

Discord Bot python 3.6 report command

I am making a custom discord bot in python. I am trying to add a !report command.
I am very confused and cant find the answer anywhere. Can anyone help me make it?
I want any user to be able to do !report #example reason.
and save it in a database such as excel or sql3 or preferably in a staff channel.
how would I do this?
I have tried to use on_message()
You could use the on_message() command:
#client.event
async def on_message(message):
if message.content.startswith("!report"):
report_channel = client.get_channel(channel id)
member = text.split(" ")[1]
reason = ' '.join(text.split(" ")[1:])
await report_channel.send(f"Member: {member}, Reason: {reason}")
So the first thing is to look to see if the person used the "!report" command with an if statement.
Next, you find the member by taking the second word of the message.
After that, you find the reason by taking the rest of the words in the message.
Then you send it to the pre-defined report channel on discord.

No output when pushing from R to slack using slackr

Tried setting up slackr using the following code:
slackr_setup(channel=channel,
incoming_webhook_url="https://hooks.slack.com/services/xxxxxxx/xxxxxx/xxxxxxxxxx",
api_token = "xxxx-xxxxxxxx-xxxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxxxx",
username=username)
However, when I run the slackr function, there is no output. (There's no error message)
Okay,so I found the reason for this. Didn't want to answer my own question, but it might help someone else having the same issue.
The channel that I was trying to post to was a private channel and once I changed the channel to a public one, it started working.

What does "rerun in upstream in pipeline"?

I am defining a pipeline in data factory, I had some errors that I correct.
The first activity is calling an usql script to do some aggregation, I changed the script plenty of time but the error is still:
[{"errorId":"E_CSC_USER_SYNTAXERROR","severity":"Error","component":"CSC","source":"USER","message":"syntax
error. Final statement did not end with a semicolon","details":"at
token 'usql', line 4\r\nnear the ###:\r\n**************\r\nCLARE
#lineitemsfile string =
\"/datalakerepo/input/2016/01/01lineitems.txt\";\nDECLARE #ordersfile
string = \"/datalakerepo/input/2016/01/01orders.txt\";\nsales.usql ###
\n","description":"Invalid syntax found in the
script.","resolution":"Correct the script syntax, using expected
token(s) as a
guide.","helpLink":"","filePath":"","lineNumber":4,"startOffset":228,"endOffset":232}].
seem like not all usql script is read from the data factory, so I though that may be the "rerun in upstream in pipeline" have something to do with this, like clear cache from previous script.
Anyone knows what "rerun in upstream in pipeline" does?
Many thanks!
"Rerun with upstream in pipeline" basically means "recalculate with all dependencies". For example, if one has pipeline1 -> dataset1 -> pipeline2 and tries to rerun pipeline2 with dependecies, then pipeline1 and pipeline2 will be both executed. I believe it works same with several chained activities within single pipeline.

How do I have my Bot respond with arguments?

So I've built a Telegram bot, which can receive the following commands:
/list
/info 123
This works great, as I can catch /info and pass the additional arguments as ints. But, sadly, the Telegram clients don't see /info 123 as a complete command, but just the /info part. Is there a way to make it recognize the entirety of the command as the command?
I've tried Markdown-ing it: [/info 123](/info 123), but no joy. Is this possible?
I've reached out to #BotSupport with the same question, and he/they/it responded swiftly with the following answer:
Hi, at the moment it is not possible to highlight parameters of a command. I any case, you may can find a workaround if you use correct custom keyboards ;)
— #BotSupport
Custom keyboards may be an option for someone, but not for me. The solution I've gone for is to give the command as /info123. As the bot receives all / commands, I check if the received command starts with info, and if so, I remove the info part. I convert the remaining string/int to arguments, and pass that along to the relevant command.
If you mean to pass the 123 as an argument for your command info and if you happen to use the python-telegram-bot, then here's how you do it:
dispatcher.add_handler(CommandHandler('hello', SayHello, pass_args=True))
According to the documentation: pass_args Determines whether the handler should be passed the arguments passed to the command as a keyword argument called args. It will contain a list of strings, which is the text following the command split on single or consecutive whitespace characters. Default is False.
you can use RegexHandler() to do this.
Here is an example
def info(bot, update):
id = update.message.text.replace('/info_', '')
update.message.reply_text(id, parse_mode='Markdown')
def main():
updater = Updater(TOKEN)
updater.dispatcher.add_handler(RegexHandler('^(/info_[\d]+)$', info))
updater.start_polling()
Usage
The command /info_120 will return 120
and /info_007 will return 007
UPDATE
for newer versions, you may use this method instead!
MessageHandler(filters.Regex(r'^(/info_[\d]+)$'), info)
To get the argument of command you don't even need to use pass_args as said Moein you can simply get it from context.args look at Github page. So you can pass as many arguments as you want and you will get a list of arguments! Here is an example from Github.
def start_callback(update, context):
user_says = " ".join(context.args)
update.message.reply_text("You said: " + user_says)
...
dispatcher.add_handler(CommandHandler("start", start_callback))
ForceReply
Upon receiving a message with this object, Telegram clients will display a reply interface to the user (act as if the user has selected the bot's message and tapped 'Reply'). This can be extremely useful if you want to create user-friendly step-by-step interfaces without having to sacrifice privacy mode.
a simple shot
In this case, a user should send a valid number with /audio command (e.g. /audio 3, if they forgot it, we can inform and force them to do so.
source:
https://core.telegram.org/bots/api#forcereply
This is a fairly rudimentary way of creating kwargs from user input.
Unfortunately, it does require the user to be aware of the fields that can be used as parameters, but if you can provide informative response when the user doesnt provide any detectable kwarg style messages then you could probably make a better experience.
As I say, extremely rudimentary idea, and would probably be achieved faster with the regex filters available. And this would be much more reliable when checking input from the user of the "pesky" variety.
The script relies on || delimiter preceeding the command and as is shown will trim any extra characters like new lines and spaces
You can remove the extra check for commit as this is provided in order to tell the bot that you want to save your input to the database explicitly.
def parse_kwargs(update):
commit = False
kwargs = {}
if update.message:
for args in update.message.text.split('||')[1:]:
for kw_pair in args.split(','):
key, value = kw_pair.split('=')
if key.strip() != 'commit':
kwargs[key.strip()] = value.strip()
elif key.strip() == 'commit' and value.strip().lower() == 'true':
commit = True
return kwargs, commit

Resources