I uses pyhive in jupyter to connect to hive/presto for some adhoc analysis. Something annoying is if I cancel a submitted query job via 'ctrl + c', it only stops the jupyter, but won't stop the query job remotely. Is there a way, when 'ctrl + c', it also sent a signal to stop the remote job?
def get_data_from_presto(sql):
conn = presto.connect(...)
cursor = conn.cursor()
cursor.execute(sql)
while cursor.poll() is not None:
response_json = cursor.poll()
time.sleep(2)
col_names = [i[0] for i in cursor.description]
df = pd.DataFrame(cursor.fetchall(), columns=col_names)
Related
def set_timer(update: Update, context: CallbackContext) -> None:
"""Add a job to the queue."""
chat_id = update.message.chat_id
try:
# args[0] should contain the time for the timer in seconds
due = int(context.args[0])
if due < 0:
update.message.reply_text('Sorry we can not go back to future!')
return
job_removed = remove_job_if_exists(str(chat_id), context)
context.job_queue.run_once(alarm, due, context=chat_id, name=str(chat_id))
text = 'Timer successfully set!'
if job_removed:
text += ' Old one was removed.'
update.message.reply_text(text)
except (IndexError, ValueError):
update.message.reply_text('Usage: /set <seconds>')
How do I compile here by putting job queue run_repeated?
I have some function to execute queries in my database and in that functions I have pyodbc.connect().
What happens if I don't close this connection and call other function that have pyodbc.connect() again? It will ignore the function because the connection is already open?
If it works the way i'm thinking, I want to do it to prevent the connection open and close everytime a function is called
Ps: I know it is not the better way to do it, I want to know only if it works how I'm thinking
It will ignore the function because the connection is already open?
No:
import pyodbc
conn_str = (
r'DRIVER=ODBC Driver 17 for SQL Server;'
r'SERVER=(local)\SQLEXPRESS;'
r'DATABASE=myDb;'
"Trusted_Connection=yes;"
)
def get_connections(cnxn):
crsr = cnxn.cursor()
sql = (
"SELECT session_id, login_name, status "
"FROM sys.dm_exec_sessions "
"WHERE program_name = 'Python' "
)
connections = crsr.execute(sql).fetchall()
return connections
def subroutine():
cnxn = pyodbc.connect(conn_str)
return get_connections(cnxn)
cnxn = pyodbc.connect(conn_str)
print(get_connections(cnxn))
# [(56, 'GORD-HP\\Gord', 'running')]
print(subroutine())
# [(56, 'GORD-HP\\Gord', 'sleeping'), (57, 'GORD-HP\\Gord', 'running')]
print(get_connections(cnxn))
# [(56, 'GORD-HP\\Gord', 'running')]
I am scraping from a website that will give every request session a sid, after getting the sid, I perform further search query with this sid and scrape the results.
I want to change the sid every time I've finished scraping all results of a single query, I've tried clearing the cookies but it doesn't work.
However, if I restart my crawler, it wll get a different sid each time, I just don't know how to get a new sid without restart the crawler.
I am wondering if there're something else that let the server know two requests are from the same connection.
Thanks!
Here is my current code:
class MySpider(scrapy.Spider):
name = 'my_spider'
allowed_domains = ['xxx.com']
start_urls = ['http://xxx/']
sid_pattern = r'SID=(\w+)&'
SID = None
query_list = ['aaa', 'bbb', 'ccc']
i = 0
def parse(self, response):
if self.i >= len(self.query_list):
return
pattern = re.compile(self.sid_pattern)
result = re.search(pattern, response.url)
if result is not None:
self.SID = result.group(1)
else:
exit(-1)
search_url = 'http://xxxx/AdvancedSearch.do'
query = self.query_list[i]
self.i += 1
query_form = {
'aaa':'bbb'
}
yield FormRequest(adv_search_url, method='POST', formdata=query_form, dont_filter=True,
callback=self.parse_result_entry)
yield Request(self.start_urls[0], cookies={}, callback=self.parse,dont_filter=True)
def parse_result(self, response):
do something
Setting COOKIES_ENABLED = False can achieve this, but is there another way other than a global settings?
When I switch to another tag, a new client gets selected, but it is sometimes not a client that I have my mouse cursor over. To get a client under my mouse pointer focused, I have to either click somewhere on it, or switch to it with Mod4+j / k, or move mouse cursor out and back on that client.
I want awesome to give focus to a client that is under the mouse cursor whenever a tag is changed. How do I do that?
I found a function mouse.object_under_pointer() that finds the client I need, but I don't know when to call that function. Should I connect a handler to some particular signal? I tried connecting to various signals from Signals page on the wiki and checking with naughty.notify() if that is the right signal, but none of them were triggered when I was switching between tags.
This code did the trick, however there should be a better way to do this than setting up a huge 200 ms timer (smaller timeouts didn't properly focus some clients for me, but you can try setting a smaller one).
tag.connect_signal(
"property::selected",
function (t)
local selected = tostring(t.selected) == "false"
if selected then
local focus_timer = timer({ timeout = 0.2 })
focus_timer:connect_signal("timeout", function()
local c = awful.mouse.client_under_pointer()
if not (c == nil) then
client.focus = c
c:raise()
end
focus_timer:stop()
end)
focus_timer:start()
end
end
)
tag is this global object, so you should just place this code anywhere in your rc.lua.
Two things should be done:
First, you should remove require("awful.autofocus") from your config, so that this module no longer tries to focus some client via the focus history when you switch tags.
Then, this code works for me:
do
local pending = false
local glib = require("lgi").GLib
tag.connect_signal("property::selected", function()
if not pending then
pending = true
glib.idle_add(glib.PRIORITY_DEFAULT_IDLE, function()
pending = false
local c = mouse.current_client
if c then
client.focus = c
end
return false
end)
end
end)
end
This uses GLib directly to get a callback for when no other events are pending. This should mean that "everything else" was handled.
I know this is pretty old, but it helped me to come up with this
function focus_client_under_mouse()
gears.timer( { timeout = 0.1,
autostart = true,
single_shot = true,
callback = function()
local n = mouse.object_under_pointer()
if n ~= nil and n ~= client.focus then
client.focus = n end
end
} )
end
screen.connect_signal( "tag::history::update", focus_client_under_mouse )
I am trying to read .pcap file, and aggregate number of data packets for each client(client ip here is destination address). For example, if 5 data packets have been sent to xxx.ccc.vvv.bbb, i am outputting into a file in this format:
xxx.ccc.vvv.bbb 5
This is the program i have written below:
#!/usr/bin/lua
do
numberofpkts = 0
stat = {client1 = {numberofpkts = {}}}
local file = io.open("luawrite","w")
local function init_listener()
local tap = Listener.new("wlan")
local dest_addr = Field.new("wlan.da")
local pkt_type = Field.new("wlan.fc.type")
function tap.reset()
numberofpkts = 0;
end
function tap.packet(pinfo, tvb)
client = dest_addr()
client1 = tostring(client)
type = pkt_type()
if(tostring(type) == '2') then
stat.client1.numberofpkts = stat.client1.numberofpkts+1
file:write(tostring(client1),"\t", tostring(stat.client1.numberofpkts),"\n")
end
end
end
init_listener()
end
Here, wlan.da gives the destination address. wlan.fc.type indicates that it is data packet(type = 2). I am running this using a tshark on wireless traffic.
I am getting an error:
tshark: Lua: on packet 3 Error During execution of Listener Packet Callback:
/root/statistics.lua:21: attempt to call field 'tostring' (a nil value)
tshark: Lua: on packet 12 Error During execution of Listener Packet Callback happened 2 times:
/root/statistics.lua:21: attempt to call field 'tostring' (a nil value)
Please help me how i should solve this problem. Thanks in advance!
Seems that you're trying to make the stat table a dict of statistics; if so, make sure to initialize its members correctly (by the client, whatever its value is). Maybe this helps?
do
stat = {}
local file = io.open("luawrite","w")
local function init_listener()
local tap = Listener.new("wlan")
local dest_addr = Field.new("wlan.da")
local pkt_type = Field.new("wlan.fc.type")
function tap.reset()
local client = dest_addr()
stat[client] = stat[client] or {numberofpkts = 0}
stat[client].numberofpkts = 0
end
function tap.packet(pinfo, tvb)
local client, type = dest_addr(), pkt_type()
if(tostring(type) == '2') then
stat[client] = stat[client] or {numberofpkts = 0}
stat[client].numberofpkts = stat[client].numberofpkts + 1
file:write(tostring(client),"\t", tostring(stat.client1.numberofpkts),"\n")
end
end
end
init_listener()
end