Regd: WebDriver on secure Certificate - webdriver

I have been using WebDriver for past 6 months.
There are couple of issues am facing right now [Version 2.3.1]:
a) when i try to get the element for the override link on the security certificate [https] page in IE through webdriver findElement, its not able to find that element but the selenium RC works fine.
Then i got a fix for tht by using:
webDriver.navigate().to(javascript:document.getElementById('overridelink').click());
Note :
I tried using the below code to fetch the element on security certificate page , but it returns
the body element
WebElement activeElement() [WebElement with focus, or the body element if no element with focus can be detected.], why its not able to pick the element by using findelement ?
b) i connected the remote network through SSL for running the webdriver test, am not able to click the override link on secure certificate [https] page?
c) is it better approach implementing webdriver [currently am using this] directly instead of using any framework like jbehave ?
Please provide your suggestions
Thanks,
Jayaraj A

Thank you for workaround!
For Java, your solution will look just a bit different and it helped me:
//driver is initialised somewhere before, for example, as RemoteWebDriver
driver.navigate().to("javascript:document.getElementById('overridelink').click()");

Yeah, I had similar problems. Webdriver doesn't seem to have complete information on
the Certificate error page for some reason.
I'm on Windows XP SP3, running IE 7 with Python/Webdriver
I'm using this hack to get around the certificate error page:
(Help, I still can't get freeeking Markdown to format a code block...)
#!/c/Python27/python
import win32con
import win32gui
def certificate_continue():
"""
Find the IE Window that has a Certificate Error and try to continue anyway.
We'll use the win32 modules to find the right window & child window,
then write some Javascript into the address bar and execute to continue.
"""
def _enumWindowsCallback(hwnd, windows):
"""
Cannibalized from Gigi Sayfan (WindowMover)
http://www.devx.com/opensource/Article/37773/1954
This appends window information as a 3-tuple to the list
passed into win32gui.EnumWindows()
"""
class_name = win32gui.GetClassName(hwnd)
# apparently win32gui.GetWindowText() only works to get the text
# on a button or a label not really for edit windows.
text = win32gui.GetWindowText(hwnd)
windows.append((hwnd, class_name, text))
def _get_certificate_error_window():
"""
all_windows[] gets filled up with a list of tuples, then loop through
it filtering on class and the window text (title bar text).
Assumes only one 'Certificate Error' window.
"""
all_windows = []
win32gui.EnumWindows(_enumWindowsCallback, all_windows)
for win in all_windows:
class_name = win[1]
title_bar_text = win[2]
if class_name == 'IEFrame' and \
'Certificate Error: Navigation Blocked' in title_bar_text:
return win
def _get_edit_text(hwnd):
"""
This function courtesy of Omar Raviv with huge help from Simon Brunning.
http://www.brunningonline.net/simon/blog/archives/000664.html
"""
buf_size = win32gui.SendMessage(hwnd, win32con.WM_GETTEXTLENGTH, 0, 0)
buf_size += 1 # don't forget that null character boys...
buffer = win32gui.PyMakeBuffer(buf_size)
# odd, we're telling them how big the text is that they're giving
# back to us
win32gui.SendMessage(hwnd, win32con.WM_GETTEXT, buf_size, buffer)
# don't need the null character now for Python
return buffer[:buf_size]
def _get_address_bar(parent_handle):
"""
There appears to be several 'Edit' windows within each browser window.
From Microsoft: If a child window has created child windows of its own,
EnumChildWindows enumerates those windows as well.
"""
childwins = []
win32gui.EnumChildWindows(parent_handle, _enumWindowsCallback,
childwins)
for win in childwins:
child_handle = win[0]
class_name = win[1]
if 'Edit' in class_name:
edit_text = _get_edit_text(child_handle)
if 'http://' in edit_text or 'https://' in edit_text:
return child_handle # then this must be it...
# begin certificate_continue
target_win = _get_certificate_error_window()
try:
cert_err_handle = target_win[0]
except TypeError:
print "OK, no Certificate Error window available"
return(1)
address_bar_handle = _get_address_bar(cert_err_handle)
# any better way to check the handle ?
if not win32gui.IsWindow( address_bar_handle):
print "Choked getting IE edit window"
return(1)
# now, need to send this JavaScript text to the browser Address Bar
javascript_continue = 'javascript: var continue_element = document.getElementById("overridelink"); continue_element.click();'
win32gui.SendMessage(address_bar_handle, win32con.WM_SETTEXT, 0,
javascript_continue)
# OK, and finally, send a carriage return to the address bar
# This last abomination, courtesy of Claudiu
# http://stackoverflow.com/#questions/5080777/
# what-sendmessage-to-use-to-send-keys-directly-to-another-window
win32gui.SendMessage(address_bar_handle, win32con.WM_KEYDOWN,
win32con.VK_RETURN, 0)
return(0)
if __name__ == '__main__':
status = certificate_continue()
exit(status)

Related

How to make this function where I can give it an argument or not

So I made this very small function. it is a bonehead easy function but frankly borderline my capabilities.. Im learning. The function works as expected, but I would like to go further. I would like to make it so I can either give it an argument (a username) and just get the information for that single user, or default to reporting all users. is this possible w/o starting over from what I have so far?
I have just poked around and seen some examples but nothing that I can fit into my script. that I can understand at least.
import boto3
iam = boto3.client('iam')
def user_group():
for myusers in iam.list_users()['Users']:
Group = iam.list_groups_for_user(UserName=myusers['UserName'])
print("User: " + myusers['UserName'])
for groupName in Group['Groups']:
print("Group: " + groupName['GroupName'])
print("----------------------------")
user_group()
I would like to have the ability to run this script in two fashions.
1) add an argument(s) of 'username' so I can get the response for a particular user
2) default to getting response for all users if no argument is given.
This can be done by using an argument with a default value:
def user_group(user = None):
if user is None:
print("No user")
else:
print(user)
user_group()
user_group('some user')
prints
No user
some user
In your case you may want to write
def user_group(user = None):
users_to_list = iam.list_users()['Users'] if user is None else [user]
for myusers in user_to_list:
...

Python 3.7- PhantomJS - Driver.get(url) with 'Window handle/name is invalid or closed?'

Using two functions to scrape a website results in a driver.get error.
I've tried different variations of while and for loops to get this to work. Now I get a driver.get error. The initial function works on its own, but when running both functions one after another I get this error.
import requests, sys, webbrowser, bs4, time
import urllib.request
import pandas as pd
from selenium import webdriver
driver = webdriver.PhantomJS(executable_path = 'C:\\PhantomJS\\bin\\phantomjs.exe')
jobtit = 'some+job'
location = 'some+city'
urlpag = ('https://www.indeed.com/jobs?q=' + jobtit + '&l=' + location + '%2C+CA')
def initial_scrape():
data = []
try:
driver.get(urlpag)
results = driver.find_elements_by_tag_name('h2')
print('Finding the results for the first page of the search.')
for result in results: # loop 2
job_name = result.text
link = result.find_element_by_tag_name('a')
job_link = link.get_attribute('href')
data.append({'Job' : job_name, 'link' : job_link})
print('Appending the first page results to the data table.')
if result == len(results):
return
except Exception:
print('An error has occurred when trying to run this script. Please see the attached error message and screenshot.')
driver.save_screenshot('screenshot.png')
driver.close()
return data
def second_scrape():
data = []
try:
#driver.get(urlpag)
pages = driver.find_element_by_class_name('pagination')
print('Variable nxt_pg is ' + str(nxt_pg))
for page in pages:
page_ = page.find_element_by_tag_name('a')
page_link = page_.get_attribute('href')
print('Taking a look at the different page links..')
for page_link in range(1,pg_amount,1):
driver.click(page_link)
items = driver.find_elements_by_tag_name('h2')
print('Going through each new page and getting the jobs for ya...')
for item in items:
job_name = item.text
link = item.find_element_by_tag_name('a')
job_link = link.get_attribute('href')
data.append({'Job' : job_name, 'link' : job_link})
print('Appending the jobs to the data table....')
if page_link == pg_amount:
print('Oh boy! pg_link == pg_amount...time to exit the loops')
return
except Exception:
print('An error has occurred when trying to run this script. Please see the attached error message and screenshot.')
driver.save_screenshot('screenshot.png')
driver.close()
return data
Expected:
Initial Function
Get website from urlpag
Find element by tag name and loop through elements while appending to a list.
When done will all elements exit and return the list.
Second Function
While still on urlpag, find element by class name and get the links for the next pages to scrape.
As we have each page to scrape, go through each page scraping and appending the elements to a different table.
Once we reach our pg_amount limit - exit and return the finalized list.
Actual:
Initial Function
Get website from urlpag
Find element by tag name and loop through elements while appending to a list.
When done will all elements exit and return the list.
Second Function
Finds class pagination, prints nxt_variable and then throws the error below.
Traceback (most recent call last):
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\Scripts\Indeedscraper\indeedscrape.py", line 23, in initial_scrape
driver.get(urlpag)
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 333, in get
self.execute(Command.GET, {'url': url})
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchWindowException: Message: {"errorMessage":"Currently Window handle/name is invalid (closed?)"
For individuals having this error, I ended up switching to chromedriver and using that instead for webscraping. It appears that using the PhantomJS driver will sometimes return this error.
I was having the same issue, until I placed my driver.close() after I was done interacting with selenium objects. I ended up closing the driver at the end of my script just to be on the safe side.

How to solve a tkinter memory leak?

I have a dynamic table with a fixed row number (like a FIFO Queue), which updates continuously through tkinter's after() function. Inside the table is a Button, which text is editable.
To make the Button's text editable I used the solution of BrenBarn and reference a loop variable into a function call at the command-attribute.
When the function update_content_items() is cycled, I found, that the memory usage is increasing MB by MB per second. I can confirm that after commenting out the lambda expression, the memory leak was gone. (as seen live running 'top' in the terminal)
It seems I have to use the lambda, otherwise the Button will have a wrong index and the user edits the wrong row, when I just used self.list_items[i], though the user clicked the right one.
Is there a way to solve the problem? How can the user click the right button and edit it while having the right index and getting rid of the leak?
The corresponding code:
def update_content_items(self):
"""
Continuously fills and updates the Table with rows and content.
The size of the table rows is initially fixed by an external value at config.ini
:return: nothing
"""
if len(self.list_items) > self.queueMaxlen:
self.queueMaxlen = len(self.list_items)
self.build_table()
try:
for i in range(len(self.list_items)):
item = self.list_items[i]
self.barcodeImgList[i].image = item.plateimage
orig_image = Image.open(io.BytesIO(item.plateimage))
ein_image = ImageTk.PhotoImage(orig_image)
self.barcodeImgList[i].configure(image=ein_image)
# keeps a reference, because somehow tkinter forgets it...??? Bug of my implementation???
self.barcodeImgList[i].image = ein_image
orig_image = None
ein_image = None
#FIXME Memory LEAK?
self.numberList[i].configure(text=item.number,
command=lambda K=i: self.edit_barcode(self.list_items[K]))
self.timestampList[i].configure(text=item.timestamp)
self.search_hitlist[i].config(bg='white', cursor="xterm")
self.search_hitlist[i].unbind("<Button-1>")
if item.queryresult is not None:
if item.queryresult.gesamtstatus != 'Gruen':
self.search_hitlist[i].insert(tk.END, item.queryresult.barcode +
'\n' + item.queryresult.permitlevel)
self.search_hitlist[i].configure(bg='red', cursor="hand2")
self.search_hitlist[i].bind("<Button-1>", item.url_callback)
else:
self.search_hitlist[i].configure(bg='green', cursor="xterm")
self.search_hitlist[i].configure(state=tk.DISABLED)
self.on_frame_configure(None)
self.canvas.after(10, self.update_content_items)
except IndexError as ie:
for number, thing in enumerate(self.list_items):
print(number, thing)
raise ie
def edit_barcode(self, item=None):
"""
Opens the number plate edit dialogue and updates the corresponding list item.
:param item: as Hit DAO
:return: nothing
"""
if item is not None:
new_item_number = EditBarcodeEntry(self.master.master, item)
if new_item_number.mynumber != 0:
item.number = new_item_number.mynumber
self.list_items.request_work(item, 'update')
self.list_items.edit_hititem_by_id(item)
self.parent.master.queryQueue.put(item)
else:
print("You shouldn't get here at all. Please see edit_barcode function.")
EDIT: It seems there is indeed a deeper memory leak (python itself). The images won't get garbage collected. Memory is slowly leaking in Python 3.x and I do use PIL. Also here: Image loading by file name memory leak is not properly fixed
What can I do, because I have to cycle through a list with records and update Labels with images? Is there a workaround? PhotoImage has no explicit close() function, and if I call del, the reference is gc'ed and no configuring of the Label possible.
an example of my proposed changes, with indentation fixed:
def update_content_items(self):
"""
Continuously fills and updates the Table with rows and content.
The size of the table rows is initially fixed by an external value at config.ini
:return: nothing
"""
if len(self.list_items) > self.queueMaxlen:
self.queueMaxlen = len(self.list_items)
self.build_table()
try:
for i in range(len(self.list_items)):
item = self.list_items[i]
self.barcodeImgList[i].image = item.plateimage
orig_image = Image.open(io.BytesIO(item.plateimage))
ein_image = ImageTk.PhotoImage(orig_image)
self.barcodeImgList[i].configure(image=ein_image)
# keeps a reference, because somehow tkinter forgets it...??? Bug of my implementation???
self.barcodeImgList[i].image = ein_image
orig_image = None
ein_image = None
self.numberList[i].configure(text=item.number) # removed lambda
self.numberList[i].bind("<Button-1>", self.edit_barcode_binding) # added binding
self.timestampList[i].configure(text=item.timestamp)
self.search_hitlist[i].config(bg='white', cursor="xterm")
self.search_hitlist[i].unbind("<Button-1>")
if item.queryresult is not None:
if item.queryresult.gesamtstatus != 'Gruen':
self.search_hitlist[i].insert(tk.END, item.queryresult.barcode +
'\n' + item.queryresult.permitlevel)
self.search_hitlist[i].configure(bg='red', cursor="hand2")
self.search_hitlist[i].bind("<Button-1>", item.url_callback)
else:
self.search_hitlist[i].configure(bg='green', cursor="xterm")
self.search_hitlist[i].configure(state=tk.DISABLED)
self.on_frame_configure(None)
self.canvas.after(10, self.update_content_items)
except IndexError as ie:
for number, thing in enumerate(self.list_items):
print(number, thing)
raise ie
def edit_barcode_binding(self, event): # new wrapper for binding
K = self.numberList.index(event.widget) # get index from list
self.edit_barcode(self.list_items[K]) # call the original function
def edit_barcode(self, item=None):
"""
Opens the number plate edit dialogue and updates the corresponding list item.
:param item: as Hit DAO
:return: nothing
"""
if item is not None:
new_item_number = EditBarcodeEntry(self.master.master, item)
if new_item_number.mynumber != 0:
item.number = new_item_number.mynumber
self.list_items.request_work(item, 'update')
self.list_items.edit_hititem_by_id(item)
self.parent.master.queryQueue.put(item)
else:
print("You shouldn't get here at all. Please see edit_barcode function.")

How to expand a tree view using AutoIt

The UI is as following:
The tool "AutoIt Window Info" can only locate the elements in red (red rectangle area), the sub items can not be located.
So how can I expand or operate these items?
Usually Windows controls can be accessed using keystrokes as well.
In the screen-dump the Farmtt element is selected. That would be your starting point.
You may try:
Send("{DOWN}") Move down an element.
Send("{TAB}") Navigate to next control (button, checkbox, etc)
Send("{NumPadMult}") Recursively expands folders in a SysTreeView32.
Send("{ENTER}") ENTER key on the main keyboard
etc.
Reference:
https://www.autoitscript.com/autoit3/docs/appendix/SendKeys.htm
There are two things over here:
1) Use the following code snippet:
;Gets the handle for the text
Func readFirstlevelTreeNodes($hWndCtrl)
Local $firstItemHandle = _GUICtrlTreeView_GetFirstItem ($hWndCtrl)
Local $iCount = _GUICtrlTreeView_GetSiblingCount( $hWndCtrl, $firstItemHandle )
Dim $aRet[$iCount]
$aRet[0] = $firstItemHandle
For $index = 1 To $iCount - 1
$aRet[$index] = _GUICtrlTreeView_GetNextSibling ( $hWndCtrl, $firstItemHandle )
$firstItemHandle = $aRet[$index]
Next
getTreeNodeTextList($hWndCtrl,$aRet)
EndFunc
; Gets the text for given sibling node handle lists
Func getTreeNodeTextList($hWndCtrl,$aRet)
ConsoleWrite("Tree Node first level list"&#CRLF)
For $index = 0 To Ubound($aRet) -1
ConsoleWrite(_GUICtrlTreeView_GetText ( $hWndCtrl, $aRet[$index] )&#CRLF)
Next
EndFunc
You may see the output for the first level tree nodes.
2) If you still dont see the output then please verify the control handle values and window handles. If they are correct and it still doesnt show the first level tree nodes then try running your sciTE editor as administrator.
I think this should help.

Why does the "command" directly open and not when clicked on Button? Python 3

I pretty new in this whole Python thing and my question is how to make, that a button runs the command, when clicking it and not before.
I searched much in the Internet but i didnt find anything.
I dont understand the classes at all. Is there no other way to do this?
Here is my work, i did on the programm.
Thanks for your help
from tkinter import *
import os
t = ""
def ordner(x):
print ("def")
if os.path.exists(os.path.join("/Kunden/",x)) == True:
pass
else:
os.mkdir(os.path.join("/Kunden/",x))
def E1holen():
x = E1.get()
ordner(x)
#Hauptfenster
main=Tk(className='Kundendatenbank')
main.iconbitmap('icon.ico')
#Inhalt Hauptfenster
L1 = Label(main, text="Kundenname:")
L1.pack(side = LEFT)
E1 = Entry(main, bd =5, textvariable=t)
E1.pack(side = RIGHT)
a = Button (main, text=("erstellen/bearbeiten"), command=E1holen()).pack()
main.mainloop()
It runs immediately ecause you tell it to.
What is the syntax for calling a function in Python? It's foo(), right? So, when you do command=E1holen(), what should python do? It should call E1holen(), and then pass the result to the command attribute.
Put another way, the command attribute takes a reference to a function, but because of the () you were calling the function and giving the command attribute whatever that function returned. The solution? Remove the ():
a = Button(..., command=E1holen)

Resources