I am struggling to make RSelenium work on a unix server. It has Mozilla Firefox 60.6.1, and running the two commands:
binman::list_versions("geckodriver")
$linux64
[1] "0.22.0" "0.23.0" "0.24.0"
binman::list_versions("seleniumserver")
$generic
[1] "3.141.59" "4.0.0-alpha-1" "4.0.0-alpha-2"
it seems that the geckodriver is available (is it ?). But when I try to launch a driver :
> library(RSelenium)
> rD <- rsDriver(browser = "firefox",
+ extraCapabilities = list(
+ "moz:firefoxOptions" = list(
+ binary = "/usr/lib64/firefox/firefox",
+ args = list('--headless')
+ )
+ ))
...
[1] "Connecting to remote server"
Selenium message:invalid argument: can't kill an exited process
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
System info: host: 'login2.cluster', ip: '192.168.100.12', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-957.5.1.el7.x86_64', java.version: '1.8.0_181'
Driver info: driver.version: unknown
remote stacktrace:
Could not open firefox browser.
Client error message:
Summary: UnknownError
Detail: An unknown server-side error occurred while processing the command.
Further Details: run errorDetails method
Check server log for further details.
From this question and others I tried to downgrade the version of geckodriver, and make use of the headless mode of firefox:
rD <- rsDriver(browser = "firefox",
version = "3.141.59",
geckover = "0.22.0",
extraCapabilities = list(
"moz:firefoxOptions" = list(
binary = "/usr/lib64/firefox/firefox",
args = list('--headless')
)
))
But still get the same error.
My suspicion would be that geckodriver is actually not installed. Is this possible ? How to check it ?
Thank you for your help
You need to stick to your exact requirements regarding Selenium, GeckoDriver and Firefox versions which would be used in your test framework and remove the unwanted versions of binaries totally. GeckoDriver v0.24.0 being the latest release must be the chosen one.
Selenium v4.0.0-alpha-1 and Selenium v4.0.0-alpha-2 are alpha releases and must be avoided for Production usage. So Selenium v3.141.59 being the latest release must be the chosen one.
For GeckoDriver, Selenium and Firefox Browser compatibility you can find a detailed discussion in Which Firefox browser versions supported for given Geckodriver version?
Note: You don't need to install the GeckoDriver binary but put the binary in the desired location.
So an ideal usage would be:
rD <- rsDriver(browser = "firefox",
version = "3.141.59",
geckover = "0.24.0",
extraCapabilities = list(
"moz:firefoxOptions" = list(
binary = "/usr/lib64/firefox/firefox",
args = list('--headless')
)
))
If you are still facing the issue follow the below mentioned steps.
This error message...
message:invalid argument: can't kill an exited process
...can surface for different reasons. The possible solution can be any/either of the following:
Ensure that GeckoDriver v0.24.0 is downloaded and placed within the directory that is already in your path, e.g. /usr/local/bin
Ensure that GeckoDriver is having executable permission for non-root users.
Ensure that firefox (> v57.0) is installed (mandatory) within /usr/lib64/firefox/ directory as per your code block and the version is compatable.
Ensure that if you are running Firefox on a system with no display you have to use headless mode.
The correct usage of headless mode with GeckoDriver v0.24.0 is:
options.headless = True
There is no need for xvfb-run anymore if you set MOZ_HEADLESS=1 as follows:
$ export MOZ_HEADLESS=1 # this way you only have to set it once
If you have changed your system path, take a System Reboot.
Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.
Always execute your #Tests as a non-root user.
References
You can find a relevant detailed discussions in:
WebDriverException: Message: invalid argument: can't kill an exited process with GeckoDriver, Selenium and Python on RaspberryPi3
Related
I am trying (for the first time) to scrape content from a dynamic webpage, for which RSelenium appears to be the go-to. I cannot however get past the first step of calling rsDriver.
My code:
rdriver <- rsDriver(browser = "chrome",
port = free_port(),
chromever = "109.0.5414.25")
The rsDriver() function started throwing an error every time I tried to open it
[1] "Connecting to remote server" Could not open chrome browser.
Client error message: Undefined error in httr call. httr output:
Failed to connect to localhost port 14415: Connection refused Check
server log for further details. Warning message: In rsDriver(browser =
"chrome", port = free_port(), chromever = "109.0.5414.25") : Could
not determine server status.`
Version:
R 4.2.2
Java(TM) SE Development Kit 19.0.2 (64 bit)
> binman::list_versions("chromedriver")
$win32
[1] "109.0.5414.25" "109.0.5414.74" "110.0.5481.30"
> binman::list_versions("seleniumserver")
$generic
[1] "3.141.59" "4.0.0-alpha-1" "4.0.0-alpha-2"`
Any recommendations are much appreciated.
I installed all the necessary programs from scratch.
I searched for help on the internet and couldn't find a solution
there is an outstanding issue with how the wdman package reads the latest versions of Chrome. This is causing issues for lots of users (examples here https://stackoverflow.com/a/75176907/15363011 and here https://github.com/ropensci/RSelenium/issues/264)
You can specify a version of Chrome before the issue took place and binman/wdman will download and start using it:
rdriver <- rsDriver(browser = "chrome",
port = free_port(),
chromever = "108.0.5359.71")
If you'd like to use the newest versions, the fix is to delete the LICENSE.chromedriver file found in the same directory as the driver. You can find out how to do that in the other issues that I linked. If you want to use the latest version of Chrome you will have to do this any time a new chrome driver is released.
I have been using RSelenium for years and have never had this issue. I recently updated my google chrome to the latest version available 110.0.5481.78. I am now getting the following error when I go to use rsDriver
require(RSelenium)
rD <- rsDriver(browser = "chrome",port = 9537L, chromever = "110.0.5481.77")
"> Could not open chrome browser.
> Client error message:
> Undefined error in httr call. httr output: Failed to connect to localhost port 9537: Connection refused
> Check server log for further details.
> Warning message:
> In rsDriver(browser = "chrome", port = 9537L, chromever = "110.0.5481.77") :
> Could not determine server status."
R Console
I have tried with different versions of chromever from binman::list_versions("chromedriver") as well as leaving rsDriver blank all together. In the past when chrome has updated it has been a very simple change to chromever and everything works perfectly. Not sure if or what has changed with this latest update.
Thanks in advance.
I just fixed this same problem by removing a file LICENSE.chromedriver as per this thread: https://github.com/ropensci/RSelenium/issues/264
Use
wdman::selenium(retcommand=T)
to find the file location of binman_chromedriver files.
Navigate to this file location, go to the driver version you're using and delete the LICENSE.chromedriver file. Mine worked immediately after this action, but note that I also tried downgading wdman version to 0.2.5 (I was on 0.2.6) first:
remotes::install_version('wdman',version = '0.2.5')
I'm not sure if it was both actions that fixed it or just the file delete!
I have used RSelenium to scrape webpages without any issue for over a year including updating Chrome regularly. Today I updated Chrome to its latest version (89.0.4389.82). After the update, RSelenium can't connect to the server and times out. Here is the error message:
Selenium message:Timed out waiting for driver server to start.
System info: host: 'xxxx', ip: xxx, os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.15.5', java.version: '1.8.0_60'
Driver info: driver.version: unknown
Could not open chrome browser.
I also don't know why it says driver version unknown. I followed the guides on other simular questions and updated chromedriver to version 89.0.4389.23.
Here is the code that used to work before updating chrome:
driver <- rsDriver(browser=c("chrome"), chromever="87.0.4280.88", port = 4568)
Here is what I'm working with now which results in an error:
driver <- rsDriver(browser=c("chrome"), chromever="89.0.4389.23", port = as.integer(4568))
I was able to fix this thanks to this info I found. You can visit the link for more detailed information.
rsDriver has a boolean argument check that is TRUE by default. Its official description:
If TRUE check the versions of selenium available and the versions of
associated drivers (chromever, geckover, phantomver, iedrver). If new
versions are available they will be downloaded.
This downloads both chromedriver_mac64.zip and chromedriver_mac64_m1.zip. The chromedriver installed from the m1 file on Macs. What I did to fix the problem was go to
~/Library/Application Support/binman_chromedriver/mac64/89.0.4389.23
and then delete the m1 file and chromedriver. Then unzip chromedriver_mac64.zip to get the correct chromedriver. Go back to R and run rsDriver with the argument check set to false:
driver <- rsDriver(browser=c("chrome"), chromever="87.0.4280.88", port = 4568, check = FALSE)
This made it work for me
I built a webscraper using RSelenium and Chrome. It used to work. Suddenly, it won't even open the browser!
I ran:
rD <- rsDriver(port=sample(4000:5999,1),
browser="chrome",
chromever = "88.0.4324.96",
extraCapabilities = eCaps)
And I receive the message:
Selenium message:Timed out waiting for driver server to start.
Build info: version: '4.0.0-alpha-2', revision: 'f148142cf8', time: '2019-07-01T21:30:10'
System info: host: 'Lees-Air.attlocal.net', ip: '2600:1700:4a53:2010:0:0:0:2f%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.15.5', java.version: '15'
Driver info: driver.version: unknown
Could not open chrome browser.
Client error message:
Summary: UnknownError
Detail: An unknown server-side error occurred while processing the command.
Further Details: run errorDetails method
Check server log for further details.
Has anyone else seen this issue --where Chrome 88 keeps timing out trying to reach the driver on a Mac? Do you have a solution? I saw some people saying they experienced this and that switching to Firefox solved it. Firefox does, at minimum, open the browser for me. However, Firefox is too slow. I can run tests manually only --anytime I try to automate them, Firefox slows down to an absurd degree and on the FIRST line of code. If I run literally the exact same single command manually, it works just fine. Any recommendations to either get chrome to run again or speed up firefox?
Appears to be a typo in the configuration part and you need to replace chromever with chromedriver.
When I try to open the RSelenium I receive this error
[1] "Connecting to remote server"
Error: Summary: UnknownError
Detail: An unknown server-side error occurred while processing the command.
class: org.openqa.selenium.firefox.NotConnectedException
The version of Firefox I have is
Firefox version: 480b10
According to this I tried to update the server version
library("RSelenium")
startServer()
unlink(system.file("bin", package = "RSelenium"), recursive = T)
checkForServer(update = TRUE)
remDr <- remoteDriver()
Sys.sleep(5)
remDr$open()
Sys.sleep(5)
but the problem still exist. Does anyone face this problem? Any possible solution?
From Firefox 48 on-wards the gecko driver/ marionette will be needed to run Firefox with Selenium.
If you have Firefox 48 you can run the gecko driver as follows:
Refer to the guidelines
https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/WebDriver
Download the relevant gecko driver from https://github.com/mozilla/geckodriver/releases
Add it to your PATH or refer to the location when starting binary (see below)
# get beta selenium standalone
RSelenium::checkForServer(beta = TRUE)
# assume gecko driver is not in our path (assume windows and we downloaded to docs folder)
# if the driver is in your PATH the javaargs call is not needed
selServ <- RSelenium::startServer(javaargs = c("-Dwebdriver.gecko.driver=\"C:/Users/john/Documents/geckodriver.exe\""))
remDr <- remoteDriver(extraCapabilities = list(marionette = TRUE))
remDr$open()
....
....
remDr$close()
selServ$stop()
The above currently requires the dev version of RSelenium. Alternatively you can download the Selenium binary from http://selenium-release.storage.googleapis.com/index.html . Pick the 3.0 beta 2 binary to currently run with Firefox 48. Run the binary
java -Dwebdriver.gecko.driver=C:/Users/john/Documents/geckodriver.exe -jar selenium-server-standalone-3.0.0-beta2.jar