I have used RSelenium to scrape webpages without any issue for over a year including updating Chrome regularly. Today I updated Chrome to its latest version (89.0.4389.82). After the update, RSelenium can't connect to the server and times out. Here is the error message:
Selenium message:Timed out waiting for driver server to start.
System info: host: 'xxxx', ip: xxx, os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.15.5', java.version: '1.8.0_60'
Driver info: driver.version: unknown
Could not open chrome browser.
I also don't know why it says driver version unknown. I followed the guides on other simular questions and updated chromedriver to version 89.0.4389.23.
Here is the code that used to work before updating chrome:
driver <- rsDriver(browser=c("chrome"), chromever="87.0.4280.88", port = 4568)
Here is what I'm working with now which results in an error:
driver <- rsDriver(browser=c("chrome"), chromever="89.0.4389.23", port = as.integer(4568))
I was able to fix this thanks to this info I found. You can visit the link for more detailed information.
rsDriver has a boolean argument check that is TRUE by default. Its official description:
If TRUE check the versions of selenium available and the versions of
associated drivers (chromever, geckover, phantomver, iedrver). If new
versions are available they will be downloaded.
This downloads both chromedriver_mac64.zip and chromedriver_mac64_m1.zip. The chromedriver installed from the m1 file on Macs. What I did to fix the problem was go to
~/Library/Application Support/binman_chromedriver/mac64/89.0.4389.23
and then delete the m1 file and chromedriver. Then unzip chromedriver_mac64.zip to get the correct chromedriver. Go back to R and run rsDriver with the argument check set to false:
driver <- rsDriver(browser=c("chrome"), chromever="87.0.4280.88", port = 4568, check = FALSE)
This made it work for me
Related
I am trying (for the first time) to scrape content from a dynamic webpage, for which RSelenium appears to be the go-to. I cannot however get past the first step of calling rsDriver.
My code:
rdriver <- rsDriver(browser = "chrome",
port = free_port(),
chromever = "109.0.5414.25")
The rsDriver() function started throwing an error every time I tried to open it
[1] "Connecting to remote server" Could not open chrome browser.
Client error message: Undefined error in httr call. httr output:
Failed to connect to localhost port 14415: Connection refused Check
server log for further details. Warning message: In rsDriver(browser =
"chrome", port = free_port(), chromever = "109.0.5414.25") : Could
not determine server status.`
Version:
R 4.2.2
Java(TM) SE Development Kit 19.0.2 (64 bit)
> binman::list_versions("chromedriver")
$win32
[1] "109.0.5414.25" "109.0.5414.74" "110.0.5481.30"
> binman::list_versions("seleniumserver")
$generic
[1] "3.141.59" "4.0.0-alpha-1" "4.0.0-alpha-2"`
Any recommendations are much appreciated.
I installed all the necessary programs from scratch.
I searched for help on the internet and couldn't find a solution
there is an outstanding issue with how the wdman package reads the latest versions of Chrome. This is causing issues for lots of users (examples here https://stackoverflow.com/a/75176907/15363011 and here https://github.com/ropensci/RSelenium/issues/264)
You can specify a version of Chrome before the issue took place and binman/wdman will download and start using it:
rdriver <- rsDriver(browser = "chrome",
port = free_port(),
chromever = "108.0.5359.71")
If you'd like to use the newest versions, the fix is to delete the LICENSE.chromedriver file found in the same directory as the driver. You can find out how to do that in the other issues that I linked. If you want to use the latest version of Chrome you will have to do this any time a new chrome driver is released.
I built a webscraper using RSelenium and Chrome. It used to work. Suddenly, it won't even open the browser!
I ran:
rD <- rsDriver(port=sample(4000:5999,1),
browser="chrome",
chromever = "88.0.4324.96",
extraCapabilities = eCaps)
And I receive the message:
Selenium message:Timed out waiting for driver server to start.
Build info: version: '4.0.0-alpha-2', revision: 'f148142cf8', time: '2019-07-01T21:30:10'
System info: host: 'Lees-Air.attlocal.net', ip: '2600:1700:4a53:2010:0:0:0:2f%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.15.5', java.version: '15'
Driver info: driver.version: unknown
Could not open chrome browser.
Client error message:
Summary: UnknownError
Detail: An unknown server-side error occurred while processing the command.
Further Details: run errorDetails method
Check server log for further details.
Has anyone else seen this issue --where Chrome 88 keeps timing out trying to reach the driver on a Mac? Do you have a solution? I saw some people saying they experienced this and that switching to Firefox solved it. Firefox does, at minimum, open the browser for me. However, Firefox is too slow. I can run tests manually only --anytime I try to automate them, Firefox slows down to an absurd degree and on the FIRST line of code. If I run literally the exact same single command manually, it works just fine. Any recommendations to either get chrome to run again or speed up firefox?
Appears to be a typo in the configuration part and you need to replace chromever with chromedriver.
I am struggling to make RSelenium work on a unix server. It has Mozilla Firefox 60.6.1, and running the two commands:
binman::list_versions("geckodriver")
$linux64
[1] "0.22.0" "0.23.0" "0.24.0"
binman::list_versions("seleniumserver")
$generic
[1] "3.141.59" "4.0.0-alpha-1" "4.0.0-alpha-2"
it seems that the geckodriver is available (is it ?). But when I try to launch a driver :
> library(RSelenium)
> rD <- rsDriver(browser = "firefox",
+ extraCapabilities = list(
+ "moz:firefoxOptions" = list(
+ binary = "/usr/lib64/firefox/firefox",
+ args = list('--headless')
+ )
+ ))
...
[1] "Connecting to remote server"
Selenium message:invalid argument: can't kill an exited process
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
System info: host: 'login2.cluster', ip: '192.168.100.12', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-957.5.1.el7.x86_64', java.version: '1.8.0_181'
Driver info: driver.version: unknown
remote stacktrace:
Could not open firefox browser.
Client error message:
Summary: UnknownError
Detail: An unknown server-side error occurred while processing the command.
Further Details: run errorDetails method
Check server log for further details.
From this question and others I tried to downgrade the version of geckodriver, and make use of the headless mode of firefox:
rD <- rsDriver(browser = "firefox",
version = "3.141.59",
geckover = "0.22.0",
extraCapabilities = list(
"moz:firefoxOptions" = list(
binary = "/usr/lib64/firefox/firefox",
args = list('--headless')
)
))
But still get the same error.
My suspicion would be that geckodriver is actually not installed. Is this possible ? How to check it ?
Thank you for your help
You need to stick to your exact requirements regarding Selenium, GeckoDriver and Firefox versions which would be used in your test framework and remove the unwanted versions of binaries totally. GeckoDriver v0.24.0 being the latest release must be the chosen one.
Selenium v4.0.0-alpha-1 and Selenium v4.0.0-alpha-2 are alpha releases and must be avoided for Production usage. So Selenium v3.141.59 being the latest release must be the chosen one.
For GeckoDriver, Selenium and Firefox Browser compatibility you can find a detailed discussion in Which Firefox browser versions supported for given Geckodriver version?
Note: You don't need to install the GeckoDriver binary but put the binary in the desired location.
So an ideal usage would be:
rD <- rsDriver(browser = "firefox",
version = "3.141.59",
geckover = "0.24.0",
extraCapabilities = list(
"moz:firefoxOptions" = list(
binary = "/usr/lib64/firefox/firefox",
args = list('--headless')
)
))
If you are still facing the issue follow the below mentioned steps.
This error message...
message:invalid argument: can't kill an exited process
...can surface for different reasons. The possible solution can be any/either of the following:
Ensure that GeckoDriver v0.24.0 is downloaded and placed within the directory that is already in your path, e.g. /usr/local/bin
Ensure that GeckoDriver is having executable permission for non-root users.
Ensure that firefox (> v57.0) is installed (mandatory) within /usr/lib64/firefox/ directory as per your code block and the version is compatable.
Ensure that if you are running Firefox on a system with no display you have to use headless mode.
The correct usage of headless mode with GeckoDriver v0.24.0 is:
options.headless = True
There is no need for xvfb-run anymore if you set MOZ_HEADLESS=1 as follows:
$ export MOZ_HEADLESS=1 # this way you only have to set it once
If you have changed your system path, take a System Reboot.
Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.
Always execute your #Tests as a non-root user.
References
You can find a relevant detailed discussions in:
WebDriverException: Message: invalid argument: can't kill an exited process with GeckoDriver, Selenium and Python on RaspberryPi3
I know this question has been asked multiply times before but none of the solutions helps in my case.
I successfully did this no problem 3 months ago but am getting stuck now without changing anything which baffles me a lot and i really don't know what to do!
1:basic info:
win 7
Rselenium standalone 3.13.0 (tried 3.9.1 not working,used to work 3 months ago)
chrome version v64.0.3282.140 (32bit)
chromedriver 2.37 should be compatible.(also tried 2.38, 2.39 not work)
(I even updated chrome to 67 and use chromedriver 2.40 not work and then downgrade back)
2:launch Rselenium:
java -Dwebdriver.chrome.driver="C:\Program Files (x86)\Google\Chrome\Application\chromedriver.exe" -jar D:\selenium-server-standalone-3.13.0.jar
up and running no problem see the bottom pic-cmd snapshot
3:Rcode:
remDr <- remoteDriver(remoteServerAddr = "127.0.0.1"
, port = 4444
, browserName = "chrome")
remDr$open()
3 months ago it would launch a chrome page, but now the error shows:
Selenium message:Unable to create new service: ChromeDriverService
Build info: version: '3.13.0', revision: '2f0d292', time: '2018-06-25T15:32:19.891Z'
System info: host: 'A-PC', ip: '192.168.16.2', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '10.0.1'
Driver info: driver.version: unknown
Error: Summary: SessionNotCreatedException
Detail: A new session could not be created.
Further Details: run errorDetails method
I am not sure if it's because it doesn't detect the chrome version or the chromedriver version? But when I type chromedriver -version it shows the version is 2.37.
Please help!
cmd snapshot
I have had RSelenium fail with messages like that. I don't know the cause, but what worked to fix it was to delete all the downloaded driver files. On MacOS, they are in the ~/Library/Application Support/binman_* directories.
The next time I ran it, they were reinstalled automatically, and things worked fine again for a year or more.
Had the same issue. Yesterday everything was working fine, today also this Selenium message:Unable to create new service: ChromeDriverService error.
I saw a new chrome driver had downloaded overnight (on windows can be found in AppData/Local/binman/binman_chromedriver/win32/ - see picture below).
Deleting it did not help for me, because the non-working driver would be redownloaded again. It is not very pretty, but what worked for me was copying the chromedriver.exe file from the 2.42 folder into the 70.0.3538.16 folder.
I only had this issue with windows 8.1. On windows 10 the new driver did not cause issues.
On Linux, the RSelenium/Selenium seems to be behaving erratically. I start the server manually and it seems to start up fine. Sometimes I am able to connect to it from my R session and other times I get an error. I cannot yet pinpoint the cause: the same script seems to work sometimes and not other times. Any ideas?
Here is the output from starting the server:
12:41:25.811 INFO - Launching a standalone server
12:41:26.102 INFO - Java: Sun Microsystems Inc. 11.0-b16
12:41:26.102 INFO - OS: Linux 2.6.32-431.17.1.el6.x86_64 amd64
12:41:26.157 INFO - v2.44.0, with Core v2.44.0. Built from revision 76d78cf
12:41:26.492 INFO - Default driver org.openqa.selenium.ie.InternetExplorerDriver registration is skipped: registration capabilities Capabilities [{platform=WINDOWS, ensureCleanSession=true, browserName=internet explorer, version=}] does not match with current platform: LINUX
12:41:26.589 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
12:41:26.589 INFO - Version Jetty/5.1.x
12:41:26.590 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
12:41:26.590 INFO - Started HttpContext[/selenium-server,/selenium-server]
12:41:26.590 INFO - Started HttpContext[/,/]
12:41:36.597 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler#2993a66f
12:41:36.598 INFO - Started HttpContext[/wd,/wd]
12:41:36.601 INFO - Started SocketListener on 0.0.0.0:4444
12:41:36.601 INFO - Started org.openqa.jetty.jetty.Server#6f507fb2
The error I receive in my R session:
Couldnt connect to host on http://localhost:4444/wd/hub.
Please ensure a Selenium server is running.Error in queryRD(paste0(serverURL, "/session"), "POST", qdata = toJSON(serverOpts))
Other times, running the exact same script everything proceeds successfully. It could be a difference of what computer node actually executes the script but I cannot tell what the problem is when it will not connect.
library(RCurl)
library(RJSONIO)
library(XML)
# running selenium
system("java -jar selenium-server-standalone-2.44.0.jar")
I faced same issue and try to run above code. Finally know that there only support 1.7 but not 1.8 as time being. You might try to download jre7 and jdk7.
I faced the same error in Mac, but I noticed that due to security settings the system does not let me to run the file with that piece of code. I downloaded the selenium standalone file and using control key I single-clicked the file and then from the menu I chose the open. Then the problem get solved.