Can no longer run RSelenium Chrome Driver - r

I'll start by saying I have ran into this issue many times in the past, and all that's been required is updating my Google Chrome version and then updating the chromever = param in rsDriver(). These are no longer working for me.
I've tried the solutions in many posts (this one in particular) but I still can't get it to work.
Here are some details:
Computer/browser/R info:
Chrome Version: 89.0.4389.90
Mac Version: 10.15.4
RStudio Version: 1.3.959
For the longest time, I've been able to use chromever = "87.0.4280.20" even though my browser wasn't on that version. I could open up a remote driver with remDrall <- rsDriver(port = 4445L, browser = "chrome", chromever = "87.0.4280.20")
When I try this now, I get an error saying
Selenium message:session not created: This version of ChromeDriver only supports Chrome version 87
Current browser version is 89.0.4389.90 with binary path /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
This prompted me to try updating my ChromeDriver if it was stuck on 87. I updated it by manually downloading and moving to /usr/local/bin/chromedriver as well as updating via brew upgrade chromedriver. As far as I know it worked, testing via:
ls /usr/local/Caskroom/chromedriver/ gives me 89.0.4389.23
/usr/local/bin/chromedriver starts a chromedriver session with 89.0.4389.23
I've tried using chromever = "89.0.4389.23" but I get an unknown server-side error. I know my chrome version is 89.0.4389.90, but that version isn't available to use in rsDriver.
I'm really not sure where to go from here. I just don't get why it says my ChromeDriver only supports Chrome version 87 when I clearly have it updated to 89. Could it be that my rsDriver function is still picking up some old version of ChromeDriver? Can I direct it specifically to the one in /usr/local/bin/?
Any thoughts on what I can try next?

Try to replace the chromedriver.exe file on you local directory with the updated version from here:
https://chromedriver.storage.googleapis.com/index.html?path=89.0.4389.23/

I ran into a similar problem last week and was able to get the command working again by making the following change to the chromever option:
Previous version that has stopped working:
rsDriver(chromever = "87.0.4280.88", browser = "chrome", extraCapabilities = eCaps)
New version that works:
rsDriver(chromever = "89.0.4389.23", browser = "chrome", extraCapabilities = eCaps)

Related

RSelenium:: Connection refused

I am trying (for the first time) to scrape content from a dynamic webpage, for which RSelenium appears to be the go-to. I cannot however get past the first step of calling rsDriver.
My code:
rdriver <- rsDriver(browser = "chrome",
port = free_port(),
chromever = "109.0.5414.25")
The rsDriver() function started throwing an error every time I tried to open it
[1] "Connecting to remote server" Could not open chrome browser.
Client error message: Undefined error in httr call. httr output:
Failed to connect to localhost port 14415: Connection refused Check
server log for further details. Warning message: In rsDriver(browser =
"chrome", port = free_port(), chromever = "109.0.5414.25") : Could
not determine server status.`
Version:
R 4.2.2
Java(TM) SE Development Kit 19.0.2 (64 bit)
> binman::list_versions("chromedriver")
$win32
[1] "109.0.5414.25" "109.0.5414.74" "110.0.5481.30"
> binman::list_versions("seleniumserver")
$generic
[1] "3.141.59" "4.0.0-alpha-1" "4.0.0-alpha-2"`
Any recommendations are much appreciated.
I installed all the necessary programs from scratch.
I searched for help on the internet and couldn't find a solution
there is an outstanding issue with how the wdman package reads the latest versions of Chrome. This is causing issues for lots of users (examples here https://stackoverflow.com/a/75176907/15363011 and here https://github.com/ropensci/RSelenium/issues/264)
You can specify a version of Chrome before the issue took place and binman/wdman will download and start using it:
rdriver <- rsDriver(browser = "chrome",
port = free_port(),
chromever = "108.0.5359.71")
If you'd like to use the newest versions, the fix is to delete the LICENSE.chromedriver file found in the same directory as the driver. You can find out how to do that in the other issues that I linked. If you want to use the latest version of Chrome you will have to do this any time a new chrome driver is released.

Problems with RSelenium and ChromeDriver - "Could not open chrome browser"

I have been using RSelenium for years and have never had this issue. I recently updated my google chrome to the latest version available 110.0.5481.78. I am now getting the following error when I go to use rsDriver
require(RSelenium)
rD <- rsDriver(browser = "chrome",port = 9537L, chromever = "110.0.5481.77")
"> Could not open chrome browser.
> Client error message:
> Undefined error in httr call. httr output: Failed to connect to localhost port 9537: Connection refused
> Check server log for further details.
> Warning message:
> In rsDriver(browser = "chrome", port = 9537L, chromever = "110.0.5481.77") :
> Could not determine server status."
R Console
I have tried with different versions of chromever from binman::list_versions("chromedriver") as well as leaving rsDriver blank all together. In the past when chrome has updated it has been a very simple change to chromever and everything works perfectly. Not sure if or what has changed with this latest update.
Thanks in advance.
I just fixed this same problem by removing a file LICENSE.chromedriver as per this thread: https://github.com/ropensci/RSelenium/issues/264
Use
wdman::selenium(retcommand=T)
to find the file location of binman_chromedriver files.
Navigate to this file location, go to the driver version you're using and delete the LICENSE.chromedriver file. Mine worked immediately after this action, but note that I also tried downgading wdman version to 0.2.5 (I was on 0.2.6) first:
remotes::install_version('wdman',version = '0.2.5')
I'm not sure if it was both actions that fixed it or just the file delete!

RSelenium - starting server in the background (without window popup)

I recently got a new MacBook which seems to have changed several RSelenium settings. I've gotten my scraping to work properly again, except when I start a server with rsDriver, an actual window pops up and I don't know how to stop it.
I do like having this feature because it makes it so much easier to debug, but I run several scripts in the background while working and whenever rsDriver gets used it becomes the desktop focus. This is very frustrating since it disrupts whatever I'm currently working on.
To start a driver, I run:
remDrall <- rsDriver(port = 4446L, browser = "chrome", chromever = "84.0.4147.30", verbose = F)
When it's completed, a chrome window pops up with the url = "data:,". I feel like it should be an easy fix (like a silent = TRUE param), but I've looked through the documentation and can't find anything.
Running Catalina v 10.15.4
RSelenium package is v 1.7.1
Side note, but I'm not really sure how my rsDriver statement is working, since the port isn't the same as my docker image. I open up my docker with $ docker run -d -p 4567:4444 selenium/standalone-chrome, but if I try to use rsDriver(port = 4567L) then it doesn't work because it says the port is already in use. So I don't even have a docker image with port 4446, but my rsDriver statement still works...
Try adding:
extraCapabilities = list("chromeOptions" = list(args = list('--headless')))
as an extra argument to rsDriver.

Why do I still have Selenium chromedriver mismatch problem when I have the correct chromedriver downloaded and saved to the path

My problem is related to this post: session not created: This version of ChromeDriver only supports Chrome version 74 error with ChromeDriver Chrome using Selenium. Basically there is a mismatch between the version of chrome and chromedriver that's sourced by the code.
I'm running chrome 73.0.3683.86 (Official Build) (32-bit) on a corporate computer (so can't be upgraded) and have downloaded chromedriver (v73.0.3683.68) which has been saved to the path (saved to users path as I can't access system path). R version is 3.6.2. RSelenium version is 1.7.7.
cprof <- getChromeProfile("C:/Users/sizhu/AppData/Local/Google/Chrome/UserData/Default","Default")
rD1 <- rsDriver(browser = "chrome",chromever = "73.0.3683.68",extraCapabilities = cprof)
When I ran the above lines, it gives me error:
version requested doesn't match versions available = 80.0.3987.106,80.0.3987.16,81.0.4044.20,81.0.4044.69
I have run binman::list_versions("chromedriver") to see what chrome driver version is sourced, it shows the ones above in bold and not the one I saved in the path. Is there a way to force the program to use the chrome driver I downloaded? (sorry I'm a newbie to programming in general so it might just be very trivial...)
Thanks very much in advance!
update not quite an eventual solution to this but made some changes so the codes can now open up the chrome browser: 1)going into wdman>yaml>chromedriver 2)change history to 20 (it was 3 hence every time I run this line, 3 latest chromever drivers (v80-81) are downloaded to binman; well since what I need is v73, I need to go back 20 versions) 3)save and specify chromever = "73.0.3683.68" which can now be found. The problem with this approach is obvious, and still doesn't solve the puzzling fact that why the heck the v73 chromedriver that I've saved to the path is not found
If you are using Chrome version 81, please download ChromeDriver 81.0.4044.69
If you are using Chrome version 80, please download ChromeDriver 80.0.3987.106
If you are using Chrome version 79, please download ChromeDriver 79.0.3945.36
https://chromedriver.chromium.org/downloads
😀😀😀
Had a similar issue, and this worked for me.
Check the selenium server version: binman::list_versions("seleniumserver"),
and then in your rsDriver(), specify the version out, not using "latest" or default.
rD1 <- rsDriver(browser = "chrome", chromever = "73.0.3683.68", version = "the version number you got", extraCapabilities = cprof)

Chromedriver vs. Chrome update incompatibility

I was working on setting up RSelenium in R to interact with Chrome; however, I keep receiving an error message that the Chrome driver can't work with my version of Chrome even though I already specified the version of the Chromedriver to match Chrome on my desktop.
Below is the code producing an error: (MacOS Mojave Version 10.14.5)
library(RSelenium)
library(xml2)
library(rvest)
library(tidyverse)
library(wdman)
library(binman)
remDr <- RSelenium::remoteDriver(remoteServerAddr = "localhost",
port = 4445L,
browserName = "chrome")
remDr$open()
binman::list_versions("chromedriver")
rD <- rsDriver(browser = "chrome", chromever="75.0.3770.90")
list versions from binman are:"75.0.3770.90" "76.0.3809.12" "76.0.3809.25"
The error that I kept receiving is as follows:
Selenium message:session not created: This version of ChromeDriver only supports Chrome version 76
(Driver info: chromedriver=76.0.3809.25 (a0c95f440512e06df1c9c206f2d79cc20be18bb1-refs/branch-heads/3809#{#271}),platform=Mac OS X 10.14.5 x86_64)
However, I checked the version that Chrome is updated to and it is 75.0.3770.100 so I assumed that the chrome driver that I specified would be suffice.
I tried a couple different methods such as adding the following functions; however, I keep receiving the same error.
eCaps <- list(chromeOptions = list(
args = c('--no-sandbox','--headless', '--disable-gpu', '--window-size=1280,800')
))
cDrv <- chrome()
I was wondering if there was anyway to remove the higher versions of chrome driver so there is only one chrome driver the code to possibly use. Any other solutions would also be very much appreciated!

Resources