Latest version of RSelenium and Firefox - r

When I try to open the RSelenium I receive this error
[1] "Connecting to remote server"
Error: Summary: UnknownError
Detail: An unknown server-side error occurred while processing the command.
class: org.openqa.selenium.firefox.NotConnectedException
The version of Firefox I have is
Firefox version: 480b10
According to this I tried to update the server version
library("RSelenium")
startServer()
unlink(system.file("bin", package = "RSelenium"), recursive = T)
checkForServer(update = TRUE)
remDr <- remoteDriver()
Sys.sleep(5)
remDr$open()
Sys.sleep(5)
but the problem still exist. Does anyone face this problem? Any possible solution?

From Firefox 48 on-wards the gecko driver/ marionette will be needed to run Firefox with Selenium.
If you have Firefox 48 you can run the gecko driver as follows:
Refer to the guidelines
https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/WebDriver
Download the relevant gecko driver from https://github.com/mozilla/geckodriver/releases
Add it to your PATH or refer to the location when starting binary (see below)
# get beta selenium standalone
RSelenium::checkForServer(beta = TRUE)
# assume gecko driver is not in our path (assume windows and we downloaded to docs folder)
# if the driver is in your PATH the javaargs call is not needed
selServ <- RSelenium::startServer(javaargs = c("-Dwebdriver.gecko.driver=\"C:/Users/john/Documents/geckodriver.exe\""))
remDr <- remoteDriver(extraCapabilities = list(marionette = TRUE))
remDr$open()
....
....
remDr$close()
selServ$stop()
The above currently requires the dev version of RSelenium. Alternatively you can download the Selenium binary from http://selenium-release.storage.googleapis.com/index.html . Pick the 3.0 beta 2 binary to currently run with Firefox 48. Run the binary
java -Dwebdriver.gecko.driver=C:/Users/john/Documents/geckodriver.exe -jar selenium-server-standalone-3.0.0-beta2.jar

Related

RSelenium:: Connection refused

I am trying (for the first time) to scrape content from a dynamic webpage, for which RSelenium appears to be the go-to. I cannot however get past the first step of calling rsDriver.
My code:
rdriver <- rsDriver(browser = "chrome",
port = free_port(),
chromever = "109.0.5414.25")
The rsDriver() function started throwing an error every time I tried to open it
[1] "Connecting to remote server" Could not open chrome browser.
Client error message: Undefined error in httr call. httr output:
Failed to connect to localhost port 14415: Connection refused Check
server log for further details. Warning message: In rsDriver(browser =
"chrome", port = free_port(), chromever = "109.0.5414.25") : Could
not determine server status.`
Version:
R 4.2.2
Java(TM) SE Development Kit 19.0.2 (64 bit)
> binman::list_versions("chromedriver")
$win32
[1] "109.0.5414.25" "109.0.5414.74" "110.0.5481.30"
> binman::list_versions("seleniumserver")
$generic
[1] "3.141.59" "4.0.0-alpha-1" "4.0.0-alpha-2"`
Any recommendations are much appreciated.
I installed all the necessary programs from scratch.
I searched for help on the internet and couldn't find a solution
there is an outstanding issue with how the wdman package reads the latest versions of Chrome. This is causing issues for lots of users (examples here https://stackoverflow.com/a/75176907/15363011 and here https://github.com/ropensci/RSelenium/issues/264)
You can specify a version of Chrome before the issue took place and binman/wdman will download and start using it:
rdriver <- rsDriver(browser = "chrome",
port = free_port(),
chromever = "108.0.5359.71")
If you'd like to use the newest versions, the fix is to delete the LICENSE.chromedriver file found in the same directory as the driver. You can find out how to do that in the other issues that I linked. If you want to use the latest version of Chrome you will have to do this any time a new chrome driver is released.

Problems with RSelenium and ChromeDriver - "Could not open chrome browser"

I have been using RSelenium for years and have never had this issue. I recently updated my google chrome to the latest version available 110.0.5481.78. I am now getting the following error when I go to use rsDriver
require(RSelenium)
rD <- rsDriver(browser = "chrome",port = 9537L, chromever = "110.0.5481.77")
"> Could not open chrome browser.
> Client error message:
> Undefined error in httr call. httr output: Failed to connect to localhost port 9537: Connection refused
> Check server log for further details.
> Warning message:
> In rsDriver(browser = "chrome", port = 9537L, chromever = "110.0.5481.77") :
> Could not determine server status."
R Console
I have tried with different versions of chromever from binman::list_versions("chromedriver") as well as leaving rsDriver blank all together. In the past when chrome has updated it has been a very simple change to chromever and everything works perfectly. Not sure if or what has changed with this latest update.
Thanks in advance.
I just fixed this same problem by removing a file LICENSE.chromedriver as per this thread: https://github.com/ropensci/RSelenium/issues/264
Use
wdman::selenium(retcommand=T)
to find the file location of binman_chromedriver files.
Navigate to this file location, go to the driver version you're using and delete the LICENSE.chromedriver file. Mine worked immediately after this action, but note that I also tried downgading wdman version to 0.2.5 (I was on 0.2.6) first:
remotes::install_version('wdman',version = '0.2.5')
I'm not sure if it was both actions that fixed it or just the file delete!

RSelenium can't connect to host rsDriver()

Until recently, I was using RSelenium without any problem. When I try to connect to hostr rsDriver() to start a selenium server and browser, I get the following error message:
Could not open firefox browser.
Client error message:
Undefined error in httr call. httr output: Failed to connect to localhost port 5111: Connection refused
Check server log for further details.
Warning message:
In rsDriver(browser = "firefox", port = 5111L, verbose = F) :
Could not determine server status.
I have looked at some answers already proposed, like in this page. However, when I run the command,
> selServ <- wdman::selenium(verbose = FALSE)
> selServ$log()
I get the following error message:
$stderr
[1] "Error: Could not find or load main class c(-Dwebdriver.chrome.driver="C:\\\\Users\\\\user\\\\AppData\\\\Local\\\\binman\\\\binman_chromedriver\\\\win32\\\\109.0.5414.25.chromedriver.exe\","
$stdout
character(0)
I tried to reinstall Java, but without success.
Ok I just solved the issue. The issue was that RSelenium recognises the chromedriver even when using other browsers so when the driver is not compatible to your current chrome version, it would have an issue with rsDriver. In order to solve this, you can either input NULL next to chromever or older chrome driver version from C:\Users\NAME\AppData\Local\binman\binman_chromedriver\win32
rD <- rsDriver(port= sample(7600)[1], browser=c("firefox"), chromever = NULL)
rD <- rsDriver(port= sample(7600)[1], browser=c("firefox"), chromever = 108.0.5359.22)

Chromedriver vs. Chrome update incompatibility

I was working on setting up RSelenium in R to interact with Chrome; however, I keep receiving an error message that the Chrome driver can't work with my version of Chrome even though I already specified the version of the Chromedriver to match Chrome on my desktop.
Below is the code producing an error: (MacOS Mojave Version 10.14.5)
library(RSelenium)
library(xml2)
library(rvest)
library(tidyverse)
library(wdman)
library(binman)
remDr <- RSelenium::remoteDriver(remoteServerAddr = "localhost",
port = 4445L,
browserName = "chrome")
remDr$open()
binman::list_versions("chromedriver")
rD <- rsDriver(browser = "chrome", chromever="75.0.3770.90")
list versions from binman are:"75.0.3770.90" "76.0.3809.12" "76.0.3809.25"
The error that I kept receiving is as follows:
Selenium message:session not created: This version of ChromeDriver only supports Chrome version 76
(Driver info: chromedriver=76.0.3809.25 (a0c95f440512e06df1c9c206f2d79cc20be18bb1-refs/branch-heads/3809#{#271}),platform=Mac OS X 10.14.5 x86_64)
However, I checked the version that Chrome is updated to and it is 75.0.3770.100 so I assumed that the chrome driver that I specified would be suffice.
I tried a couple different methods such as adding the following functions; however, I keep receiving the same error.
eCaps <- list(chromeOptions = list(
args = c('--no-sandbox','--headless', '--disable-gpu', '--window-size=1280,800')
))
cDrv <- chrome()
I was wondering if there was anyway to remove the higher versions of chrome driver so there is only one chrome driver the code to possibly use. Any other solutions would also be very much appreciated!

RSelenium through docker

My OS is windows 8.1 and I have the version 3.3.3 of R.
I have installed the RSelenium packages and I try to run it using this:
library("RSelenium")
#start RSelenium server
startServer()
checkForServer()
and I receive this error:
Error: checkForServer is now defunct. Users in future can find the function in
file.path(find.package("RSelenium"), "examples/serverUtils"). The
recommended way to run a selenium server is via Docker. Alternatively
see the RSelenium::rsDriver function.
Is there anything changed in the way RSelenium opens? I search for the error and I found only this but it doesn't help me. What can I do?
Also an alternative I tried is to download the chromedrive from here 'https://sites.google.com/a/chromium.org/chromedriver/downloads'
and using this script:
require(RSelenium)
cprof <- getChromeProfile("C:/Users/Peri/Desktop/chromedriver/chromedriver.exe", "Profile 1")
require(RSelenium)
remDr <- remoteDriver(remoteServerAddr = "localhost"
, port = 4444
, browserName = "chrome", extraCapabilities = cprof
)
remDr$open()
and I receive this error:
Error in checkError(res) :
Couldnt connect to host on http://localhost:4444/wd/hub.
Please ensure a Selenium server is running.
what can I do to run chrome instead of the pre-default browser Firefox?
You need to use the function rsDriver. The Selenium Version wants you to use Docker (which I also would recommend), but if you are not familiar with this you can go this way.
rsdriver will manage the binaries needed for running a Selenium Server. This provides a wrapper around the wdman::selenium function.
Here is what you have to do to start a Chrome Browser:
driver<- rsDriver()
remDr <- driver[["client"]]
And then you can work with it:
remDr$navigate("http://www.google.de")
remDr$navigate("http://www.spiegel.de")
And stop it:
remDr$close()

Resources