I am new to Rselenium and I want to use it to scrape data from a website. When trying to establish a connection I get the error that my chrome driver is not compatible with my current browser version, here is the code and the error I got so far
library(RSelenium)
library(RCurl)
driver <- rsDriver()
remDr <- driver[["client"]]
remDr$open()
> remDr$open()
[1] "Connecting to remote server"
Selenium message:session not created: This version of ChromeDriver only supports Chrome version 97
Current browser version is 96.0.4664.45 with binary path C:\Program Files (x86)\Google\Chrome\Application\chrome.exe
The error report is very long, but I think the essence of the error is described in the first sentence; my chromedriver and current version of chrome are not compatible.
So I did some research, since I don't know what a chrome driver is. I found out I can download older versions of chrome driver. I searched the 96.0.4664.45 version since my browser is that version (it seems there is only a 32 bit version, which I thought was odd but its the only one for windows so I downloaded that one).
After I downloaded the chromedriver.exe, I moved it to a specific folder and I added that folder to my system path as I read online this is what I have to do in order for R to recognize. Then, I used the cmd prompt to execute the chrome driver, which seems to work;
Then when I start up R and retry, I still got the same error, so it seems I am still doing something wrong. Do I need to 'overide' the chromedriver 97? And why do I even have a version 97 when my browser is a 96? Any help would be greatly appreciated.
Related
Main purpose: Use selenium in non-internet private network with R code (Ubuntu 20.04).
Steps: Install Java, install Selenium Server 4.7.2, install Chrome (specific version), Download and use chromeDriver (same version as Chrome), Download and used desire R package (RSelenium) and start coding.
library("RSelenium")
rd <- rsDriver()
rd$open()
Problem: When I call open function I face this error
checking Selenium Server versions:
BEGIN: PREDOWNLOAD
Error in open.connection(con, "rb") :
Could not resolve host: www.googleapis.com
I do some R&D and find Selenium need to download some relevant driver files! Our server are in private network and there is not any proxy for internet at all. So regardless of I use of R on any other languages, can I use Selenium in non-internet private network? If yes which files should I download offline and where should I copy them?
Thanks in advance
I think the issue here is that rsDriver creates both the server and the client. As such it includes a wrapper for the function wdman::selenium() that is meant to download and manage the drivers needed. I would look into one of the two options: 1) using rsDriver() as the package manager and let it download the drivers for you or 2) using remoteDriver() on its own (which will not install drivers) to connect to your Selenium Server instead.
In the description for rsDriver:
A list containing a server and a client. The server is the object returned by selenium() and the client is an object of class remoteDriver()
For people who wants to use Selenium in private non internet network:
As #bingbongtelecom mention rsDrive() manage to download some drives as chromeDriver, Phantomjs, geckodriver and etc to use them. You should download them in other network and copy them in your private network. After that use 'check = False' option to stop checking driver and download process
Regards
I'm having a difficult time getting RSelenium to work with my ChromeDriver in R v4.1.2. Initially, I installed the ChromeDriver v97 because I had Chrome v97 running on my Mac as my browser. However, despite installing the v97 ChromeDriver, I would get an error every time that read
Selenium message:session not created: This version of ChromeDriver only supports Chrome version 98
So I did some research and everything I read said I could either upgrade Google Chrome or downgrade my driver. I tried both. I uninstalled the driver and installed an earlier version (v96) but it still gave me the exact same error. Then I tried upgrading my Chrome browser, but it said it was up to date and that no upgrade was available. So I just figured I'd wait until v98 was ready to install. As soon as v98 was out, I upgraded my browser to Chrome v98 and Selenium with the ChromeDriver ran smoothly without issues. I haven't changed anything since. It ran for a couple of days, but now, all of a sudden, I'm getting the error again but now it says
"This version of ChromeDriver only supports Chrome version 99"
But I have not upgraded my ChromeDriver since installing the v97 driver. It seems as though the driver must have some code in it that checks what the most current version of Chrome is (even if that is a Beta release) and requires that I have it, regardless of which driver or browser I have installed. Has anyone come across this issue before? Any idea of how I can resolve this? I cannot upgrade to v99, it's not available yet, and uninstalling my ChromeDriver and reinstalling a previous version doesn't seem to do anything either.
Thank you in advance.
To know the chrome version use,
binman::list_versions("chromedriver")
$win32
[1] "85.0.4183.87" "86.0.4240.22" "87.0.4280.20" "87.0.4280.88" "88.0.4324.27" "91.0.4472.101" "91.0.4472.19" "92.0.4515.107" "92.0.4515.43"
Then you can try the versions which works for you,
library(RSelenium)
driver <- rsDriver(browser = "chrome",port = 9537L, chromever = "96.0.4664.45")
or else you can use firefox,
driver = rsDriver(port = 4841L, browser = c("firefox"))
The Mac has problems because there are two architectures (Intel and M1), and the code that loads the RSelenium driver hasn't been updated since the M1 came out.
This patch https://github.com/ropensci/wdman/pull/26 fixed the issue for me, working on an Intel Mac. You should also see this issue: https://github.com/ropensci/RSelenium/issues/221, which offers some code to sequentially try drivers until it finds one that works.
I am going bonkers as I switched from a windows tablet to my macbook pro earlier today and cannot seem to run the identical (seemingly trivial) command in R4.02, namely to download a zip file located at the following link:
http://askebsa.dol.gov/FOIA%20Files/2018/Latest/F_5500_SF_2018_Latest.zip
I used the R base syntax of: download.file(url, destfile= "xyz.csv") and it worked as expected in windows, but when I run it in Mac, I get 'HTTP status was '400 Bad Request'. I can manually download it in Mac, I just need to do it in R.
Any ideas if this a cache, cookie or some other issue?
I'm trying to execute code on R with the package RSelenium to do some webscraping, but I'm blocked at the very first step. After loading the library, I try to run this line of code:
rmDr <- rsDriver(browser = "chrome", chromever = 'latest')
But the console returns :
Error in java_check() :
PATH to JAVA not found. Please check JAVA is installed.
Java is indeed installed on my computer, but I'm guessing the path isn't the one the package is waiting for. Does someone know where I could modify the path in the RSelenium package code so that I can run this?
To be noted, I'm working on a company computer and so I don't have every admin rights.
Thanks for your help!
You can use method "remoteDriver()" instead of "rsDriver()". I checked it today with the last stable version of selenium driver (3.141.59) and it works perfectly.
Here's the code sample:
library(RSelenium)
driver <- remoteDriver()
driver$open()
driver$navigate("https://www.google.com/")
Just got the same error, installed the latest Java Development Kit (JDK), restarted a machine and everything worked ok.
The best way to use RSelenium is to go through Docker.
I used this tutorial, https://rpubs.com/johndharrison/RSelenium-Docker, not a long time ago, all went smooth.
Besides, you need a debugger, you can not scrape without it. That's why this tutorial is a good idea.
Let know if anything goes wrong.
I have written a simple test case to open Firefox browser and go to "https://www.google.co.in". I am getting this error.
Test script
*** Settings ***
Library Selenium2Library
*** Test Cases ***
case1
Open Browser https://www.google.co.in/ ff
To Run in Firefox we need geckodriver. Place geckodriver in C:/python/scripts and then try to run the scripts.
To Run in Chrome, place chrome driver X32 in C:/python/scripts and then run your scrpts
you need download driver firefox first
Download Here
Searching for the error: "Selenium OSerror [Errno 8] firefox" in Google the first few results were quite informative and it seems to me that this was not done.
The first few options highlight that not the correct webdriver is installed. Example here. (A different error is presented when the Gecko driver is not installed).
In a follow-up result item an explanation was given that the script is missing a shebang: #!/usr/bin/env bash when a script is fired.
Check the browser version and browser operation system(32 bit or 64 bit) then install the driver put inside python installation directory.
For Example :
C:\Python27/yourinstalleddriver.exe
either
C:\Python27\Scripts/yourinstalleddriver.exe
Most of the links relate to chrome driver not firefox. Well I degrade my firefox version and selenium version and this was solved.