I have used RStudio to submit a job a few months ago to cloudml (AI platform) and it was successful.
Today I tried to use AI platform notebook to submit the same job but I get:
"ERROR: (gcloud.ai-platform.jobs.submit.training) INVALID_ARGUMENT: Field: runtime_version Error: The specified runtime version '1.9' with the Python version ''"
I even ran which python in the terminal and then in the R env.:
library(reticulate)
use_python("result of the which python")
I tried R in the terminal as well and get the same error.
I don't know if it helps or not but the previous run and this one were in different regions.
us-central was successful
australia-southeast1 was getting this error.
This error occurs because as of March 16, 2020, you can no longer create training jobs that use runtime version 1.9. You can try submitting the job with version 1.15 which is the only Tensorflow 1.x version that is currently supported for training jobs. It is still possible though that you may experience errors due to incompatibilities in the code.
Related
Anyone run into the R haven package call generating: ***Recursive gc invocation error in a shared environment?
Here is what I am running.
RStudio Ver. 2021.09.2 Build 382
R Ver. 4.1.2
Rtools Vers 4.0
Haven package release 2.4.3
The code works on my local install. I have users accessing a shared instance to work with large data files and can't get it to consistently run in that environ. The essence of the call is:
library(haven)
SFF05 <- read_sas("file/path/name/filename.sas7bdat”)
Users in this space often get this error off the library statement:
*******recursive gc invocation
Adding gc() to the start of the code will fix it temporarily, but eventually stops working.
My users also get this error in relation to the file read_sas call:
Error in df_parse_sas_file(….)
They occasionally give us other recursive errors or namespace errors. It’s not very consistent, and sometimes it just hangs until R crashes and reloads.
These programs worked flawlessly on R3.6.1 and RStudio July 19th, 2018 (1.1.456). Also, it works in R4.1.2 console just fine, it appears be an issue with the combination of R 4.1.2 and RStudio Ver. 2021.09.2 Build 382 in a shared environment.
This could be related to a bug in your RStudio, e.g see:
https://github.com/rstudio/rstudio/issues/9868
https://github.com/rstudio/rstudio/issues/10565
https://github.com/rstudio/rstudio/issues/10040
It seems to be fixed in the latest RStudio release which should arrive any day now:
https://github.com/rstudio/rstudio/milestone/20?closed=1
A temporary fix might be to set session-handle-offline-timeout-ms=0 in your rsession.conf. See: https://github.com/rstudio/rstudio/issues/10565#issuecomment-1035517692
I'm having issues with Azure Machine Learning SDK for R: "module 'azureml' has no attribute 'core'"...
For reasons that aren't my own, I have to use azureml to apply machine learning (my own stuff, written in R) to data from our data warehouse that is put in the blob storage. The modelled output should be put back into the blob storage so it can be accessed from the data warehouse.
I've written the code in R on my local machine (stored in a git repo). Preferably, I'd find some method to pull my code from git into a pipeline in the azureml environment so that it can be directly run whenever new data is available in the blob storage.
I've embarked on a tutorial-spree and found this seemingly relevant walkthrough: Train and deploy your first model with Azure ML (and this one).
But... after trying all I could think of, I'm stuck on the first steps. After installing all (or at least.. that's what I think) packages, modules, apps etc, and running the following code in RStudio:
library(azuremlsdk)
existing_ws <- get_workspace(name = name,
subscription_id = subscription_id,
resource_group)
I run into an error that I haven't been able to fix:
AttributeError: module 'azureml' has no attribute 'core'
It seems that the azuerml is supposed to have an attribute "core", but when looking at it more closely, there is indeed no such attribute.
The function "get_workspace()" is trying to access: "azureml$core$Workspace$get".
I found that "azuerML$Workspace" does exist, but then I can't figure out how to make that work.
Can anyone explain to me why I'm encountering this error?
Does anyone know of a better tutorial on how to connect my R code the azureml's cloud service?
Any pointers in the right direction are much appreciated!
EDITS - still not solved:
After advice from others, I double, triple and quadruple checked the installation.
I updated R and I'm now running:
R.version
platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 3
minor 6.2
year 2019
month 12
day 12
svn rev 77560
language R
version.string R version 3.6.2 (2019-12-12)
nickname Dark and Stormy Night
I installed Conda with Python 3.6.10.
I installed the azuremlsdk R package (I tried both provided options).
I then realized that there are some inconsistencies with the versions of the azure-modules, so I also tried installing it with the keyword 'multi-arch':
remotes::install_cran('azuremlsdk', repos = 'http://cran.us.r-project.org', INSTALL_opts=c("--no-multiarch"))
Then, I installed the azureml python sdk.
I had a look at all the versions again (using python -m pip freeze):
azure-common==1.1.24
azure-graphrbac==0.61.1
azure-mgmt-authorization==0.60.0
azure-mgmt-containerregistry==2.8.0
azure-mgmt-keyvault==2.0.0
azure-mgmt-resource==7.0.0
azure-mgmt-storage==7.1.0
azureml==0.2.7
azureml-automl-core==1.0.83.1
azureml-core==1.0.69
azureml-dataprep==1.1.36
azureml-dataprep-native==13.2.0
azureml-pipeline==1.0.69
azureml-pipeline-core==1.0.69
azureml-pipeline-steps==1.0.69
azureml-sdk==1.0.69
azureml-telemetry==1.0.69
azureml-train==1.0.69
azureml-train-automl-client==1.0.83
azureml-train-core==1.0.69
azureml-train-restclients-hyperdrive==1.0.69
As I was surprised to see all the 1.0.69 versions, instead of the 1.0.83 versions, I re-installed the azureml python sdk using:
azuremlsdk::install_azureml(version = "1.0.83")
This worked, in the sense that indeed all versions are now 1.0.83:
azure-common==1.1.24
azure-graphrbac==0.61.1
azure-mgmt-authorization==0.60.0
azure-mgmt-containerregistry==2.8.0
azure-mgmt-keyvault==2.0.0
azure-mgmt-resource==7.0.0
azure-mgmt-storage==7.1.0
azureml==0.2.7
azureml-automl-core==1.0.83.1
azureml-core==1.0.83
azureml-dataprep==1.1.36
azureml-dataprep-native==13.2.0
azureml-pipeline==1.0.83
azureml-pipeline-core==1.0.83
azureml-pipeline-steps==1.0.83
azureml-sdk==1.0.83
azureml-telemetry==1.0.83
azureml-train==1.0.83
azureml-train-automl-client==1.0.83
azureml-train-core==1.0.83
azureml-train-restclients-hyperdrive==1.0.83
But still... I get the error with the missing core. I get it both when running:
library(azuremlsdk)
get_current_run()
and also when running:
library(azuremlsdk)
existing_ws <- get_workspace(name = name,
subscription_id = subscription_id,
resource_group)
Note that the first time running this code after starting up RStudio, I get the error:
Error in py_get_attr_impl(x, name, silent) :
AttributeError: module 'azureml' has no attribute '_base_sdk_common'
And every time after that I get this error:
Error in py_get_attr_impl(x, name, silent) :
AttributeError: module 'azureml' has no attribute 'core'
Any help would be much appreciated!
This issue was introduced by the latest reticulate 1.14 release, in which reticulate would create a default r-reticulate conda environment. Since Azure ML was installing the python SDK in an environment named r-azureml, the r-reticulate environment used by reticulate was missing the python SDK. A fix for this issue was addressed in a PR and has been merged into master. Please install from GitHub for now if you have reticulate version 1.14 and are running into this issue. We will be releasing an update to CRAN shortly.
I seemed to have fixed the issue by specifically installing the python package azureml AND azureml.core:
python -m pip install azureml
and then...
python -m pip install azureml.core
I did this for the Conda version that was called by R (r-reticulate). It's a bit odd to not be able to use the Conda environment 'r-azureml' without R switching back to 'r-reticulate', but ah well... at least I don't get my 'azureml' has no attribute 'core' anymore.
I am using Eclipse 2018-09 and StatET 3.6.1. I have two R environments: one for R 3.3.2, another for R 3.5.1. When I execute a command like ?lm to call up a help page in v3.3.2, the page appears in the StatET "R Help" pane, as I want it to. But when I execute the same command to call up a help page in v3.5.1, the console tells me "starting httpd help server," and the help page loads in my browser. How can I get help to load in the "R Help" pane when using v3.5.1?
The problem arises whether or not I run Eclipse as an administrator, and whether or not I "Auto Run" R 3.5.1 upon booting. And as far as I can tell, the configurations for the two versions of R are nearly identical:
Both are running Windows 10 and JRE 1.8.0_121.
Both with version 2.1 of the rj package.
For both, I've checked every box in "Run Configurations > R Console > R Console > Eclipse Integration," including "Enable R Help for StatET."
For both, options("help_type") == 'html'.
Inspection of the different log files that I get when using Auto Run with the two versions of R suggests that there is some sort of StatET config problem that has to do with Derby. These messages appear only when I boot R 3.5.1:
!ENTRY de.walware.statet.r.core 4 -1 2018-11-16 02:50:29.092
!MESSAGE An error occurred when initializing DB for model.
!STACK 1
org.eclipse.core.runtime.CoreException: An error occurred when loading embedded DB (Derby + DBCP)
DB ConnectionURL=[path redacted]\eclipse-workspace\.metadata\.plugins\de.walware.statet.r.core\db
[...]
Caused by: java.sql.SQLException: Another instance of Derby may have already booted the database C:\[path redacted]\eclipse-workspace\.metadata\.plugins\de.walware.statet.r.core\db.
[...]
Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database C:\[path redacted]\eclipse-workspace\.metadata\.plugins\de.walware.statet.r.core\db.
But I don't know what to make of this problem or how to fix it.
Stephan Wahlbrink, the creator of StatET, recommended running this command at startup:
registerS3method("print", "help_files_with_topic", rj::print.help_files_with_topic)
I ran the command, and it solved the problem.
(Stephan added that the Derby errors that I saw in the logs were unrelated to the problem of displaying help in the "R Help" pane.)
I've been trying to produce Kernel Density Estimates using the "kde" tool from
Geospatial Modeling Environment (GME, see documentation on kde). But I keep getting the following error regardless of valid input:
Code:
kde(in="C:\Users\Richard\Desktop\KDE_Scripting_Local\kde.gdb!BB_90sJAN",
out="C:\Users\Richard\Desktop\KDE_Scripting_Local\kde.gdb!KDE_BB90sJAN",
bandwidth="100000", cellsize=6000, kernel="QUARTIC",
ext="C:\Users\Richard\Desktop\KDE_Scripting_Local\kde.gdb!rect_extent");
Error message:
Error: The command text could not be interpreted. Please check the syntax of the command. Error: An important error has occurred. Please include the information below if you submit a query about this error.
Exception from HRESULT: 0x8004025A
The most frustrating part is that I had this exact code working last week. I tried restarting, reinstalling GME, copying the input to a new GDB as suggested here, subprocesses with PYTHON 2.7. Everything still produces this error with the same HRESULT.
I'm running GME Version 0.7.3.0, ArcGIS For Desktop 10.2.2, R Version 3.1.1, and Python 2.7 on Windows 7. There's not much community support for GME, so any help here would be much appreciated.
I'm using Plone 4.1 and trying to run bin/buildout in a fresh directory, having just managed to get python bootstrap.py --distribute to work. bin/buildout runs along fine for a couple of minutes, downloading various distributions, then crashes with the following report:
Getting distribution for 'plone.recipe.zope2instance==4.1.7'.
Got plone.recipe.zope2instance 4.1.7.
... (many distributions omitted for brevity) ...
Getting distribution for 'zope.container==3.11.2'.
Got zope.container 3.11.2.
Getting distribution for 'zope.configuration==3.7.4'.
error: Not a recognized archive type: /Users/Jon/.buildout/downloads/dist/zope.configuration-3.7.4.zip
An error occured when trying to install zope.configuration 3.7.4. Look above this message for any errors that were output by easy_install.
While:
Installing.
Getting section instance.
Initializing section instance.
Installing recipe plone.recipe.zope2instance.
Getting distribution for 'zope.configuration==3.7.4'.
Error: Couldn't install: zope.configuration 3.7.4
It looks like the important error message is: Not a recognized archive type: /Users/Jon/.buildout/downloads/dist/zope.configuration-3.7.4.zip
Incidentally, I tried simply re-running bin/buildout and it picked up in a new place, starting with:
Getting distribution for 'zdaemon==2.0.4'.
Got zdaemon 2.0.4.
Getting distribution for 'pytz==2011g'.
Got pytz 2011g.
...and so on. Can I just keep retrying the script until it successfully get all the stuff that it needs?
Also, I tried running bin/buildout for version 4.2 in another directory, and that seemed to work okay.
EDIT The second time I ran bin/buildout it completed without crashing. However, I did a search of the output and it never once downloaded anything with "configuration" in it's name. So what is going on???
EDIT Just for good measure I tried running bin/buildout a third time and it exited quickly after printing the single line "Updating instance."
You're fine. I suspect you have a less than ideal network connection and/or that PyPI is having problems (one of your other posts was about network related issues as well, I think).
The error you saw is that it's tried to download a zip file that's been corrupted, and so it fails to extract the zip file. On your second attempt, it downloaded it successfully.