I managed to generate keys and tokens to be able to access google cloud storage files from within the r console.
However, upon executing the code below, i run into below error -
Error: $installed not found in JSON - have you downloaded the correct JSON file?
(Service account client > Desktop, not Service Account Keys)
While the code has been addressed/provided from previous SO post, the error is still enigmatic to me.
-- SO post below
Accessing files from Google cloud storage in RStudio
-- full code below
rm(list = ls(all.names = TRUE))
options(java.parameters = "-Xmx8000m")
library(googleCloudStorageR)
library(googleAuthR)
library(jsonlite)
# for working in google cloud storage
GCS_AUTH_FILE = "serviceaccount.json"
GAR_CLIENT_WEB_JSON = "Oauthclient.json"
# fromJSON(GCS_AUTH_FILE)
# fromJSON(GAR_CLIENT_WEB_JSON)
gar_auth_service(GCS_AUTH_FILE)
#set the scope
gar_set_client(scopes = c("https://www.googleapis.com/auth/devstorage.read_write",
"https://www.googleapis.com/auth/cloud-platform"), json = GAR_CLIENT_WEB_JSON
)
It seems that you are not using the correct JSON file. I found this Github post that mentions how to configure this correctly.
These are the steps:
Enable Google Cloud Storage API
Credentials >> Create Credentials >> OAuth Client ID >> Other
Save the JSON file as something like my-ClientID-credentials.json
Credentials >> Create Credentials >> Service Account Key
Save the JSON file as something like my-service-account-credentials.json
This is the code mentioned in the post
# Google cloud storage
# Setup
library(googleCloudStorageR)
options(googleAuthR.scopes.selected = "https://www.googleapis.com/auth/devstorage.full_control")
# Authenticate ClientID
googleAuthR::gar_set_client("path/to/my-ClientID-credentials.json")
# Authenticate service account
Sys.setenv("GCS_AUTH_FILE" = "full/path/to/my-service-account-credentials.json")
googleCloudStorageR::gcs_auth()
## Get bucket info
gcs_list_buckets(projectId = "my-project")
Related
Locally I have no issue authenticating my google drive account and pushing all user inputs and outputs to my google drive account from the shiny app using the googledrive and shinylogs package. I have now deployed the app through an AWS EC2 instance on ubuntu but in the server logs I recieve this warning store_googledrive: no token for Google Drive API found, using store_null(console = TRUE) as fallback.
I followed this tutorial.
I run this command in the console googledrive::drive_auth() and authenticate my account. I can see a .secrets folder is created outside the app folder (ui.R. global.R, and server.R) with my token but still in the home folder.
/home_folder/.secrets
/home_folder/app/ui.R
/home_folder/app/global.R
/home_folder/app/server.R
In my global.R file I have the options set up as
options(
gargle_oauth_cache = ".secrets",
gargle_oauth_email = TRUE,
gargle_oob_default = TRUE
)
In my server.R file I have the track_usage() set up as
track_usage(storage_mode = store_googledrive(path = "url to google drive folder"))
Now on the EC2 instance, the app deploys just fine but does not push any user inputs or outputs to the folder and I recieve the above warning in my logs. On the server I have tried file paths like so with no luck.
1)
/ubuntu/.secrets
/ubuntu/app/ui.R
/ubuntu/app/global.R
/ubuntu/app/server.R
2)
/ubuntu/app/.secrets
/ubuntu/app/ui.R
/ubuntu/app/global.R
/ubuntu/app/server.R
Is my file path incorrect for informing my app where to look for the token in the .secrets file? Do I need to authenticate in a different way? Do I need to specify in my track_usage() the token?
I am trying to connect to sharepoint to load excel files within an unattended R script using the package Microsoft365R
I have created the app within Azure, and assigned it permissions. I have been able to successfully connect to a users onedrive and list the files within it.
Code that works
library(AzureGraph)
library(Microsoft365R)
tenant <- "your-tenant-here"
# the application/client ID of the app registration you created in AAD
# - not to be confused with the 'object ID' or 'service principal ID'
app <- "your-app-id-here"
# retrieve the client secret (password) from an environment variable
pwd <- Sys.getenv("EXAMPLE_MS365R_CLIENT_SECRET")
# retrieve the user whose OneDrive we want to access
# - this should be their 'userPrincipalName', which is of the form 'name#tenant.com'
# - note this may be different to their regular email address
user <- Sys.getenv("EXAMPLE_MS365R_TARGET_USER")
# create a Microsoft Graph login
gr <- create_graph_login(tenant, app, password=pwd, auth_type="client_credentials")
drv <- gr$get_user(user)$get_drive()
drv$list_files()
When running the below code, i get the error
# the application/client ID of the app registration to use
app <- "your-app-id-here"
# get the service account username and password
user <- Sys.getenv("EXAMPLE_MS365R_SERVICE_USER")
pwd <- Sys.getenv("EXAMPLE_MS365R_SERVICE_PASSWORD")
# SharePoint site and path to folder
sitename <- Sys.getenv("EXAMPLE_MS365R_SPO_SITENAME")
folderpath <- Sys.getenv("EXAMPLE_MS365R_SPO_FOLDERPATH")
# use the 'resource_owner' auth type for a non-interactive login
site <- get_sharepoint_site(sitename, tenant=tenant, app=app, username=user, password=pwd,
auth_type="resource_owner")
Output:
Error in process_aad_response(res) :
Unauthorized (HTTP 401). Failed to obtain Azure Active Directory token. Message:
AADSTS7000218: The request body must contain the following parameter: 'client_assertion' or 'client_secret'.
The function doesn't appear to have an argument for client secret.
I have followed the vignettes within the package, but i must be missing something. Is anyone able to provide assistance?
Resource
https://cran.r-project.org/web/packages/Microsoft365R/vignettes/scripted.html
You will get the AADSTS7000218 error when you try to get an authentication token for an application with the "Web" platform configuration without the client_secret parameter.
How about when the platform configuration of the application to Mobile and desktop applications ?
To change the platform configuration:
Open the application from App registrations page on Azure AD, and open Manage - Authentication page.
Delete the existing Web platform configuration.
Open Add a platform and select Mobile and desktop applications.
I'm trying to use GitHub Actions to query the GitHub API using an OAuth 2.0 token through an R script. It works fine on my local machine when I run the code, where a browser window pops up indicating "Waiting for authentication in browser..." that I can manually close. When run through GitHub Actions, the workflow hangs at the "Waiting for authentication in browser..." since it's on a remote machine.
I'm using a custom R script with the httr library. The API credentials are stored as secrets for the repository I'm trying to query.
library(httr)
gh_key <- Sys.getenv('GH_KEY')
gh_secret <- Sys.getenv('GH_SECRET')
# setup app credentials
myapp <- oauth_app(appname = "data-in-r",
key = gh_key,
secret = gh_secret)
# get oauth credentials
github_token <- oauth2.0_token(oauth_endpoints('github'), app = myapp, cache = F)
# use api
gtoken <- config(token = github_token)
# get list of remote files in data folder
req <- GET("https://api.github.com/repos/tbep-tech/piney-point/contents/data", gtoken)
When the script is run through GitHub Actions, it looks like as below, where I had to manually cancel the workflow since it was hung at the browser step.
Is there a workaround for skipping the browser step so it will run on GitHub Actions? The offending repo is here.
You cannot use the 3-legged variant of OAuth2.0 (aka "web-application flow") in a headless environment like Github Actions.
If you want to use OAuth (I list other possibilities below), then you need to utilize what gitlab calls the "device-flow". See github documentation.
In this flow, there is no redirect to a given URL, so the app does not need a browser window. Instead it displays a code to the user. The user must the enter that code on a fixed URL (https://github.com/login/device). As soon as this is done, the app can request the authentication token. (So the app must keep polling until the user has entered the code).
Unfortunately, httr does not have nice wrapper functions for this variant, so you have to do the calls yourself. It can work like this:
library(httr)
app_id <- "*redacted*"
r <- POST("https://github.com/login/device/code",
body = list(
client_id = app_id,
scope = "user repo delete_repo" #Scope must be given! https://docs.github.com/en/developers/apps/building-oauth-apps/scopes-for-oauth-apps
))
device_code <- content(r)$device_code
print(paste0("Enter the code ", content(r)$user_code, " at ", content(r)$verification_uri))
## NOTE: In reality this request has to run in a loop, until the user has entered the code und the request succeeds.
## For the demo we can execute it manually after the code has been entered.
r <- POST("https://github.com/login/oauth/access_token",
body = list(
client_id = app_id,
device_code = device_code,
grant_type = "urn:ietf:params:oauth:grant-type:device_code"
))
token <- content(r)$access_token
## create and delete a private testrepository to check if everything worked
r <-
POST("https://api.github.com/user/repos",
add_headers(Authorization = paste("token", token)),
body = list(name = "testrepo",
private = TRUE,
auto_init = FALSE),
encode = "json")
r <- DELETE(paste0("https://api.github.com/repos/", content(r)$full_name),
add_headers(Authorization = paste("token", token)))
I have seen that there is httr2, and that it offers convenience functions for this flow. I have however never used it and do not know if it already works reliable. See here.
Since this flow still requires user interaction, you may be better of with one of the following variants (I do not know if they fit your use case.):
Basic Auth:
You can define what github calls a "personal access token" beforehand. With this token you can authenticate without further interaction. Creation is described here. In R you can use it most easily together with httr::authenticate.
GITHUB_TOKEN:
Github automatically creates special secrets specifically Github Actions. These can be used to execute actions in the containing repository. For more info see here.
Following the instructions on this SO question I was able to create a repo for an organization that I own using the GitHub REST API and the R packages httr and RCurl. Now, I'm having issues deleting the repo using the API.
I created an OAuth app on GitHub and gave the app access to my organization. Then I ran the code below to create a token with the delete_repo scope.
library(httr)
library(RCurl)
# 1. Find OAuth settings for github:
# http://developer.github.com/v3/oauth/
oauth_endpoints("github")
# 2. Register an application at https://github.com/settings/applications
# Insert your values below - if secret is omitted, it will look it up in
# the GITHUB_CONSUMER_SECRET environmental variable.
#
# Use http://localhost:1410 as the callback url
myapp <- oauth_app("TestApp", "app-number","secret-number")
scope <- 'delete_repo'
# 3. Get OAuth credentials
github_token <- oauth2.0_token(oauth_endpoints("github"),scope=scope, myapp)
Per the GitHub API v3 developer guide, the URL to create a new repo in an organization is
https://api.github.com/orgs/:org/repos
Using this URL I was able to create a private repo for my organization named 'Reliability' by running:
#Push repository to Github
url_c = "https://api.github.com/orgs/Reliability/repos"
data = list("name"= "newRepo", "private" = "true")
POST(url = url_c, body = data, config(token = github_token))
To delete a repo, the developer guide states that the URL should be of the form
DELETE repos/:owner/:repo
For an organizational repo, I interpret that this URL should be
https://api.github.com/orgs/:org/repos/:owner/:repo
However, when I run the following code I get a response of 404 not found.
# Delete repository from Github organization
url_d = "https://api.github.com/orgs/Reliability/repos/Auburngrads/newRepo"
DELETE(url = url_d, config(token = github_token))
What am I missing?
The wording in the GitHub developer guide is misleading with respect to URL that should be used to delete organizational repos using the REST API.
The developer guide states that to delete a repo, the URL should be of the form
DELETE repos/:owner/:repo
However, for an organizational repo, the URL should be of the form
DELETE repos/:org/:repo
I was able to successfully delete a repo from my organization called 'Reliability' by 1) ensuring that my app token had the appropriate delete_repo scope and 2) running the following code
# Delete repository from Github organization
url_d = "https://api.github.com/repos/Reliability/newRepo"
DELETE(url = url_d, config(token = github_token))
I'm running R Studio on an AWS "Ubuntu Server 12.04.2" and accessing R Studio via my browser.
When I try to authenticate google auth API using the package googlesheets with the code:
gs_auth(token = NULL, new_user = FALSE,
key = getOption("googlesheets.client_id"),
secret = getOption("googlesheets.client_secret"),
cache = getOption("googlesheets.httr_oauth_cache"), verbose = TRUE)
The problem here is that it redirects me to browser which is of local machine (windows based).
Even if I authorize it, it redirects to a URL like "http://localhost:1410/?state=blahblah&code=blahblah".
How do I authorize googlesheets in such case?
I have even tried transfering existing httr-oauth token from my windows machine to remove ubuntu server.
The simplest way to create a gs_auth token from a server is to set the httr_oob_default option to true, which will tell httr to use the out of band method for authenticating. You will be given a URL and expected to return an authorization code.
library(googlesheets)
options(httr_oob_default=TRUE)
gs_auth(new_user = TRUE)
gs_ls()
One thing httr does when you set the httr_oob_default option is to redefine the URI to urn:ietf:wg:oauth:2.0:oob as seen in the code for oauth-init.
Alternatively, you can create a .httr-oauth token manually using httr commands. Use the out of band authentication mode by setting use_oob=TRUE in the oauth2.0_token command.
library(googlesheets)
library(httr)
file.remove('.httr-oauth')
oauth2.0_token(
endpoint = oauth_endpoints("google"),
app = oauth_app(
"google",
key = getOption("googlesheets.client_id"),
secret = getOption("googlesheets.client_secret")
),
scope = c(
"https://spreadsheets.google.com/feeds",
"https://www.googleapis.com/auth/drive"),
use_oob = TRUE,
cache = TRUE
)
gs_ls()
Another, less elegant, solution is to create the .httr-oauth token on your desktop and then copying the file to a server.
After lot of head banging, I found that a package "httpuv" which supports HTTP handling and WebSocket requests from R was creating the problem. It was forcing R to open web browser.
Once I uninstalled this package, "googlesheets" gave me a link which I could paste in browser separately and then paste the auth code back in R server.