I love RGoogleDocs and use it a lot. However, I don't like entering my password all the time. Obviously I could just type the password into the R script and would never have to enter it again. But thats not viable since it means that my password would be left unencrypted on my harddrive. Furthermore I share my scripts with colleagues.
To get around the problem I came up with this.
if(exists("ps")){
print("got password, keep going")
} else {
ps <-readline(prompt="get the password in ")
}
options(RCurlOptions = list(
capath = system.file("CurlSSL", "cacert.pem",
package = "RCurl"), ssl.verifypeer = FALSE)
)
sheets.con = getGoogleDocsConnection(
getGoogleAuth("notreal#gmail.com", ps, service ="wise"))
#WARNING: this would prevent curl from detecting a 'man in the middle' attack
ts2=getWorksheets("hpv type",sheets.con)
I love using RStudio. I feel uncomfortable that it is displaying my password for any colleague in my office at the time to see. I used a fake password but look at the image. . Furthermore, if I saved a workspace my password would be saved with it and I am afraid that I would be giving it to someone else if, a few months later, when I had long forgotten about what was in it, I sent my .RData file to a colleague.
I read something general about passwords in R in an earlier post. It did not give me enough information to be able to conceal my password when using RGoogleDocs.
My approach is to set the login-name & password in the R options list
within the R startup file .Rprofile. Then my code gets the value
with getOption() and then the value is never visible or stored
in a top-level variable in globalenv(). (It could be save if
one does post-mortem debugging via dump.frames).
It is vital that the .Rprofile cannot be read by anybody other than you.
So
options(GoogleDocsPassword = c(login = 'password'))
in the .Rprofile and then
auth = getGoogleAuth()
just works as the default value for the first parameter is to look for the GoogleDocsPassword option.
D.
I had the same problem, and no real solution. The workaround I use is, I create a google account just for this purpose, with a password that I do not care about. I then share the documents that I want R to access with that account.
But if someone has an answer to the initial question I am interested as well.
Seems like uou could store the password in your options and the instead of "ps" directly use "getOption". LIkely there are better solutions though.
You could store the password in a file on you computer, encoded and all and call it with somthing like
getPassword <- function(file = location of password file){unencode(readLines(file))}
set this in your .Rprofile and use in the code
getPassword().
This doesn't store your password in any R files and you can build in checks in the file.
If you really don't want to store it anywhere, then one solution to this is not to use a variable for the password, maybe even for the google account address! Building on the linked answer, why not try
library(tcltk)
library(RGoogleDocs)
getHiddenText <- function(label = "Enter text:", symbol = "*", defaultText = ""){
wnd <- tktoplevel()
entryVar <- tclVar(defaultText)
tkgrid(tklabel(wnd, text = label))
#Entry box
tkgrid(entryBox <- tkentry(wnd, textvariable = entryVar, show = symbol))
#Hitting return will also submit text
tkbind(entryBox, "<Return>", function() tkdestroy(wnd))
#OK button
tkgrid(tkbutton(wnd, text="OK", command=function() tkdestroy(wnd)))
#Wait for user to submit
tkwait.window(wnd)
return(tclvalue(entryVar))
}
repeat {
con <- try(getGoogleDocsConnection(getGoogleAuth(
getHiddenText(
label = "Enter google account:",
symbol = "", # or set to "*" to obscure email entry
defaultText = "#gmail.com"), # a little timesaver
getHiddenText(
label = "Enter password:",
symbol = "*",
defaultText = ""),
service = "wise")))
if (inherits(con, "try-error")) {
userResponse <- tkmessageBox(
title = "Error",
message = "Couldn't connect to Google Docs. Try again?",
icon = "error", type = "yesno")
if (tclvalue(userResponse) == "no") {
stop("Unable to connect to Google Docs, user cancelled.")
}
} else { # connection successfully authenticated
break() # so escape the repeat loop
}
}
For things like this I share the google doc with a made up email address, create a google account and then use it for sharing and authorization. Thus, seperating my personal login details from what's necessasry for the script to run.
what about 2 step authentication with application specific password ?
you can use the application specific password without revealing your real one.
and you can revoke it if you want !
Related
I can retrieve EPPO DB info from GET requests.
I am looking for help to retrieve the info from POST requests.
Example code and other info in the linked Rmarkdown HTMP output
As suggested, I have gone trough the https://httr.r-lib.org/ site.
Interesting. I followed the links to https://httr.r-lib.org/articles/api-packages.html and then to https://cdn.zapier.com/storage/learn_ebooks/e06a35cfcf092ec6dd22670383d9fd12.pdf.
I suppose that the arguments for the POST() function should be (more or less) as follows, but yet the response is always 404
url = "https://data.eppo.int/api/rest/1.0/"
config = list(authtoken=my_authtoken)
body = list(intext = "Fraxinus anomala|Tobacco ringspot virus|invalide name|Sequoiadendron giganteum")
encode = "json"
#handle = ???
Created on 2021-04-26 by the reprex package (v0.3.0)
How do I find the missing pieces?
It is a little bit tricky:
You need to use correct url with name of the service from https://data.eppo.int/documentation/rest, e.g. to use Search preferred names from EPPOCodes list:
url = "https://data.eppo.int/api/rest/1.0/tools/codes2prefnames"
Authorization should be passed to body:
body = list(authtoken = "yourtoken", intext = "BEMITA")
So, if you want to check names for two eppocodes: XYLEFA and BEMITA the code should look like:
httr::POST(
url = "https://data.eppo.int/api/rest/1.0/tools/codes2prefnames",
body = list(authtoken = "yourtoken", intext = "BEMITA|XYLEFA")
)
Nonetheless, I would also recommend you to just use the pestr package. However, to find eppocodes it uses SQLite database under the hood instead of EPPO REST API. Since the db is not big itself (circa 40MB) this shouldn't be an issue.
I found the easy way following a suggestion in the DataCamp course:
"To find an R client for an API service search 'CRAN '"
I found the 'pestr' package that gives great access to EPPO database.
I still do not know how to use the POST() function myself. Any hint on that side is warmly welcome.
Here is a solution to loop over names to get EPPO-codes. Whit minor adjustments this also works for distribution and other information in the EPPO DB
# vector of species names
host_name <- c("Fraxinus anomala", "Tobacco ringspot virus", "invalide name",
"Sequoiadendron giganteum")
EPPO_key <- "enter personal key"
# EPPO url
path_eppo_code <- "https://data.eppo.int/api/rest/1.0/tools/names2codes"
# epty list
my_list <- list()
for (i in host_name) {
# POST request on eppo database
response <- httr::POST(path_eppo_code, body=list(authtoken=EPPO_key,
intext=i))
# get EPPO code
pest_eppo_code <- strsplit(httr::content(response)[[1]], ";")[[1]][2]
# add to list
my_list[[i]] <- pest_eppo_code
}
# list to data frame
data <- plyr::ldply(my_list)
I am currently creating a REST API with restrserve.
While trying to add a logger to my application I ran into this problem.
By setting the log level as: application$logger$set_log_level("error"),
my console output on error has a JSON structure like:
{
"timestamp":"2020-09-22 18:25:37.425642",
"level":"ERROR",
"name":"Application",
"pid":38662,
"msg":"",
"context":{
"request_id":"alphanumeric string",
"message":{ here is the description of the error }
}
}
I set up a printer function following the instructions on the site, but among the available fields (timestamp, level, logger_name, pid, message), "context" is not present,so basically, by just printing the message my logs are empty files.
This is my printer function:
application$logger$set_printer(FUN = function(timestamp, level, logger_name, pid, message, ...){
write(message, file = paste("/Log/Monitor_", pid,
timestamp, ".txt", sep = ""))
# attempting to print "context" raises an error!
})
Is there a way to print the "context" field present in the console output to a file?
Thanks in advance, any suggestions would be very helpful
It is always a good idea to take a look to the source. There is extra ... argument where context is passed.
Generally it is advisable to use lgr package for user-level logging.
Im attempting to setup some R code to create a new work item task in Azure Devops. Im okay with a mostly empty work item to start with if thats okay to do (my example code is only trying to create a work item with a title).
I receive a 203 response but the work item doesn't appear in Devops.
Ive been following this documentation from Microsoft, I suspect that I might be formatting the body incorrectly.
https://learn.microsoft.com/en-us/rest/api/azure/devops/wit/work%20items/create?view=azure-devops-rest-5.1
Ive tried updating different fields and formatting the body differently with no success. I have attempted to create either a bug or feature work item but both return the same 203 response.
To validate that my token is working I can GET work item data by ID but the POST continues to return a 203.
require(httr)
require(jsonlite)
url <- 'https://dev.azure.com/{organization}/{project}/_apis/wit/workitems/$bug?api-version=5.1'
headers = c(
'Authorization' = sprintf('basic %s',token),
'Content-Type' = 'application/json-patch+json',
'Host' = 'dev.azure.com'
)
data <- toJSON(list('body'= list("op"= "add",
"path"= "/fields/System.AreaPath",
"value"= "Sample task")), auto_unbox = TRUE, pretty = TRUE)
res <- httr::POST(url,
httr::add_headers(.headers=headers),
httr::verbose(),
body = data)
Im expecting a 200 response (similar to the example in the link above) and a work item task in Azure DevOps Services when I navigate to the website.
Im not the best with R so please be detailed. Thank you in advanced!
The POST continues to return a 203.
The HTTP response code 203 means Non-Authoritative Information, it should caused by your token format is converted incorrectly.
If you wish to provide the personal access token through an HTTP
header, you must first convert it to a Base64 string.
Refer to this doc described, if you want to use VSTS rest api, you must convert your token to a Base64 string. But in your script, you did not have this script to achieve this convert.
So, please try with the follow script to convert the token to make the key conformant with the requirements(load the base64enc package first):
require(base64enc)
key <- token
keys <- charToRaw(paste0(key,":token"))
auth <- paste0("Basic ",base64encode(keys))
Hope this help you get 200 response code
I know this question is fairly old, but I cannot seem to find a good solution posted yet. So, I will add my solution in case others find themselves in this situation. Note, this did take some reading through other SO posts and trial-and-error.
Mengdi is correct that you do need to convert your token to a Base64 string.
Additionally, Daniel from this SO question pointed out that:
In my experience with doing this via other similar mechanisms, you have to include a leading colon on the PAT, before base64 encoding.
Mengdi came up big in another SO solution
Please try with adding [{ }] outside your request body.
From there, I just made slight modifications to your headers and data objects. Removed 'body' from your json, and made use of paste to add square brackets as well. I found that the Rcurl package made base64 encoding a breeze. Then I was able to successfully create a blank ticket (just titled) using the API! Hope this helps someone!
library(httr)
library(jsonlite)
library(RCurl)
#user and PAT for api
userid <- ''
token= 'whateveryourtokenis'
url <- 'https://dev.azure.com/{organization}/{project}/_apis/wit/workitems/$bug?api-version=5.1'
#create a combined user/pat
#user id can actually be a blank string
#RCurl's base64 seemed to work well
c_id <- RCurl::base64(txt = paste0(userid,
":",
devops_pat
),
mode = "character"
)
#headers for api call
headers <- c(
"Authorization" = paste("Basic",
c_id,
sep = " "
),
'Content-Type' = 'application/json-patch+json',
'Host' = 'dev.azure.com'
)
#body
data <- paste0("[",
toJSON(list( "op"= "add",
"path"= "/fields/System.Title",
"value"= "API test - please ignore"),
auto_unbox = TRUE,
pretty = TRUE
),
"]"
)
#make the call
res <- httr::POST(url,
httr::add_headers(.headers=headers),
httr::verbose(),
body = data
)
#check status
status <- res$status_code
#check content of response
check <- content(res)
I do that:
library(encryptr)
genkeys()
And I created the password: 0)]30l^8
password<-"0)]30l^8"
data(gp)
write.csv(gp, "gp.csv")
encrypt_file("gp.csv")
My problem is: How do I automatically enter the password on the decrypt_file("gp.csv.encryptr.bin", file_name = "gp2.csv")
I need this to decrypt many files in a short time.
Many thanks for the question. Saving a password in a script is not recommended as that defeats the purpose of encrypting a file in most circumstances. You can work around this intentional feature although it is not recommended.
password<-"0)]30l^8"
.crypt = readRDS("gp.csv.encryptr.bin") # in file
zz = file("gp2.csv", "wb") # out file
openssl::decrypt_envelope(.crypt$data, .crypt$iv, .crypt$session, key = "id_rsa", password = password) %>%
writeBin(zz)
close(zz)
For example:
if(url.exists("http://www.google.com")) {
# Two ways to submit a query to google. Searching for RCurl
getURL("http://www.google.com/search?hl=en&lr=&ie=ISO-8859-1&q=RCurl&btnG=Search")
# Here we let getForm do the hard work of combining the names and values.
getForm("http://www.google.com/search", hl="en", lr="",ie="ISO-8859-1", q="RCurl", btnG="Search")
# And here if we already have the parameters as a list/vector.
getForm("http://www.google.com/search", .params = c(hl="en", lr="", ie="ISO-8859-1", q="RCurl", btnG="Search"))
}
This is an example from RCurl package manual. However, it does not work:
> url.exists("http://www.google.com")
[1] FALSE
I found there is an answer to this here Rcurl: url.exists returns false when url does exists. It said this is because of the default user agent is not useful. But I do not understand what user agent is and how to use it.
Also, this error happened when I worked in my company. I tried the same code at home, and it worked find. So I am guessing this is because of proxy. Or there is some other reasons that I did not realize.
I need to use RCurl to search my queries from Google, and then extract the information such as title and descriptions from the website. In this case, how to use user agent? Or, does the package httr can do this?
guys. Thanks a lot for help. I think I just figured out how to do it. The important thing is proxy. If I use:
> opts <- list(
proxy = "http://*******",
proxyusername = "*****",
proxypassword = "*****",
proxyport = 8080
)
> url.exists("http://www.google.com",.opts = opts)
[1] TRUE
Then all done! You can find your proxy under System-->proxy if you use win 10. At the same time:
> site <- getForm("http://www.google.com.au", hl="en",
lr="", q="r-project", btnG="Search",.opts = opts)
> htmlTreeParse(site)
$file
[1] "<buffer>"
.........
In getForm, opts needs to be put in as well. There are two posters here (RCurl default proxy settings and Proxy setting for R) answering the same question. I have not tried how to extract information from here.