Related
I have been banging my head over this the whole day. I am trying to access StockTwits API (https://api.stocktwits.com/developers) from an R session. I have earlier accessed the twitter API (via rtweet) without hassles.
I have created an app and got the client id and key (the below are just examples).
app_name = "some.name";
consumer_key = "my_client_id";
consumer_secret = "my_client_key";
uri = "http://iimb.ac.in" # this is my institute's homepage. It doesn't allow locahost OR 127.0.0.1
scope = "read,watch_lists,publish_messages,publish_watch_lists,direct_messages,follow_users,follow_stocks";
base_url = "https://api.stocktwits.com/api/2/oauth"; # see https://api.stocktwits.com/developers/docs/api
The procedure is to create an oauth2.0 app and endpoint. Then call oauth2.0_token.
oa = httr::oauth_app(app_name, key = consumer_key, secret = consumer_secret, redirect_uri = uri);
oe = httr::oauth_endpoint("stocktwits", "authorize", "token", base_url = base_url);
mytoken = httr::oauth2.0_token(oe, oa, user_params = list(resource = base_url), use_oob = F); # use_oob = T doesn't work.
After firing the above, it takes me to the browser for sign-in. I sign-in and it asks me to connect. After that, I am taken back to my URI plus a code, i.e. https://www.iimb.ac.in/?code=295ea3114c3d8680a0ed295d52313d7092dd90ae&state=j9jXzEqri1
Is the code my access token or something else? The oauth2.0_token() call keeps waiting for the code since the callback is not localhost. I didn't seem to get a hang of that.
I then try to access the API using the above code as access token but I am thrown "invalid access token" error. The format is described in https://api.stocktwits.com/developers/docs/api#search-index-docs
Can someone tell me what I have missed? If required I can share my app_name, consumer_key and consumer_secret for replication.
Good afternoon!
I'm a self-taught R developer so I might not be using the correct jargon, but please bear with me.
I have written a script to clean and process data which has been manually exported from Qualtrics, however I would like to further automate my data process workflow by tapping directly into the Qualtrics API. I am attempting to use the qualtRics package to handle the API call, but when I execute the function I get a return of port 443: Connection refused. I have yet to find a workable solution for this. I have followed the standard workflow Code below:
#Save API Key and Base URL as environment variables
qualtrics_api_credentials(
api_key = "<API_KEY>",
base_url = "<BASE_URL>",
overwrite = TRUE,
install = TRUE
)
#Refresh environment
readRenviron("~/.Renviron")
#Pull list of surveys
Surveys <- all_surveys()
#Fetch survey data from Survey based on Survey ID
SurveyData <- fetch_survey("Survey_ID",
last_response = NULL,
start_date = "2019-11-18",
end_date = NULL,
unanswer_recode = NULL,
limit = NULL,
include_questions = NULL,
save_dir = NULL,
force_request = FALSE,
verbose = TRUE,
label = TRUE,
convert = TRUE,
import_id = FALSE,
local_time = FALSE)
For both the all_surveys() and fetch_survey() functions, if I use https:// I get a return of:
"Error in curl::curl_fetch_memory(url, handle = handle) :
Failed to connect to port 443: Connection refused"
and if I use http:// a return of:
"Error in curl::curl_fetch_memory(url, handle = handle) :
Failed to connect to port 80: Connection refused"
Is there a workaround or a different approach I can take to pull Qualtrics survey data into a data frame via the API?
Best regards
I have a scheduled an R script running from a windows machine.
After it finishes, I wish this script to automatically send an email with some log file attached.
Using shell() with some other scripts may be possible, but I was wondering if there is a better solution within R.
Thanks.
sendmailR works for me on Windows 7. I referenced http://cran.es.r-project.org/web/packages/sendmailR/sendmailR.pdf
smtpServer= info for Outlook 2010 is in File -> Account Settings -> Account Settings -> double click your account -> text in "Server" box
library(sendmailR)
#set working directory
setwd("C:/workingdirectorypath")
#####send plain email
from <- "you#account.com"
to <- "recipient#account.com"
subject <- "Email Subject"
body <- "Email body."
mailControl=list(smtpServer="serverinfo")
sendmail(from=from,to=to,subject=subject,msg=body,control=mailControl)
#####send same email with attachment
#needs full path if not in working directory
attachmentPath <- "subfolder/log.txt"
#same as attachmentPath if using working directory
attachmentName <- "log.txt"
#key part for attachments, put the body and the mime_part in a list for msg
attachmentObject <- mime_part(x=attachmentPath,name=attachmentName)
bodyWithAttachment <- list(body,attachmentObject)
sendmail(from=from,to=to,subject=subject,msg=bodyWithAttachment,control=mailControl)
In addition, multiple files can be sent by adding another mime_part to the msg list as follows (I also condensed it):
attachmentObject <- mime_part(x="subfolder/log.txt",name="log.txt")
attachmentObject2 <- mime_part(x="subfolder/log2.txt",name="log2.txt")
bodyWithAttachment <- list(body,attachmentObject,attachmentObject2)
Use mailR - it works with authentication, attachments, it automatically send txt message along with html and more.
mailR requires rJava which can be a bit of a pain sometimes. On windows I haven't had any problems. On ubuntu this solved the one issue I've had:
sudo apt-get install openjdk-jdk
in R
install.packages("devtools", dep = T)
library(devtools)
install_github("rpremraj/mailR")
(if you have trouble with rJava - try sudo R CMD javareconf in terminal)
mailR is easy to work with and well documented on the github page.
Example from the documentaion
library(mailR)
send.mail(from = "sender#gmail.com",
to = c("recipient1#gmail.com", "recipient2#gmail.com"),
subject = "Subject of the email",
body = "Body of the email",
smtp = list(host.name = "smtp.gmail.com", port = 465, user.name = "gmail_username", passwd = "password", ssl = TRUE),
authenticate = TRUE,
send = TRUE,
attach.files = c("./download.log", "upload.log", "https://dl.dropboxusercontent.com/u/5031586/How%20to%20use%20the%20Public%20folder.rtf"),
file.names = c("Download log.log", "Upload log.log", "DropBox File.rtf"), # optional parameter
file.descriptions = c("Description for download log", "Description for upload log", "DropBox File"), # optional parameter
debug = TRUE)
Note: your smtp server might find excessive use suspicious. This is the case with e.g. gmail. So after sending a few mails you probably have to log in to the gmail account and see if the account has been temporarily disabled. Also note that if you use a gmail account with two-factor authentication you need to use an application specific password.
Would you settle for a twitter message? You could use Rcurl to post an update to twitter, which can then be forwarded to your cell phone as a text, or to your email via the notification settings.
See here: http://www.sakana.fr/blog/2007/03/18/scripting-twitter-with-curl/
Have you looked into the sendmailR package yet? It allows SMTP to submit a message and you could probably edit the function to allow an attachment. Then again, if its only one log file it might just be worth it to use shell() as you mentioned.
For Windows one might parse together a VB-Script (see e.g. http://www.paulsadowski.com/wsh/cdo.htm ) and then call it via shell.
This might look like this:
SendMail <- function(from="me#my-server.de",to="me#my-server.de",text="Hallo",subject="Sag Hallo",smtp="smtp.my.server.de",user="me.myself.and.i",pw="123"){
require(stringr)
part1 <- "Const cdoSendUsingPickup = 1 'Send message using the local SMTP service pickup directory.
Const cdoSendUsingPort = 2 'Send the message using the network (SMTP over the network).
Const cdoAnonymous = 0 'Do not authenticate
Const cdoBasic = 1 'basic (clear-text) authentication
Const cdoNTLM = 2 'NTLM "
part2 <- paste(paste("Set objMessage = CreateObject(",'"',"CDO.Message",'"',")" ,sep=""),
paste("objMessage.Subject = ",'"',subject,'"',sep=""),
paste("objMessage.From = ",'"',from,'"',sep=""),
paste("objMessage.To = ",'"',to,'"',sep=""),
paste("objMessage.TextBody = ",'"',text,'"',sep=""),
sep="\n")
part3 <- paste(
"'==This section provides the configuration information for the remote SMTP server.
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/sendusing\") = 2
'Name or IP of Remote SMTP Server
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/smtpserver\") = ",'"',smtp,'"',"
'Type of authentication, NONE, Basic (Base64 encoded), NTLM
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/smtpauthenticate\") = cdoBasic
'Your UserID on the SMTP server
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/sendusername\") = ",'"',user,'"',"
'Your password on the SMTP server
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/sendpassword\") = ",'"',pw,'"', "
'Server port (typically 25)
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/smtpserverport\") = 25
'Use SSL for the connection (False or True)
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/smtpusessl\") = False
'Connection Timeout in seconds (the maximum time CDO will try to establish a connection to the SMTP server)
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/smtpconnectiontimeout\") = 60
objMessage.Configuration.Fields.Update
'==End remote SMTP server configuration section==
objMessage.Send
",sep="")
vbsscript <- paste(part1,part2,part3,sep="\n\n\n")
str_split(vbsscript,"\n")
writeLines(vbsscript, "sendmail.vbs")
shell("sendmail.vbs")
unlink("sendmail.vbs")
}
Just want to remind people who wants self-notifying feature of a service called twilio, they provide free service to send sms to your own cellphone. A walk-through using R is available here https://dreamtolearn.com/ryan/data_analytics_viz/78
An example code is attached, just replace the credentials with the ones of your own.
library(jsonlite)
library(XML)
library(httr)
library(rjson)
library(RCurl)
options(RCurlOptions = list(cainfo = system.file("CurlSSL", "cacert.pem", package = "RCurl")))
authenticate_twilio <- "https://[ACCOUNT SID]:[AUTH TOKEN]#api.twilio.com/2010-04-01/Accounts"
authenticate_response <- getURL(authenticate_twilio)
print(authenticate_response)
postForm("https://[ACCOUNT SID]:[AUTH TOKEN]#api.twilio.com/2010-04-01/Accounts/[ACCOUNT SID]/Messages.XML",.params = c(From = "+1[twilio phone#]", To = "+1[self phone#]",Body = "Hello from twilio"))
Here is a simple code snippet for sending an e-mail by using "mailR" package
library(mailR)
# 1. Sender
sender <- "x#me.com"
# 2. Recipients
recipients <- c("y#you.com", "z#our.com")
# 3. Attached files
list_files_attached_location <- c("mtcars.csv")
list_files_attached_names <- c("mtcars name")
list_files_attached_description <- c("mtcars desc")
# 4. Send email
tryCatch({
send.mail(from = sender,
to = recipients,
subject = "Your subject",
body = "Your mail body",
smtp = list(
host.name = "smtp.gmail.com",
port = 465,
user.name = sender,
passwd = "psw",
ssl = TRUE),
authenticate = TRUE,
send = TRUE,
attach.files = list_files_attached_location,
file.names = list_files_attached_names,
file.descriptions = list_files_attached_description,
debug = TRUE
)
},
error = function(e) {
print(e)
stop()
})
For those who suggested using 'mailR' - this is the way to go, but it is difficult to install of ubuntu. Here is a resource to install 'javaR' on ubuntu which will allow you to install 'mailR'
https://www.r-bloggers.com/2018/02/installing-rjava-on-ubuntu/
Late to this, but you can avoid shelling out to vbscript by using RDCOMClient correctly, as in this example from R-Help.
> sendEmail(ema = "r-help at r-project.org",
name = "R-help mailing list",
subject = "How to send Email from R using the RDCOMClient"
msgBody = "here is the body of the message")
The package RDCOMClient is available at http://www.omegahat.org/RDCOMClient.
"sendEmail" <-
function(ema, name, subject, msgBody, deliverNow = TRUE)
{
require(RDCOMClient)
ema <- paste("SMPT:", ema, sep="") ## prepend protocol to address
## create an e-mail session
session <- COMCreate("Mapi.Session")
session$Logon()
## add a message to the outbox collection of messages
outbox <- session[["Outbox"]]
msg <- outbox[["Messages"]]$Add(subject, msgBody)
## add recipient's name (TODO: addMultiple() or loop, if many recipients)
msg[["Recipients"]]$Add(name, ema)
msg$Send()
if(deliverNow)
msg$DeliverNow()
session$Logoff() ## wrap up
}
I am trying to use the google analytics apis v3 to embed the analytics graph to my website using Certificate file. I got it to work on the local machine but when I deploy the code to my web server it produces this error:
The remote server returned an error: (400) Bad Request.
Does anyone know what is causing this error message and why I can't get the google graph on my production website?
Here is the code that is working on my local machine:
string scope = AnalyticsService.Scopes.AnalyticsReadonly.GetStringValue(),
x509Certificate2File = HttpContext.Current.Server.MapPath("~/App_Data/privatekey.p12"),
x509Certificate2FilePassword = "notasecret",
serviceAccountId = "xxxxxxx#developer.gserviceaccount.com",
profileId = "ga:xxxxxxxx",
startDate = "2013-10-06",
endDate = "2013-11-07";
AuthorizationServerDescription desc = GoogleAuthenticationServer.Description;
X509Certificate2 key = new X509Certificate2(x509Certificate2File, x509Certificate2FilePassword, X509KeyStorageFlags.Exportable);
AssertionFlowClient client = new AssertionFlowClient(desc, key) { ServiceAccountId = serviceAccountId, Scope = scope };
OAuth2Authenticator<AssertionFlowClient> auth = new OAuth2Authenticator<AssertionFlowClient>(client, AssertionFlowClient.GetState);
AnalyticsService gas = new AnalyticsService(new BaseClientService.Initializer() { Authenticator = auth });
DataResource.GaResource.GetRequest request = gas.Data.Ga.Get(profileId, startDate, endDate, "ga:visits");
request.Dimensions = "ga:day";
request.MaxResults = 10;
/* error 400 occur here on the production */
Google.Apis.Analytics.v3.Data.GaData data = request.Execute();
Thanks
There is only one possibility and without looking at the code you're going to have to chase it down; one of the paths are wrong and needs to be changed. This could be in the form of an IP address as well when I say "path" as you are calling something that is not there. It worked fine on your local machine (correct path) and it gave a 400 on the remote one (bad request) as in calling the wrong place.
I have a scheduled an R script running from a windows machine.
After it finishes, I wish this script to automatically send an email with some log file attached.
Using shell() with some other scripts may be possible, but I was wondering if there is a better solution within R.
Thanks.
sendmailR works for me on Windows 7. I referenced http://cran.es.r-project.org/web/packages/sendmailR/sendmailR.pdf
smtpServer= info for Outlook 2010 is in File -> Account Settings -> Account Settings -> double click your account -> text in "Server" box
library(sendmailR)
#set working directory
setwd("C:/workingdirectorypath")
#####send plain email
from <- "you#account.com"
to <- "recipient#account.com"
subject <- "Email Subject"
body <- "Email body."
mailControl=list(smtpServer="serverinfo")
sendmail(from=from,to=to,subject=subject,msg=body,control=mailControl)
#####send same email with attachment
#needs full path if not in working directory
attachmentPath <- "subfolder/log.txt"
#same as attachmentPath if using working directory
attachmentName <- "log.txt"
#key part for attachments, put the body and the mime_part in a list for msg
attachmentObject <- mime_part(x=attachmentPath,name=attachmentName)
bodyWithAttachment <- list(body,attachmentObject)
sendmail(from=from,to=to,subject=subject,msg=bodyWithAttachment,control=mailControl)
In addition, multiple files can be sent by adding another mime_part to the msg list as follows (I also condensed it):
attachmentObject <- mime_part(x="subfolder/log.txt",name="log.txt")
attachmentObject2 <- mime_part(x="subfolder/log2.txt",name="log2.txt")
bodyWithAttachment <- list(body,attachmentObject,attachmentObject2)
Use mailR - it works with authentication, attachments, it automatically send txt message along with html and more.
mailR requires rJava which can be a bit of a pain sometimes. On windows I haven't had any problems. On ubuntu this solved the one issue I've had:
sudo apt-get install openjdk-jdk
in R
install.packages("devtools", dep = T)
library(devtools)
install_github("rpremraj/mailR")
(if you have trouble with rJava - try sudo R CMD javareconf in terminal)
mailR is easy to work with and well documented on the github page.
Example from the documentaion
library(mailR)
send.mail(from = "sender#gmail.com",
to = c("recipient1#gmail.com", "recipient2#gmail.com"),
subject = "Subject of the email",
body = "Body of the email",
smtp = list(host.name = "smtp.gmail.com", port = 465, user.name = "gmail_username", passwd = "password", ssl = TRUE),
authenticate = TRUE,
send = TRUE,
attach.files = c("./download.log", "upload.log", "https://dl.dropboxusercontent.com/u/5031586/How%20to%20use%20the%20Public%20folder.rtf"),
file.names = c("Download log.log", "Upload log.log", "DropBox File.rtf"), # optional parameter
file.descriptions = c("Description for download log", "Description for upload log", "DropBox File"), # optional parameter
debug = TRUE)
Note: your smtp server might find excessive use suspicious. This is the case with e.g. gmail. So after sending a few mails you probably have to log in to the gmail account and see if the account has been temporarily disabled. Also note that if you use a gmail account with two-factor authentication you need to use an application specific password.
Would you settle for a twitter message? You could use Rcurl to post an update to twitter, which can then be forwarded to your cell phone as a text, or to your email via the notification settings.
See here: http://www.sakana.fr/blog/2007/03/18/scripting-twitter-with-curl/
Have you looked into the sendmailR package yet? It allows SMTP to submit a message and you could probably edit the function to allow an attachment. Then again, if its only one log file it might just be worth it to use shell() as you mentioned.
For Windows one might parse together a VB-Script (see e.g. http://www.paulsadowski.com/wsh/cdo.htm ) and then call it via shell.
This might look like this:
SendMail <- function(from="me#my-server.de",to="me#my-server.de",text="Hallo",subject="Sag Hallo",smtp="smtp.my.server.de",user="me.myself.and.i",pw="123"){
require(stringr)
part1 <- "Const cdoSendUsingPickup = 1 'Send message using the local SMTP service pickup directory.
Const cdoSendUsingPort = 2 'Send the message using the network (SMTP over the network).
Const cdoAnonymous = 0 'Do not authenticate
Const cdoBasic = 1 'basic (clear-text) authentication
Const cdoNTLM = 2 'NTLM "
part2 <- paste(paste("Set objMessage = CreateObject(",'"',"CDO.Message",'"',")" ,sep=""),
paste("objMessage.Subject = ",'"',subject,'"',sep=""),
paste("objMessage.From = ",'"',from,'"',sep=""),
paste("objMessage.To = ",'"',to,'"',sep=""),
paste("objMessage.TextBody = ",'"',text,'"',sep=""),
sep="\n")
part3 <- paste(
"'==This section provides the configuration information for the remote SMTP server.
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/sendusing\") = 2
'Name or IP of Remote SMTP Server
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/smtpserver\") = ",'"',smtp,'"',"
'Type of authentication, NONE, Basic (Base64 encoded), NTLM
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/smtpauthenticate\") = cdoBasic
'Your UserID on the SMTP server
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/sendusername\") = ",'"',user,'"',"
'Your password on the SMTP server
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/sendpassword\") = ",'"',pw,'"', "
'Server port (typically 25)
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/smtpserverport\") = 25
'Use SSL for the connection (False or True)
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/smtpusessl\") = False
'Connection Timeout in seconds (the maximum time CDO will try to establish a connection to the SMTP server)
objMessage.Configuration.Fields.Item _
(\"http://schemas.microsoft.com/cdo/configuration/smtpconnectiontimeout\") = 60
objMessage.Configuration.Fields.Update
'==End remote SMTP server configuration section==
objMessage.Send
",sep="")
vbsscript <- paste(part1,part2,part3,sep="\n\n\n")
str_split(vbsscript,"\n")
writeLines(vbsscript, "sendmail.vbs")
shell("sendmail.vbs")
unlink("sendmail.vbs")
}
Just want to remind people who wants self-notifying feature of a service called twilio, they provide free service to send sms to your own cellphone. A walk-through using R is available here https://dreamtolearn.com/ryan/data_analytics_viz/78
An example code is attached, just replace the credentials with the ones of your own.
library(jsonlite)
library(XML)
library(httr)
library(rjson)
library(RCurl)
options(RCurlOptions = list(cainfo = system.file("CurlSSL", "cacert.pem", package = "RCurl")))
authenticate_twilio <- "https://[ACCOUNT SID]:[AUTH TOKEN]#api.twilio.com/2010-04-01/Accounts"
authenticate_response <- getURL(authenticate_twilio)
print(authenticate_response)
postForm("https://[ACCOUNT SID]:[AUTH TOKEN]#api.twilio.com/2010-04-01/Accounts/[ACCOUNT SID]/Messages.XML",.params = c(From = "+1[twilio phone#]", To = "+1[self phone#]",Body = "Hello from twilio"))
Here is a simple code snippet for sending an e-mail by using "mailR" package
library(mailR)
# 1. Sender
sender <- "x#me.com"
# 2. Recipients
recipients <- c("y#you.com", "z#our.com")
# 3. Attached files
list_files_attached_location <- c("mtcars.csv")
list_files_attached_names <- c("mtcars name")
list_files_attached_description <- c("mtcars desc")
# 4. Send email
tryCatch({
send.mail(from = sender,
to = recipients,
subject = "Your subject",
body = "Your mail body",
smtp = list(
host.name = "smtp.gmail.com",
port = 465,
user.name = sender,
passwd = "psw",
ssl = TRUE),
authenticate = TRUE,
send = TRUE,
attach.files = list_files_attached_location,
file.names = list_files_attached_names,
file.descriptions = list_files_attached_description,
debug = TRUE
)
},
error = function(e) {
print(e)
stop()
})
For those who suggested using 'mailR' - this is the way to go, but it is difficult to install of ubuntu. Here is a resource to install 'javaR' on ubuntu which will allow you to install 'mailR'
https://www.r-bloggers.com/2018/02/installing-rjava-on-ubuntu/
Late to this, but you can avoid shelling out to vbscript by using RDCOMClient correctly, as in this example from R-Help.
> sendEmail(ema = "r-help at r-project.org",
name = "R-help mailing list",
subject = "How to send Email from R using the RDCOMClient"
msgBody = "here is the body of the message")
The package RDCOMClient is available at http://www.omegahat.org/RDCOMClient.
"sendEmail" <-
function(ema, name, subject, msgBody, deliverNow = TRUE)
{
require(RDCOMClient)
ema <- paste("SMPT:", ema, sep="") ## prepend protocol to address
## create an e-mail session
session <- COMCreate("Mapi.Session")
session$Logon()
## add a message to the outbox collection of messages
outbox <- session[["Outbox"]]
msg <- outbox[["Messages"]]$Add(subject, msgBody)
## add recipient's name (TODO: addMultiple() or loop, if many recipients)
msg[["Recipients"]]$Add(name, ema)
msg$Send()
if(deliverNow)
msg$DeliverNow()
session$Logoff() ## wrap up
}