I am getting 500 status. Can you anyone help me on this please
I am trying to connect Salesforce using R when i try perform the login operation. I am not successful, can you please look into this code and correct me where i am missing.
library(RCurl)
library(httr)
body1 = '<?xml version="1.0" encoding="utf-8"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<login xmlns="urn:partner.soap.sforce.com">
<username>xxxxxx.Test#xxxxxx.com.xxxxxxx</username>
<password>xxxxxxxxxxxx</password>
</login>
</s:Body>
</s:Envelope>'
x <- httr::POST(url = "https://test.salesforce.com/services/Soap/48.0", body = body1, content_type("text/xml;charset=UTF-8"),SOAPAction = "login")
x$status_code
Thanks for answering the questions. After several tries, I understood there was some issue in the headers. I fixed the headers and its now working great. I am able to perform a query in Salesforce without any issues. I have posted my complete code here so anyone can make use for it. I am not an expert in this but still managed with this code.
#######Required Packages
library(RCurl)
library(httr)
library(XML)
library(jsonlite)
library(xml2)
library(magrittr)
library(randomNames)
library(rlang)
library(generator)
library(stringr)
require(data.table)
body1 = '<?xml version="1.0" encoding="utf-8"?>\
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">\
<s:Body>\
<login xmlns="urn:partner.soap.sforce.com">\
<username>XXXXXXXXXXXXXX</username>\
<password>XXXXXXXXXXXXXXX</password>\
</login>\
</s:Body>\
</s:Envelope>\n'
require(httr)
################################Posting the Login##############################################
result <- POST("https://test.salesforce.com/services/Soap/u/48.0",body = body1,add_headers(.headers = c("Content-Type"="text/xml",'SOAPAction' = "https://test.salesforce.com")))
##############Parsing XML###############################################################
Output <- content(result)
doc = xmlTreeParse(Output, useInternal = TRUE)
top = xmlRoot(doc)
xmlName(top)
names(top)
names(top[[1]] [[1]] [[1]])
Session = top[[ 1 ]] [[1]] [[1]] [["sessionId"]]
SessionId <- Session [1][1]$text
tok <- xmlToList(SessionId)
Se <- paste("Bearer",tok,Sep="")
Se = trimws(Se)
Sever = top[[ 1 ]] [[1]] [[1]] [["serverUrl"]]
SessionURL <- Sever [1][1]$text
################SF Query###########################################
SFQuery <- "SELECT Id,FirstName FROM Conatct limit 10"
SFQuery_Req <- GET(Pam,query = list(q=SFQuery),add_headers(Authorization=Se))
SFQuery_Con <- content(SFQuery_Req, as = 'text') %>% fromJSON()
SFQuery_Con_DF <- data.frame(SFQuery_Con$records)
It can be many things, not all related to your code...
Your SOAP namespaces might be wrong. Maybe try with SoapUI / Postman / curl first and once you get these to work - check the code. Try with this version of the tags
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:partner.soap.sforce.com">
<soapenv:Body>
<urn:login>
<urn:username>usernamegoeshere</urn:username>
<urn:password>passwordgoeshere</urn:password>
</urn:login>
</soapenv:Body>
</soapenv:Envelope>
Log in to the SF org manually and check that user's login history at bottom of the page. Do you see your attempt at all? If not - are you sure you're connecting to right place (production vs sandbox)? Or maybe you have some proxy installed by your corporate IT and it just swallows the connections... If there's an entry in the login history - does it say invalid password? Too many failed login attempts? Security token needed?
It's possible admin disabled logging in from generic test.salesforce.com and you'll have to use what you see in Setup -> My Domain.
It's possible admin set your user for Single Sign-On (do you see normal SF login page or are redirected to some Active Directory stuff for example). If that's the case it'll be trickier to connect via API.
Maybe your user is not API-enabled, that'd be a Profile / Permission Set thing. Do you use system administrator account? Are there any login policies on the profile like access only from certain IPs or during certain login hours.
I maintain an R package called {salesforcer} that allows users to interact with 6 different Salesforce APIs (SOAP, REST, Bulk 1.0, Bulk 2.0, Metadata, Reports and Dashboards) to manage their Orgs. If one of these APIs suite your needs I would recommend using the package so that you do not need to write the code from scratch. Below is an example for how to connect using either OAuth 2.0, sometimes referred to as Single Sign-On, or with a username, password, and security token. Please visit the package's documentation site for additional detail and vignettes at: https://stevenmmortimer.github.io/salesforcer/
# install.packages('salesforcer')
library(salesforcer)
# Using OAuth 2.0 authentication
sf_auth()
# Using Basic Username-Password authentication
sf_auth(username = "test#gmail.com",
password = "{PASSWORD_HERE}",
security_token = "{SECURITY_TOKEN_HERE}")
# pull down information of person logged in
# it's a simple API call to get started
# and confirm connection to the APIs
user_info <- sf_user_info()
sprintf("Organization Id: %s", user_info$organizationId)
#> [1] "Organization Id: 00D6A0000003dN3UAI"
sprintf("User Id: %s", user_info$userId)
#> [1] "User Id: 0056A000000MPRjQAO"
Related
I try to download the reports available in Salesforce via the URL, e.g.
http://YOURInstance.my.salesforce.com/012389u13541?export=1&enc=UTF-8&xf=csv
in R.
I already did some investigation to access the report via HTTR-GET, however, up until today without any meaningful outcomes. Unfortunately, R is downloading HTML-code instead of the desired csv file. I also tried to realize the approach suggested here:
https://salesforce.stackexchange.com/questions/47414/download-a-report-using-python
The package "RForcecom" allows the interaction via an API, but I was not able to figure out how to realize above solution in R.
General GET-Request:
GET("http://YOUR_Instance.my.salesforce.com/012389u13541?export=1&enc=UTF-8&xf=csv")
I expect the output to be in csv format, but I receive the report data as html source code.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3...
<html>
<head>
<meta HTTP-EQUIV="PRAGMA" CONTENT="NO-CACHE">
...
Did anyone of you guys encounter same issues and can provide guidance? Any kind of help is much appreciated. Thanks in advance!
UPDATED and not-working R-Snippet:
library(RForcecom)
library(httr)
username='username'
password='password'
instanceURL <- "https://login.salesforce.com/"
session <- rforcecom.login(username, password, instanceURL)
sid=as.character(session['sessionID'])
url='http://YOURInstance.my.salesforce.com/012389u13541?export=1&enc=UTF-8&xf=csv'
getData=GET(url,add_headers('Content-Type'='application/json','Authorization'=paste0("Bearer ",sid),'X-PrettyPrint'='1'),set_cookies('sid'=sid))
Are you sure you have a valid report id? It doesn't look right (did you just obfuscate it for purposes of this post?). What is in that HTML you're getting, an error message? SF login screen?
What you're doing is effectively "screen scraping". This is not a real API, it can break at any time, you should find/build something that properly uses Salesforce Analytics API. You've been warned.
But if you're after a quick and dirty solution...
You need to pretend you're an authenticated user, that you have a valid session id. Add a cookie to your GET request.
How to get a valid session id?
You'd have to log in to SF first (for example use SOAP API's login call or I listed some REST api ideas here: https://stackoverflow.com/a/56034159/313628 )
or display some user's session ID in a SF formula, visualforce page and user would copy-paste it to your app.
Once you have it - add a Cookie header to your GET with value sid=<session id goes here>
Here's a raw request & response in SoapUI.
I recently struggled with the same issue, there's a magic parameter you need to add to the query : isdtp=p1
so if you try:
http://YOURInstance.my.salesforce.com/012389u13541?export=1&enc=UTF-8&xf=csv&isdtp=p1
it should return you the file directly.
In your example, I don't think that you can use the rforcecom session with httr functions as you are trying.
Here is a slightly different way to solve the problem.
Rather than trying to retrieve a report that you already created in Salesforce, why not specify the report in SOQL and use rforcecom.query function to execute the SOQL from r. That would return the data in a data frame and would require no further data wrangling in r to make it useable.
I use this technique often and once you get used to the Salesforce API I think that its probably faster and more powerful for most use cases.
Here is a simple function that I use to return select opportunity data for all opportunities in Salesforce.
getSFOpps <- function(session) {
#Construct SOQL Query
soql <- "SELECT Id,
Name,
AccountId,
Amount,
CurrencyIsoCode,
convertCurrency(Amount) usd_amount,
CloseDate,
CreatedDate,
Region__c,
IsClosed,
IsWon,
LastActivityDate,
LeadSource,
OwnerId,
Probability,
StageName,
Type,
IsDeleted
FROM Opportunity"
#Retrieve Opp information
as_tibble(RForcecom::rforcecom.query(session, soql))
}
It requires that you pass in a valid session from Rforcecom.login but you seem to have that part working from your code above.
I hope this helps ...
As of v0.2.0, the {salesforcer} R package implements the Salesforce Reports and Dashboards REST API. You can execute and manage reports without needing to write functions from scratch to pull down report data. Below is an example of how to find a report in your Org and then retrieve its data. You can also just use the report Id which appears in the URL bar when viewing the report in Salesforce (highlighted in red in the screenshot below).
# install.packages('salesforcer')
library(dplyr, warn.conflicts = FALSE)
library(salesforcer)
# Authenticate using username, password, and security token ...
sf_auth(username = "test#gmail.com",
password = "{PASSWORD_HERE}",
security_token = "{SECURITY_TOKEN_HERE}")
# ... or using OAuth 2.0 authentication
sf_auth()
# find a report in your org and run it
all_reports <- sf_query("SELECT Id, Name FROM Report")
this_report_id <- all_reports$Id[1]
results <- sf_run_report(this_report_id)
results
I am trying to crawl and scrape a website's tables. I have an account with the website, and I found out that Rcrawl could help me with getting parts of the table based on specific keywords, etc. The problem is that on the GitHub page there is no mentioning of how to crawl a site with account/password protection.
An example for signing in would be below:
login <- list(username="username", password="password",)
Do you have any idea if Rcrawler has this functionality? For example something like:
Rcrawler(Website = "http://www.glofile.com" +
list (username = "username", password = "password" + no_cores = 4, no_conn = 4, ExtractCSSPat = c(".entry-title",".entry-content"), PatternsNames = c("Title","Content"))
I'm confident my code above is wrong, but I hope it gives you an idea of what I want to do.
To crawl or scrape password-protected websites in R, more precisely HTML-based Authentication, you need to use web driver to stimulate a login session, Fortunately, this is possible since Rcrawler v0.1.9, which implement phantomjs web driver ( a browser but without graphics interface).
In the following example will try to log in a blog website
library(Rcrawler)
Dowload and install web driver
install_browser()
Run the browser session
br<- run_browser()
If you get an error than disable your antivirus or allow the program in your system setting
Run an automated login action and return a logged-in session if successful
br<-LoginSession(Browser = br, LoginURL = 'http://glofile.com/wp-login.php'
LoginCredentials = c('demo','rc#pass#r'),
cssLoginFields =c('#user_login', '#user_pass'),
cssLoginButton ='#wp-submit' )
Finally, if you know already the private pages you want to scrape/download use
DATA <- ContentScraper(... , browser =br)
Or, simply crawl/scrape/download all pages
Rcrawler(Website = "http://glofile.com/",no_cores = 1 ,no_conn = 1,LoggedSession = br ,...)
Don't use multiple parallel no_cores/no_conn as many websites reject multiple sessions by one user.
Stay legit and honor robots.txt by setting Obeyrobots = TRUE
You access the browser functions, like :
br$session$getUrl()
br$session$getTitle()
br$session$takeScreenshot(file = "image.png")
I'm trying to harvest data from www.flightradar24.com using rvest. I've got a subscription, so I want to log in and get access to more data. This is the code I'm using to log in (I'm using my email and password instead of "email" and "password"):
library(rvest)
url0 = "https://www.flightradar24.com/"
session = html_session(url0)
form = html_form(session)[[1]]
filled_form = set_values(form, 'email'="email", 'password'="password")
session_loggedin = submit_form(session, filled_form)
I get the following message:
Submitting with '<unnamed>'
I want to check whether this is actually working, so I'm trying with this:
jump_to(session_loggedin, "https://www.flightradar24.com/account")
And this is the message I get:
<session> https://www.flightradar24.com/premium
Status: 200
Type: text/html; charset=utf-8
Size: 98649
As you can see, it's going to https://www.flightradar24.com/premium (website where you can compare the different subscriptions) instead of https://www.flightradar24.com/account (your account page once you log in).
I've tried grabbing the data from this website https://www.flightradar24.com/data/aircraft/ln-lnc using html_table, but it only captures the data included in the basic subscription, so that also implies that I'm not successfully logging in.
Any help would be very much appreciated.
It appears that Yahoo has discontinued support for oauth1.0. I am trying to update my R code to use oauth2.0 using the httr package, and am stumped. I am able to get token, but am unable to use the token to query the api.
I continue to get "You are not authorized to view this league".
options("httr_oob_default" = T)
library(httr)
b_url <- "https://fantasysports.yahooapis.com" #base url
#Create Endpoint
yahoo <- httr::oauth_endpoint(authorize = "https://api.login.yahoo.com/oauth2/request_auth"
, access = "https://api.login.yahoo.com/oauth2/get_token"
, base_url = b_url)
#Create App
yahoo_app <- httr::oauth_app("yahoo", key=cKey, secret = cSecret,redirect_uri = "oob")
#Open Browser to Authorization Code
httr::BROWSE(httr::oauth2.0_authorize_url(yahoo, yahoo_app, scope="fspt-r"
, redirect_uri = yahoo_app$redirect_uri))
#Code = zp6v82a
#Create Token
yahoo_token<- httr::oauth2.0_access_token(yahoo,yahoo_app,code="zp6v82a")
Now im not sure where to go from here.
If there is any advice, or an easier method please let me know. I am an amateur coder so please, be gentle.
How can I allow new R users to send information to a Google Form? (RSelenium requires a bit of set up, at least for headless browsing, so it's not the best candidate IMO but I may be missing something that makes it the best choice).
I have some new R users I want to get responses from interactively and send to a secure location. I have chosen Google Forms to pass the information to, as it allows one way sends of the info and doesn't allow the user access to the spreadsheet that is created from the form.
Here's a url of this form:
url <- "https://docs.google.com/forms/d/1tz2RPftOLRCQrGSvgJTRELrd9sdIrSZ_kxfoFdHiqD4/viewform"
To give context here's how I'm using R to interact with the user:
question <- function(message, opts = c("Yes", "No")){
message(message)
ans <- menu(opts)
if (ans == "2") FALSE else TRUE
}
question("Was this information helpful?")
I want to then send that TRUE/FALSE to the Google form above. How can I send a response to the Google Form above from within R in a way that I can embed in code the user will interact with and doesn't require difficult set up by the user?
Add on R packages are fine if they accomplish the task.
You can send a POST query. Here an example using httr package:
For example:
library(httr)
send_response<-
function(response){
form_url <- "https://docs.google.com/forms/d/1tz2RPftOLRCQrGSvgJTRELrd9sdIrSZ_kxfoFdHiqD4/formResponse"
POST(form_url,
query = list(`entry.1651773982`=response)
)
}
Then you can call it :
send_response(question("Was this information helpful?"))