I seem to be having problem pulling out the text content of the following query without making another call:
http://tinyurl.com/mgsewz2 via the mqlread api
{
"id": "/en/alcatraz_island",
"/common/topic/description": [{}],
"/common/topic/topic_equivalent_webpage": [],
"/common/topic/official_website": null
}
I can't retrieve the following
description
equivalent webpage (I'm looking for the en wiki page)
, but I can obtain the official_website url.
It looks like I can get it via the search api via output= but I can't walk through the entire set that I'm looking for without getting search request is too large error.
http://markmail.org/message/hd6pcveta7bhuz25#query:+page:1+mid:u7xegxlhtmhwiqbl+state:results
Thanks!
It you want to download large subsets of Freebase data, your best bet is to use the Freebase RDF Dumps. They contain all the properties that you listed above.
Related
I'm trying to get images for restaurants using the HERE Places (Search) API.
I'm using the "Browse" entrypoint, and then using the href in there to get a restaurant's details. In it, I keep on getting this:
media: {
images: {
available:0
items: [ ]
}
The same for reviews and ratings.
Based on other posts here, I'm confused what the problem is, as one post seemingly says its a bug, and one post seemingly says it's just the way the API is.
First of all, "HERE Places API" is deprecated. You should migrate to "HERE Geocoding and Search API v7". Check this out https://developer.here.com/documentation/geocoding-search-api/migration_guide/index.html
As already explained in this question
Include Review,Rating and Images in places API , the API will return the place IDs of external suppliers (TripAdvisor, Yelp etc). This is true also for the latest "HERE Geocoding and Search API v7". With these IDs, you can retrieve other details (such images, reviews, etc) from external system APIs.
I'm using the Interpret and Evaluate methods from Project Academic Knowledge.
If you search for Composite(J.JN='jama') and include J.JId in the request, you'll get a response with the Id 172573765:
{
"logprob": -14.823,
"prob": 3.651345212E-07,
"Id": 2107832644,
"J": {
"JId": 172573765
}
}
You can find more details about that journal by opening: https://academic.microsoft.com/journal/172573765
However, there doesn't seem to be a way to retrieve that same information (Number of papers, number of citations, website, about) using the API. How can we get this (other than by accessing the URL of the journal)?
Project Academic Knowledge allows you to retrieve journal entities using the Evaluate method. The query is simply Id=JId. For example, to retrieve the journal name, publication and citation counts you'd use:
https://api.labs.cognitive.microsoft.com/academic/v1.0/evaluate?subscription-key=SUBSCRIPTION_KEY&attributes=Id,JN,DJN,CC,PC&expr=Id=172573765
See the journal entity documentation page for a list of the available attributes you can request.
I try to download the reports available in Salesforce via the URL, e.g.
http://YOURInstance.my.salesforce.com/012389u13541?export=1&enc=UTF-8&xf=csv
in R.
I already did some investigation to access the report via HTTR-GET, however, up until today without any meaningful outcomes. Unfortunately, R is downloading HTML-code instead of the desired csv file. I also tried to realize the approach suggested here:
https://salesforce.stackexchange.com/questions/47414/download-a-report-using-python
The package "RForcecom" allows the interaction via an API, but I was not able to figure out how to realize above solution in R.
General GET-Request:
GET("http://YOUR_Instance.my.salesforce.com/012389u13541?export=1&enc=UTF-8&xf=csv")
I expect the output to be in csv format, but I receive the report data as html source code.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3...
<html>
<head>
<meta HTTP-EQUIV="PRAGMA" CONTENT="NO-CACHE">
...
Did anyone of you guys encounter same issues and can provide guidance? Any kind of help is much appreciated. Thanks in advance!
UPDATED and not-working R-Snippet:
library(RForcecom)
library(httr)
username='username'
password='password'
instanceURL <- "https://login.salesforce.com/"
session <- rforcecom.login(username, password, instanceURL)
sid=as.character(session['sessionID'])
url='http://YOURInstance.my.salesforce.com/012389u13541?export=1&enc=UTF-8&xf=csv'
getData=GET(url,add_headers('Content-Type'='application/json','Authorization'=paste0("Bearer ",sid),'X-PrettyPrint'='1'),set_cookies('sid'=sid))
Are you sure you have a valid report id? It doesn't look right (did you just obfuscate it for purposes of this post?). What is in that HTML you're getting, an error message? SF login screen?
What you're doing is effectively "screen scraping". This is not a real API, it can break at any time, you should find/build something that properly uses Salesforce Analytics API. You've been warned.
But if you're after a quick and dirty solution...
You need to pretend you're an authenticated user, that you have a valid session id. Add a cookie to your GET request.
How to get a valid session id?
You'd have to log in to SF first (for example use SOAP API's login call or I listed some REST api ideas here: https://stackoverflow.com/a/56034159/313628 )
or display some user's session ID in a SF formula, visualforce page and user would copy-paste it to your app.
Once you have it - add a Cookie header to your GET with value sid=<session id goes here>
Here's a raw request & response in SoapUI.
I recently struggled with the same issue, there's a magic parameter you need to add to the query : isdtp=p1
so if you try:
http://YOURInstance.my.salesforce.com/012389u13541?export=1&enc=UTF-8&xf=csv&isdtp=p1
it should return you the file directly.
In your example, I don't think that you can use the rforcecom session with httr functions as you are trying.
Here is a slightly different way to solve the problem.
Rather than trying to retrieve a report that you already created in Salesforce, why not specify the report in SOQL and use rforcecom.query function to execute the SOQL from r. That would return the data in a data frame and would require no further data wrangling in r to make it useable.
I use this technique often and once you get used to the Salesforce API I think that its probably faster and more powerful for most use cases.
Here is a simple function that I use to return select opportunity data for all opportunities in Salesforce.
getSFOpps <- function(session) {
#Construct SOQL Query
soql <- "SELECT Id,
Name,
AccountId,
Amount,
CurrencyIsoCode,
convertCurrency(Amount) usd_amount,
CloseDate,
CreatedDate,
Region__c,
IsClosed,
IsWon,
LastActivityDate,
LeadSource,
OwnerId,
Probability,
StageName,
Type,
IsDeleted
FROM Opportunity"
#Retrieve Opp information
as_tibble(RForcecom::rforcecom.query(session, soql))
}
It requires that you pass in a valid session from Rforcecom.login but you seem to have that part working from your code above.
I hope this helps ...
As of v0.2.0, the {salesforcer} R package implements the Salesforce Reports and Dashboards REST API. You can execute and manage reports without needing to write functions from scratch to pull down report data. Below is an example of how to find a report in your Org and then retrieve its data. You can also just use the report Id which appears in the URL bar when viewing the report in Salesforce (highlighted in red in the screenshot below).
# install.packages('salesforcer')
library(dplyr, warn.conflicts = FALSE)
library(salesforcer)
# Authenticate using username, password, and security token ...
sf_auth(username = "test#gmail.com",
password = "{PASSWORD_HERE}",
security_token = "{SECURITY_TOKEN_HERE}")
# ... or using OAuth 2.0 authentication
sf_auth()
# find a report in your org and run it
all_reports <- sf_query("SELECT Id, Name FROM Report")
this_report_id <- all_reports$Id[1]
results <- sf_run_report(this_report_id)
results
I am using R and I need to retrieve the few most recent posts from a Twitter user (#ExpressNewsPK) using twitteR api. I have created an account and have an access token, etc. I have used the following command to extract the tweets:
setup_twitter_oauth(consumerkey,consumersecret,accesstoken,accesssecret)
express_news_tweets <- searchTwitter("#ExpressNewsPK", n = 10, lang = "en" )
However, the posts that are returned aren't the most recent ones from this user. Where have I made a mistake?
I think searchTwitter would search with the search string provided (here #ExpressNewsPK). So instead of giving tweets by #ExpressNewsPK it would give tweets which are directed to #ExpressNewsPK.
To get tweets from #ExpressNewsPK, you have a function named userTimeline which would give tweets from a particular user.
So after you are done with setup_twitter_oauth, you can try
userTimeline("ExpressNewsPK")
read more about it at ?userTimeline
When you use searchTwitter(), you call the Twitter Search API. Search API only returns a sample history of the tweets.
What you really need to do is to call Twitter Streaming API. Using it you'll be able to download tweets in near real time. You can read more about the Streaming API here: https://dev.twitter.com/streaming/overview
I'm trying to retrieve the full topic description/summary for some Freebase articles. I have been using the Freebase topic API, which returns this type of results: http://www.freebase.com/experimental/topic/standard?id=/en/jimi_hendrix
But I notice that the description is not complete and ends with "...". Is there a way to use some Freebase API to obtain the article's full description?
Does Freebase even store the complete description or does it just stores a portion of the description from Wikipedia?
Freebase just stores a portion of the Wikipedia description but there is usually more than what's given by the topic API.
To get the "full" text for a Wikipedia blurb associated with a Freebase topic you first need to query the Read API for a list of related articles like this:
{
"id": "/en/jimi_hendrix",
"/common/topic/article": [{}]
}
Try it in the Query Editor
Then choose one or more of the articles that it returns and feed its ID into the /trans/raw API like this:
http://api.freebase.com/api/trans/raw/m/043dz
You'll notice that the blurb of text that gets returned is a bit longer (1200 chars) and doesn't have the "..." but its still chopped off at the end.
When I display Freebase topic descriptions in a web page I have some code to clean it up before hand. I split it apart into paragraphs by looking for newlines and then if the last paragraph doesn't end with a period, exclamation mark or question mark I just throw away that paragraph. The way the Wikipedia blurbs are written, you usually only need the first paragraph anyways.
You can also fetch this directly from MQL with the "text" extension:
{
"id": "/en/jimi_hendrix",
"/common/topic/article": [{
"text": {
"maxlength": 16384,
"chars": null
}
}]
}
Note that you'll need to turn on MQL extensions for this to work - see here for an example of this in action.
Edit August 2012: while this works for the original freebase.com hosted APIs, the MQL extension functionality has been removed from the new googleapis.com hosted APIs, so this method shouldn't be relied on any more.