I would like to retrieve a list of tweets from Twitter for a given hashtag using package RJSONIO in R. I think I am pretty close to the solution, but I seem to miss one step.
My code reads as follows (in this example, I use #NBA as a hashtag):
library(httr)
library(RJSONIO)
# 1. Find OAuth settings for twitter:
# https://dev.twitter.com/docs/auth/oauth
oauth_endpoints("twitter")
# Replace key and secret below
myapp <- oauth_app("twitter",
key = "XXXXXXXXXXXXXXX",
secret = "YYYYYYYYYYYYYYYYY"
)
# 3. Get OAuth credentials
twitter_token <- oauth1.0_token(oauth_endpoints("twitter"), myapp)
# 4. Use API
req=GET("https://api.twitter.com/1.1/search/tweets.json?q=%23NBA&src=typd",
config(token = twitter_token))
req <- content(req, as = "text")
response=fromJSON(req)
How can I get the list of tweets from object 'response'?
Eventually, I would like to get something like:
searchTwitter("#NBA", n=5000, lang="en")
Thanks a lot in advance!
The response object should be a list of length two: statuses and metadata. So, for example, to get the text of the first tweet, try:
response$statuses[[1]]$text
However, there are a couple of R packages designed to make just this kind of thing easier: Try streamR for the streaming API, and twitteR for the REST API. The latter has a searchTwitter function exactly as you describe.
Related
I am trying to connect to Qualtrics API using Rstudio Cloud "httr" package to download mailing lists. After a review of the API documentation I was unable to download the data, getting the following error after running the code:
"{"meta":{"httpStatus":"400 - Bad Request","error":{"errorMessage":"Expected authorization in headers, but none provided.","errorCode":"ATP_2"},"requestId":"8fz33cca-f9ii-4bca-9288-5tc69acaea13"}}"
This does not makes me any sense since I am using a inherit auth from parent token. Here is the code:
install.packages("httr")
library(httr)
directoryId<-"POOL_XXXXX"
mailingListId <- "CG_XXXXXX"
apiToken<-"XXXX"
url<- paste("https://iad1.qualtrics.com/API/v3/directories/",directoryId,
"/mailinglists/",mailingListId,"/optedOutContacts", sep = "")
response <- VERB("GET",url, add_headers('X_API-TOKEN' = apiToken),
content_type("application/octet-stream"))
content(response, "text")
Any help will be appreciated.
Thanks in advance.
Your call to httr::VERB breaks the API token and the content type into two arguments to the function, but they should be passed together in a vector to a single "config" argument. Also, content_type isn't a function, it's just the name of an element in that header vector. This should work:
response <- VERB("GET", url, add_headers(c(
'X_API-TOKEN' = apiToken,
'content_type' = "application/octet-stream")))
Note that mailing lists will be returned by Qualtrics as lists that will include both a "meta" element and a "result" element, both of which will themselves be lists. If the list is long, the only the first 100 contacts on the list will be returned; there will be an element response$result$nextpage that will provide the URL required to access the next 100 results. The qualtRics::fetch_mailinglist() function does not work with XM Directory contact lists (which is probably why you got a 500 error when using it), but the code for unpacking the list and looping over each "nextpage" element might be helpful.
I am using rtweet package in R to extract tweets of particular hashtag which basically needs appname,api_key,api_secret,access_token,access_token_secret.
So i have created an app in Twitter to get all the above details. And then I pass the above as follows
twitter_token <- create_token(
app = appname,
consumer_key = api_key,
consumer_secret = api_secret)
tweets <-search_tweets("#iPhone8", n=1000, include_rts = FALSE, type = "recent",lang = "en")
So this will basically extract tweets with 88 variables. Now I want to write these tweets to an excel file for further analysis.
And for that I am using the below code.
tweets.df <-twListToDF(tweets)
When I run this I am getting the following error.
Error in twListToDF(tweets) :
Not all elements of twList are of the same class
Just want to understand what could be the potential issue here.
Problem is I cannot put the exact code here as that needs my api key and all.
Any help would be really appreciated.
Regards,
Akash
You can as well use this
tweets.df <-as.data.frame(tweets)
The answer to the question located here does not seem to work anymore: Obtaining twitter screen names from a twitter list
Does anyone know if the twitteR package has been changed since it was answered in 2015. If so, is there a way to download the members of a public list in the current version?
Here's the previous answer's code updated to include a current lists. Requires a Twitter API authorization. It now returns a list of length 0 when it should have a list of the 20 Premier League club names.
library(rjson)
library(httr)
library(twitteR)
twlist <- "premier-league-clubs"
twowner <- "TwitterUK"
api.url <- paste0("https://api.twitter.com/1.1/lists/members.json?slug=",
twlist, "&owner_screen_name=", twowner, "&count=5000")
response <- POST(api.url, config(token=twitteR:::get_oauth_sig()))
response.list <- fromJSON(content(response, as = "text", encoding = "UTF-8"))
users.names <- sapply(response.list$users, function(i) i$name)
users.screennames <- sapply(response.list$users, function(i) i$screen_name)
head(users.names)
The author of the package in his github account mentions that the twitteR package is deprecated in favor or rtweet. Probably you have to take a look to the documentation of the rtweet package.
Swapping from POST to GET when making the request seems to work for lists I am retrieving.
here's my code:
library(RCurl)
library(twitteR)
library(ROAuth)
#run the libraries ctrl+r
api_key<-"3snovyV8bNn8jVWu93UNP61rd"
api_secret<-"1U3crdPY2kS4Nx12OaxjlE2geyDFSzJXJnVSJaEvIUV3LqHdCa"
access_token<-"2457340967-8wOYs9X7m66RzywCtT6KHlow6hP8zTNpSMTid7Y"
access_token_secret<-"gTqLbHGPayMgF3FG0o0tb42yBWd3TiCZ67vhXOfnLICZT"
setup_twitter_oauth(api_key,api_secret,access_token,access_token_secret)
#ctrl+r
searchTwitter("Cristiano Ronaldo", n=25, lang="en", since=NULL,
until=NULL,
locale=NULL, geocode=NULL, sinceID=NULL, maxID=NULL,
resultType=NULL, retryOnRateLimit=120)
#ctrl+r
Rtweets(n=25, lang="en", since=NULL)
I'm looking for a command in order to specify the tweets I want to extract by account (i.e. Cristiano Ronaldo's official Twitter account).
Thanks!!!
I've used the smappR package's function getTimeline to get tweets from a single user account.
You'll need to install smappR from GitHub, if you haven't done so previously.
library(devtools)
install_github("SMAPPNYU/smappR")
You could then use the getTimeline function to get Ronaldo's tweets:
getTimeline(screen_name = "Cristiano",
filename = "ronaldo_tweets.json", # where tweets will be stored
n=500, ## number of tweets to download (max is 3,200)
oauth_folder = "~/Dropbox/credentials")
The screen_name argument refers to the user's twitter handle. Note also that smappR assumes that you have one or more OAuth objects in a folder. You can read about their setup here: https://github.com/SMAPPNYU/smappR
I hope that this helps.
I saw a cute demonstration of tweeting from R in a presentation some months ago. The scratch code used by the presenter is here:
http://www.r-bloggers.com/twitter-from-r%E2%80%A6-sure-why-not/
the code is short and sweet:
library("RCurl")
opts <- curlOptions(header = FALSE,
userpwd = "username:password", netrc = FALSE)
tweet <- function(status){
method <- "http://twitter.com/statuses/update.xml?status="
encoded_status <- URLencode(status)
request <- paste(method,encoded_status,sep = "")
postForm(request,.opts = opts)
}
With this function, you can send a tweet simply by using the update function:
tweet("This tweet comes from R! #rstats")
I thought that this could be a useful way of announcing when long jobs are completed. I tried to run this on my machine, and I got some error:
[1] "\n\n Basic authentication is not supported\n\n"
attr(,"Content-Type")
charset
"application/xml" "utf-8"
Warning message:
In postForm(request, .opts = opts) : No inputs passed to form
I'm wondering if there has been some changes on the twitter end of this, that make this code produce this error? I don't know too much about getting R to talk to webpages, so any guidance is much appreciated!!
E
Yes, the basic authentication scheme was disabled on the 16th August 2010.. You'll need to set it up to use OAuth. Unfortunately that is not nearly as simple as using basic authentication
See this twitter wiki page for more information and this StackOverflow question about OAuth for R.
Besides the code you show, there is also a full-blown twitteR package on CRAN you could look at.
The easiest way to tweet in R through the Twitter-API is to use the twitteR Package.
You can set your Twitter-API-APP here: https://apps.twitter.com/
First step is to authenticate:
consumer_key <- "yourcredentials"
consumer_secret <- "yourcredentials"
access_token <- "yourcredentials"
access_secret <- "yourcredentials"
setup_twitter_oauth(consumer_key, consumer_secret, access_token, access_secret)
And just tweet (limit per day:2400 tweets):
tweet("Hello World")
If twitteR does not work or you simply want to try to build it yourself ...
See here for a demo of how to do your own Twitter authentication and use of the API with help of the httr package.