I am using the Sandbox account with limited request. I am trying to filter out retweets in my request by the following:
consumer_key <-
consumer_secret <-
access_token <-
access_secret <-
app <-
token = rtweet::create_token(app,consumer_key,consumer_secret,access_token,access_secret)
#period 2
dataBTC29 <- search_30day("Bitcoin analysis lang:en -is:retweet", n = 100, env_name = "Tweets30", fromDate = "202210130000", toDate = "202210130759", parse = FALSE)
However, one of the tweet attributes is "is:retweet", including this you will get retweets, but I read somewhere that using -is:retweet which exclude retweets in your search. However, when I do this, I get the following error:
Error: Twitter API failed [422]
Check error message at https://developer.twitter.com/en/support/twitter-api/error-troubleshooting
When I look up the error, this is what I get:
Check that the data you are sending in your request is valid. For example, this data could be the JSON body of your request or an image.
And if I run the following in R:
consumer_key <-
consumer_secret <-
access_token <-
access_secret <-
app <-
token = rtweet::create_token(app,consumer_key,consumer_secret,access_token,access_secret)
#period 2
dataBTC29 <- search_30day("Bitcoin analysis lang:en", n = 100, env_name = "Tweets30", fromDate = "202210130000", toDate = "202210130759", parse = FALSE)
It does return me a dataset, but 100 rows of only retweets. How can I request such thin gin order for it to work?
Related
My goal is to get EVERY tweet ever for any twitter account. I picked the NYTimes for this example.
The code below works, but it only pulls the last 100 tweets. max_results does not allow you to put a value over 100.
The code below almost fully copy-paste-able, you would have to have your own bearer token.
How can I expand this to give me every tweet from an account?
One idea is that I can loop it for every day since the account was created, but that seems tedious if there is a faster way.
# NYT Example --------------------------------------------------------------------
library(httr)
library(jsonlite)
library(tidyverse)
bearer_token <- "insert your bearer token here"
headers <- c(`Authorization` = sprintf('Bearer %s', bearer_token))
params <- list(`user.fields` = 'description')
handle <- 'nytimes'
url_handle <- sprintf('https://api.twitter.com/2/users/by?usernames=%s', handle)
response <- httr::GET(url = url_handle,
httr::add_headers(.headers = headers),
query = params)
json_data <- fromJSON(httr::content(response, as = "text"), flatten = TRUE)
json_data %>%
as_tibble()
NYT_ID <- json_data$data$id
url_handle <- paste0("https://api.twitter.com/2/users/", NYT_ID, "/tweets")
params <- list(`tweet.fields` = 'id,text,author_id,created_at,attachments,public_metrics',
`max_results` = '100')
response <- httr::GET(url = url_handle,
httr::add_headers(.headers = headers),
query = params)
json_data <- fromJSON(httr::content(response, as = "text"), flatten = TRUE)
NYT_tweets <- json_data$data %>%
as_tibble() %>%
select(-id, -author_id, -9)
NYT_tweets
For anyone that finds this later on, I found a solution that works for me.
Using the parameters of start_time and end_time you can clarify dates for the tweets to be between. I was able to pull all tweets from November for example and then rbind those to the ones from December, etc. Sometimes I had to do two tweet pulls (half of March, second half of March) to get all of them, but it worked for this.
params <- list(`tweet.fields` = 'id,text,author_id,created_at,attachments,public_metrics',
`max_results` = '100',
`start_time` = '2021-11-01T00:00:01.000Z',
`end_time` = '2021-11-30T23:58:21.000Z')
This is my code:
library(httr)
library(jsonlite)
library(dplyr)
bearer_token <- Sys.getenv("BEARER_TOKEN")
headers <- c('Authorization' = sprintf('Bearer %s', bearer_token))
params <- list('expansions' = 'attachments.media_keys')
handle <- readline('BenDuBose')
url_handle <-
sprintf('https://api.twitter.com/2/users/by?username=%s', handle)
response <-
httr::GET(url = url_handle,
httr::add_headers(.headers = headers),
query = params)
obj <- httr::content(response, as = "text")
print(obj)
This is my error message:
[1] "{"errors":[{"parameters":{"ids":[""]},"message":"The number of values in the ids query parameter list [0] is not between 1 and 100"}],"title":"Invalid Request","detail":"One or more parameters to your request was invalid.","type":"https://api.twitter.com/2/problems/invalid-request"}"
My end goal is to scrape an image from a specific tweet ID/user. I already have a list of users and tweet IDs, along with attachments.media_keys. But, I don't know how to use HTTR and I am trying to copy the Twitter Developer example verbatim to learn, but it isn't working.
I am trying to get tweets using Twitter API (I know about packages that allows you to collect tweets, but for my task i need to use their own API).
I set up my application and have credentials. But I am stuck at GET stage:
I am using this to build my query, but the problem is in the token I guess...
library(httr)
library(jsonlite)
twitter_token <- create_token(
app = appname,
consumer_key = key,
consumer_secret = secret)
url<-"https://api.twitter.com/1.1/search/tweets.json?q=rstats&src=typed_query"
res=GET(url, add_headers(
'Authorization'=paste("Bearer ", twitter_token))
)
The error I am getting is
Error in as.vector(x, "character") :
cannot coerce type 'environment' to vector of type 'character'
how can I make it work? Pleeease!!!!
searched forums/twitter
it worked!!!
app_keys <- openssl::base64_encode(paste0(key, ":", secret))
r <- httr::POST("https://api.twitter.com/oauth2/token",
httr::add_headers(Authorization = paste0("Basic ", app_keys)),
body = list(grant_type = "client_credentials"))
bearer <- httr::content(r, encoding = "UTF-8")
url<-"https://api.twitter.com/1.1/search/tweets.json?q=rstats&src=typed_query"
res=GET(url, add_headers(
Authorization=paste0("Bearer ", bearer$access_token))
)
Im trying to send over signed api messages using the binance APIs I keep failing with a 404 error. can someone help me out with the below code please?
library(jsonlite)
library(httr)
library(dplyr)
library(digest)
timestamp <- 1516941586 #as.numeric(as.POSIXct(Sys.time()))
post_message <- paste0(timestamp, 'public.api' ) # data_client.id = client
id # data_key = key
sha.message <- toupper(digest::hmac('private.api', object = post_message,
algo = 'sha256', serialize = F))
url <- 'https://api.binance.com/api/v3/account'
body = list('timestamp' = timestamp, 'signature' = sha.message)
body2 <- paste("?timestamp=",timestamp,"&signature=",sha.message, sep = "")
httr::POST(url, body2 = body, verbose())
here is the documentation https://github.com/binance-exchange/binance-official-api-docs/blob/master/rest-api.md
Based on example under section "SIGNED Endpoint Examples for POST /api/v1/order" in the website, you can follow something similar. You will need to replace with your own apiKey and secretKey.
library(httr)
library(openssl)
url <- 'https://api.binance.com/api/v3/account'
apiKey <- "vmPUZE6mv9SD5VNHk4HlWFsOr6aKE2zvsw0MuIgwCIPy6utIco14y7Ju91duEh8A"
secretKey <- "NhqPtmdSJYdKjVHjA7PZj4Mge3R5YNiP1e3UZjInClVN65XAbvqqM6A7H5fATj0j"
timestamp <- 1516941586
recvWindow <- 1e20
postmsg <- paste0("timestamp=", timestamp, "&recvWindow=", recvWindow)
signature <- openssl::sha256(postmsg, key=secretKey)
GET(url,
add_headers("X-MBX-APIKEY"=apiKey),
query=list(timestamp=timestamp, recvWindow=recvWindow, signature=signature),
verbose())
I've been working on a project with the hopes of pulling off instagram post and comment information from instagram posts over the past year.
I am starting right now with a simple code just to pull out information from a single user.
Here is the code:
require(httr)
full_url <- oauth_callback()
full_url <- gsub("(.*localhost:[0-9]{1,5}/).*", x=full_url, replacement="\1")
print(full_url)
app_name <- "Cognitive Model of the Customer"
client_id <- "b03d4a910f0442b9bd1cd79fc06a086f"
client_secret <- "c35f785784fa45cd9eaf786742ae9b3f"
scope = "basic"
instagram <- oauth_endpoint(
authorize = "https://api.instagram.com/oauth/authorize",
access = "https://api.instagram.com/oauth/access_token")
myapp <- oauth_app(app_name, client_id, client_secret)
ig_oauth <- oauth2.0_token(instagram, myapp,scope="basic", type = "application/x-www-form-urlencoded",cache=FALSE)
tmp <- strsplit(toString(names(ig_oauth$credentials)), '"')
token <- tmp[[1]][4]
library(jsonlite)
library(RCurl)
user_info <- fromJSON(getURL(paste('https://api.instagram.com/v1/users/search? q=',"newbalance",'&access_token=',token,sep="")),unexpected.escape = "keep")
The error I am receiving is
Error in simplify(obj, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, :
unused argument (unexpected.escape = "keep")
I'm not sure I understand where this error comes from though.
Before running your code you should load essential packages.
Please load this package and then run your code:
library(rjson)