I was wondering whether anyone has found a way to how to use search_fullarchive() from the "rtweet" package in R with the new Twitter academic research project track?
The problem is whenever I try to run the following code:
search_fullarchive(q = "sunset", n = 500, env_name = "AcademicProject", fromDate = "202010200000", toDate = "202010220000", safedir = NULL, parse = TRUE, token = bearer_token)
I get the following error "Error: Not a valid access token". Is that because search_fullarchive() is only for paid premium accounts and that doesn't include the new academic track (even though you get full archive access)?
Also, can you retrieve more than 500 tweets (e.g., n = 6000) when using search_fullarchive()?
Thanks in advance!
I've got the same problem w/ Twitter academic research API. I think if you set n = 100 or just skip the argument, the command will return you 100 tweets. Also, the rtweet package does not (yet) support the academic research API.
Change your code to this:
search_fullarchive(q = "sunset", n = 500, env_name = "AcademicProject", fromDate = "202010200000", toDate = "202010220000", safedir = NULL, parse = TRUE, token = t, env_name = "Your Environment Name attained in the Dev Dashboard")
Also The token must be created like this:
t <- create_token(
app = "App Name",
'Key',
'Secret',
access_token = '',
access_secret = '',
set_renv = TRUE
)
Related
I can receive messages with the inputs.mqtt_consumer telegraf plugin, but it gives me a lot of data in influxdb.
How can I in the telegraf configuration just count the number of received bytes and messages and report that to influx db?
# Configuration for telegraf agent
[agent]
interval = "20s"
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = "0s"
flush_interval = "10s"
flush_jitter = "0s"
precision = ""
hostname = ""
omit_hostname = false
[[outputs.influxdb_v2]]
urls = ["XXXXXXXXXXXXXXXX"]
token = "$INFLUX_TOKEN"
organization = "XXXXXXXXXXXXXXX"
bucket = "XXXXXXXXXXXXXXX"
[[inputs.mqtt_consumer]]
servers = ["tcp://XXXXXXXXXXXXXXXXXXXXX:1883"]
topics = [
"#",
]
data_format = "value"
data_type = "string"
I tried to google around but din't find any clear ways to do it.
I just want number of bytes and messages received each minute for the selected topic
I did not manage to receive all the messages and count them, but I found a solution where I can get the data from the broker. Not exactly what I asked for but fine for what I need.
topics = [
"$SYS/broker/load/messages/received/1min",
"$SYS/broker/load/messages/sent/1min",
]
...
data_format = "value"
data_type = "float"
library(rgoogleads)
library(gargle)
token <- token_fetch()
token
gads_auth(email = 'xx#gmail.com'
Authentication complete.
ad_group_report <- gads_get_report(
resource = "ad_group",
fields = c("ad_group.campaign",
"ad_group.id",
"ad_group.name",
"ad_group.status",
"metrics.clicks",
"metrics.cost_micros"),
date_from = "2021-01-08",
date_to = "2021-01-10",
where = "ad_group.status = 'ENABLED'",
order_by = c("metrics.clicks DESC", "metrics.cost_micros")
)
i Multi account request
! The request you sent did not return any results, check the entered parameters and repeat the opposition.
Why do I receive this error? I have never received it in Radwords package. Where do I mention the argument for the multiple accounts?
https://cran.r-project.org/web/packages/rgoogleads/rgoogleads.pdf
I'm trying to get some tweets with academictwitteR, but the code throws the following error:
tweets_espn <- get_all_tweets( query = "fluminense",
+ user = "ESPNBrasil",
+ start_tweets = "2020-01-01T00: 00: 00Z " ,
+ end_tweets = "2020-31-12T00 : 00: 00Z " ,
+ n = 10000)
query: fluminense (from:ESPNBrasil) Error in make_query(url =
endpoint_url, params = params, bearer_token = bearer_token, :
something went wrong. Status code: 403 In addition: Warning messages:
1: Recommended to specify a data path in order to mitigate data loss
when ingesting large amounts of data. 2: Tweets will not be stored as
JSONs or as a .rds file and will only be available in local memory if
assigned to an object.
it seems to me that you can only access the Twitter API via academictwitteR if you have been awarded the "academic research" access from the Twitter developer portal. So i dont think it works with the essential or elevated access.
I am currently doing a project on football players. I am trying to scrape some public comments on football players for sentiment analysis. However I can't seem to scrape the comment. Any help would be MUCH appreciated. It is the comments part I can't seem to do. Weirdly enough I had it working but then it stopped and I cant seem to get scraping comments again. The website I am scraping from is : https://sofifa.com/player/192985/kevin-de-bruyne/200025/
likes = []
dislikes = []
follows = []
comments = []
driver_path = '/Users/niallmcnulty/Desktop/GeneralAssembly/Lessons/DSI11-lessons/week05/day2_web_scraping_and_apis/web_scraping/selenium-examples/chromedriver'
driver = webdriver.Chrome(executable_path=driver_path)
# i = 0
for url in tqdm_notebook(urls):
driver.get(url)
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
sleep(0.2)
soup1 = BeautifulSoup(driver.page_source,'lxml')
try:
dislike = soup1.find('button', attrs = {'class':'bp3-button bp3-minimal bp3-intent-danger dislike-btn need-sign-in'}).find('span',{'class':'count'}).text.strip()
dislikes.append(dislike)
except:
pass
try:
like = soup1.find('button', attrs = {'class':'bp3-button bp3-minimal bp3-intent-success like-btn need-sign-in'}).find('span',{'class':'count'}).text.strip()
likes.append(like)
except:
pass
try:
follow = soup1.find('button', attrs = {'class':'bp3-button bp3-minimal follow-btn need-sign-in'}).find('span',{'class':'count'}).text.strip()
follows.append(follow)
except:
pass
try:
comment = soup1.find_all('p').text[0:10]
comments.append(comment)
except:
pass
# i += 1
# if i % 5 == 0:
# sentiment = pd.DataFrame({"dislikes":dislikes,"likes":likes,"follows":follows,"comments":comments})
# sentiment.to_csv('/Users/niallmcnulty/Desktop/GeneralAssembly/Lessons/DSI11-lessons/projects/cap-csv/sentiment.csv')
sentiment_final = pd.DataFrame({"dislikes":dislikes,"likes":likes,"follows":follows,"comments":comments})
# df_sent = pd.merge(df, sentiment, left_index=True, right_index=True)
The comments section is dynamically loaded. you can try to capture it using the driver,
try:
comment_elements = driver.find_elements_by_tag_name('p')
for comment in comment_elements:
comments.append(comment.text)
except:
pass
print(Comments)
I want to access the GetDeepSearchResults info from the Zillow API.
My code:
library(ZillowR)
zapi_key = getOption('Myapikey')
GetDeepSearchResults(
address = '600 S. Quail Ct.',
zipcode = '67114',
rentzestimate = FALSE,
api_key = zapi_key
)
Error:
Error in GetDeepSearchResults(address = "600 S. Quail Ct.", zipcode = "67114", :
unused arguments (zipcode = "67114", api_key = zapi_key)
Why does this error occur? What can I do to fix this?
Edit: I changed the code according to the comments and got this:
My code:
library(ZillowR)
zapi_key = getOption('myapikey')
GetDeepSearchResults(
address = '600 S. Quail Ct.',
citystatezip = '67114',
rentzestimate = FALSE,
zws_id = 'myapikey',
url = "http://www.zillow.com/webservice/GetDeepSearchResults.htm"
)
Output:
$request
$request$address
NULL
$request$citystatezip
NULL
$message
$message$text
[1] "Error: invalid or missing ZWSID parameter"
$message$code
[1] "2"
$response
NULL
How can I fix this?
The unused arguments error is typical when you pass arguments, which are not parts of the function. So R doesn't know what to do with those and returns the error. You can check the documentation of the function with ?GetDeepSearchResults
This shows you the usage:
GetDeepSearchResults(address = NULL, citystatezip = NULL,
rentzestimate = FALSE, zws_id = getOption("ZillowR-zws_id"),
url = "http://www.zillow.com/webservice/GetDeepSearchResults.htm")
To have this work, you have to set your id first with (you can create an id on https://www.zillow.com/howto/api/APIOverview.htm):
set_zillow_web_service_id("youractualkey")
So you function does not have the argument zipcode and api_key. Let's change your arguments to some which exist:
GetDeepSearchResults(address='600 S. Quail Ct.', citystatezip ='67114',
rentzestimate=FALSE)
You surely recognized I did not use your api_key. This is because the default:zws_id = getOption("ZillowR-zws_id") calls your global 'ZillowR-zws_id' which you just set with the set_zillow_web_service_id() command. So it's not necessary to change the default value. But you can skip this when you use zws_id ="youractualkey" from zillow
I made a random account I set up for validating. This gives me the output:
$request
$request$address
NULL
$request$citystatezip
NULL
$message
$message$text
[1] "Error: this account is not authorized to execute this API call"
$message$code
[1] "6"
$response
NULL
So I could successfully contact the server and my key was recognized. The account authority is not R related and has to be set on the website.