Uploading a file to Figshare using rapiclient swagger api - r

I am trying to programmatically update my Figshare repository using rapiclient.
Following the answer to this question, I managed to authenticate and see my repository by:
library(rapiclient)
library(httr)
# figshare repo id
id = 3761562
fs_api <- get_api("https://docs.figshare.com/swagger.json")
header <- c(Authorization = sprintf("token %s", Sys.getenv("RFIGSHARE_PAT")))
fs_api <- list(operations = get_operations(fs_api, header),
schemas = get_schemas(fs_api))
reply <- fs_api$operations$article_files(id)
I also managed to delete a file using:
fs_api$operations$private_article_file_delete(article_id = id, file_id = F)
Now, I would like to upload a new file to the repository. There seem to be two methods I need:
fs_api$operations$private_article_upload_initiate
fs_api$operations$private_article_upload_complete
But I do not understand the documentation. According to fs_api$operations$private_article_upload_initiate help:
> fs_api$operations$private_article_upload_initiate
private_article_upload_initiate
Initiate Upload
Description:
Initiate new file upload within the article. Either use link to
provide only an existing file that will not be uploaded on figshare
or use the other 3 parameters(md5, name, size)
Parameters:
link (string)
Url for an existing file that will not be uploaded on figshare
md5 (string)
MD5 sum pre computed on the client side
name (string)
File name including the extension; can be omitted only for linked
files.
size (integer)
File size in bytes; can be omitted only for linked files.
What does "file that will not be uploaded on Figshare" mean? How would I use the API to upload a local file ~/foo.txt?
fs_api$operations$private_article_upload_initiate(link='~/foo.txt')
returns HTTP 400.

I feel like I sent you down a bad path with my previous answer because I am not sure how to edit some of the api endpoints when using rapiclient. For example, the corresponding endpoint for fs_api$operations$private_article_upload_initiate() will be https://api.figshare.com/v2/account/articles/{article_id}/files, and I am not sure how to substitute for {article_id} prior to sending the request.
You may have to define your own client for operations you cannot get working any other way.
Here is an example of uploading a file to an existing private article as per the goal of your question.
library(httr)
# id of previously created figshare article
my_article_id <- 99999999
# make example file to upload
my_file <- tempfile("my_file", fileext = ".txt")
writeLines("Hello World!", my_file)
# Step 1 initiate upload
# https://docs.figshare.com/#private_article_upload_initiate
r <- POST(
url = sprintf("https://api.figshare.com/v2/account/articles/%s/files", my_article_id),
add_headers(c(Authorization = sprintf("token %s", Sys.getenv("RFIGSHARE_PAT")))),
body = list(
md5 = tools::md5sum(my_file)[[1]],
name = basename(my_file),
size = file.size(my_file)
),
encode = "json"
)
initiate_upload_response <- content(r)
# Step 2 single file info (get upload url)
# https://docs.figshare.com/#private_article_file
r <- GET(url = initiate_upload_response$location,
add_headers(c(Authorization = sprintf("token %s", Sys.getenv("RFIGSHARE_PAT"))))
)
single_file_response <- content(r)
# Step 3 uploader service (get number of upload parts required)
# https://docs.figshare.com/#endpoints
r <- GET(url = single_file_response$upload_url,
add_headers(c(Authorization = sprintf("token %s", Sys.getenv("RFIGSHARE_PAT"))))
)
upload_service_response <- content(r)
# Step 4 upload parts (this example only has one part)
# https://docs.figshare.com/#endpoints_1
r <- PUT(url = single_file_response$upload_url, path = 1,
add_headers(c(Authorization = sprintf("token %s", Sys.getenv("RFIGSHARE_PAT")))),
body = upload_file(my_file)
)
upload_parts_response <- content(r)
# Step 5 complete upload (after all part uploads are successful)
# https://docs.figshare.com/#private_article_upload_complete
r <- POST(
url = initiate_upload_response$location,
add_headers(c(Authorization = sprintf("token %s", Sys.getenv("RFIGSHARE_PAT"))))
)
complete_upload_response <- content(r)

Related

Converting docx.files to pdf.files with docx2pdf

Not sure what I am doing wrong.
I want to convert multiple docx.files to pdf.files - each file into a separate one.
I decided to use the "doconv"-package with following command:
docx_files <- list.files(pattern=paste0("Protokollnr_"))[39:73]
docx_files %>% length
lapply(1:35, function(x) {
docx2pdf(input = docx_files[[x]],
output = tempfile(fileext = ".pdf"))})
I does not say anything specific in the error message - only that it cannot be converted.
Is it that I should have specified the file path - now I only define the file name in my WD.
The object "docx_files" contain:
c("Protokollnr_1.docx", "Protokollnr_10.docx", "Protokollnr_11.docx",
"Protokollnr_12.docx", "Protokollnr_13.docx", "Protokollnr_14.docx",
"Protokollnr_15.docx", "Protokollnr_16.docx", "Protokollnr_17.docx",
"Protokollnr_18.docx", "Protokollnr_19.docx", "Protokollnr_2.docx",
"Protokollnr_20.docx", "Protokollnr_21.docx", "Protokollnr_22.docx",
"Protokollnr_23.docx", "Protokollnr_24.docx", "Protokollnr_25.docx",
"Protokollnr_26.docx", "Protokollnr_27.docx", "Protokollnr_28.docx",
"Protokollnr_29.docx", "Protokollnr_3.docx", "Protokollnr_30.docx",
"Protokollnr_31.docx", "Protokollnr_32.docx", "Protokollnr_33.docx",
"Protokollnr_34.docx", "Protokollnr_35.docx", "Protokollnr_4.docx",
"Protokollnr_5.docx", "Protokollnr_6.docx", "Protokollnr_7.docx",
"Protokollnr_8.docx", "Protokollnr_9.docx")
The error message is:
Error in docx2pdf(input = docx_files[[x]], output = tempfile(fileext = ".pdf")) :
could not convert C:/Users/Nadine/OneDrive/Documents/Arbeit_Büro_papa/Protokolle_Sallapulka/fertige_Protokolle/Protokollnr_1.docx
Many thanks,
Nadine
I'd recommend specifying the file path since the function requires the following format:
docx2pdf(input, output = gsub("\\.docx$", ".pdf", input))

R retreive information from Eikon cloud

I am trying to get data from Eikon Elektron cloud platform.
I ran following codes from https://github.com/Refinitiv/websocket-api/blob/master/Applications/Examples/R/market_price_authentication.R:
library(curl)
> content = paste("grant_type=", "password","&username=", user, "&password=", password, sep="")
> h <- new_handle(copypostfields = content)
> h
<curl handle> (empty)
> handle_setheaders(h,
+ "Content-Type" = "application/x-www-form-urlencoded"
+ )
> handle_setopt(h, ssl_verifypeer = FALSE, ssl_verifyhost = FALSE)
> auth_url = paste("https://", auth_hostname, sep="")# ":", auth_port, "/getToken", sep="")
> auth_url
[1] "https://api.refinitiv.com/auth/oauth2/v1/token"
> req <- curl_fetch_memory(auth_url, **handle = h**)
> req
$url
[1] "https://api.refinitiv.com/auth/oauth2/v1/token"
$status_code
[1] 400
$type
[1] "application/json"
**> h
<curl handle> (https://api.refinitiv.com/auth/oauth2/v1/token)**
> res_headers = parse_headers(req$headers)
> auth_json_string = rawToChar(req$content)
> auth_json = fromJSON(auth_json_string)
> cat(toJSON(auth_json, pretty=TRUE, auto_unbox=TRUE))
{
"error": "invalid_request"
}
As you can see, I got invalid request error. I think the problem lies in curl_fetch_memory and that the handle=h is using same input as auth_url, however it should use something similiar to the input of content. What can I change in my code to make it work?
I found solution how to access the URL. In github was wrongly written that app_id in R file should equal to the code from the App Key generator in Reuters. However, app_id and client_id are different things and you should add client_id=value from App Key generator (not app_id).+ also do not forget to include trapi and etc.. to your content.

How to import data from a HTML table on a website to excel?

I would like to do some statistical analysis with Python on the live casino game called Crazy Time from Evolution Gaming. There is a website that has the data to do this: https://tracksino.com/crazytime. I want the data of the lowest table 'Spin History' to be imported into excel. However, I do not now how this can be done. Could anyone give me an idea where to start?
Thanks in advance!
Try the below code:
import json
import requests
from urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
import csv
import datetime
def scrap_history():
csv_headers = []
file_path = '' #mention your system where you have to save the file
file_name = 'spin_history.csv' # filename
page_number = 1
while True:
#Dynamic URL fetching data in chunks of 100
url = 'https://api.tracksino.com/crazytime_history?filter=&sort_by=&sort_desc=false&page_num=' + str(page_number) + '&per_page=100&period=24hours'
print('-' * 100)
print('URL created : ',url)
response = requests.get(url,verify=False)
result = json.loads(response.text) # loading data to convert in JSON.
history_data = result['data']
print(history_data)
if history_data != []:
with open(file_path + file_name ,'a+') as history:
#Headers for file
csv_headers = ['Occured At','Slot Result','Spin Result','Total Winners','Total Payout',]
csvwriter = csv.DictWriter(history, delimiter=',', lineterminator='\n',fieldnames=csv_headers)
if page_number == 1:
print('Writing CSV header now...')
csvwriter.writeheader()
#write exracted data in to csv file one by one
for item in history_data:
value = datetime.datetime.fromtimestamp(item['when'])
occured_at = f'{value:%d-%B-%Y # %H:%M:%S}'
csvwriter.writerow({'Occured At':occured_at,
'Slot Result': item['slot_result'],
'Spin Result': item['result'],
'Total Winners': item['total_winners'],
'Total Payout': item['total_payout'],
})
print('-' * 100)
page_number +=1
print(page_number)
print('-' * 100)
else:
break
Explanation:
I have implemented the above script using python requests way. The API url https://api.tracksino.com/crazytime_history?filter=&sort_by=&sort_desc=false&page_num=1&per_page=50&period=24hours extarcted from the web site itself(refer screenshot). In the very first step script will take the dynamic URL where page number is dynamic and changed upon on every iteration. For ex:- first it will be page_num = 1 then page_num = 2 and so on till all the data will get extracted.

MturkR, R, automatically posting microbatches

I am new to MTurk, but have some fluency with R. I am using the MTurkR package the first time, and I am trying to create "micro-batches" that post on MTurk over time. The code I am using can be found below (the XXXX parts are obviously filled with the correct values). I don't get any error messages, the code runs, and posts the HIT both in the Sandbox and in the real correctly. However, the HITs posted do not show up in the Sandbox Requester account, or the real requester account which means - as far as I understand - that I can't evaluate the workers who submit a completion code before they are paid automatically.
Could anyone point out where the error is, and how could I review the HITs?
Thanks.
K
##### Notes:
# 1) Change sandbox from TRUE to FALSE to run live (make sure to test in sandbox first!!)
##### Step 1: Load library, set parameters
#### Load MTurkR library
library(MTurkR)
#### HIT Layout ID
# Layout ID for the choice task
# my_hitlayoutid = "XXXX"
# Layout ID for the choice task in the Sandbox
my_hitlayoutid = "XXXX"
#### Set MTurk credentials
Sys.setenv(
AWS_ACCESS_KEY_ID = "XXXX",
AWS_SECRET_ACCESS_KEY = "XXXX"
)
#### HIT parameters
## Run in sandbox?
sandbox_val <- "FALSE"
## Set the name of your project here (used to retrieve HITs later)
myannotation <- "myannotation"
## Enter other HIT aspects
newhittype <- RegisterHITType(
title = "hope trial",
description = "Description",
reward = "2.5",
duration = seconds(hours = 1),
keywords = "survey, demographics, neighborhoods, employment",
sandbox = sandbox_val
)
##### Step 2: Define functions
## Define a function that will create a HIT using information above
createhit <- function() {
CreateHIT(
hit.type = newhittype$HITTypeId,
assignments = 2,
expiration = seconds(days = 30),
annotation = myannotation,
verbose = TRUE,
sandbox = sandbox_val,
hitlayoutid = my_hitlayoutid
)
}
## Define a function that will expire all running HITs
## This keeps HITs from "piling up" on a slow day
## It ensures that A) HIT appears at the top of the list, B) workers won't accidentally accept HIT twice
# expirehits <- function() {
# ExpireHIT(
# annotation = myannotation,
# sandbox = sandbox_val
# )
#}
##### Step 3: Execute a loop that runs createhit/expirehit functions every hour, and it will log the output to a file
## Define number of times to post the HIT (totalruns)
totalruns <- 2
counter <- 0
## Define log file (change the location as appropriate)
logfile <- file("/Users/kinga.makovi/Dropbox/Bias_Experiment/MTurk/logfile.txt", open="a")
sink(logfile, append=TRUE, type="message")
## Run loop (note: interval is hourly, but can be changed in Sys.sleep)
repeat {
message(Sys.time())
createhit()
Sys.sleep(10)
#expirehits()
counter = counter + 1
if (counter == totalruns){
break
}
}
## To stop the loop before it finishes, click the "STOP" button
## To stop logging, run sink()
You can't see HITs that are created via the API (through MTurkR or otherwise) in the requester website. It's a "feature". You'll have to access the HITs through MTurkR (e.g., SearchHITs() and GetHIT()).

Authenticating API in R

I'm trying to access Bitfinex via api but struggling to properly authenticate my request. There is a Python example of what I want to do (https://gist.github.com/jordanbaucke/5812039) but I can't seem to get it to work in R.
key <- c("MY API KEY")
secret = c("MY API SECRET")
req <- GET("https://api.bitfinex.com/v1/balances",
authenticate(key, secret))
add_headers(X-BFX-APIKEY = key))
stop_for_status(req)
content(req)
Can someone tell me what I'm doing wrong?
/v1/balances is a Bitfinex authenticated endpoints, thus it requires the POST request with a proper handling of payload and headers.
Here is a working example from my own script:
library(httr)
key <- "..."
secret <- "..."
# payload JSON object, the request should refer to the URL
# nonce should always be greater than for previous calls
payload_json <- list(request = '/v1/account_infos', nonce = as.character(as.numeric(as.POSIXct(Sys.time()))))
# creating string from JSON payload
payload <- jsonlite::toJSON(payload_json, auto_unbox = TRUE)
# encoding payload string
payload_str <- base64enc::base64encode(charToRaw(as.character(payload)))
# adding three Bitfinex headers:
# X-BFX-APIKEY = key
# X-BFX-PAYLOAD = base64 encoded payload string
# X-BFX-SIGNATURE = sha384 encrypted payload string with the secret key
req <- POST("https://api.bitfinex.com/v1/account_infos",
add_headers('X-BFX-APIKEY' = key,
'X-BFX-PAYLOAD' = payload_str,
'X-BFX-SIGNATURE' = openssl::sha384(payload_str, key = secret))
)
content(req)
For creating "New Order" you only need to change the payload to something, like this:
payload_json <- list(request = '/v1/account_infos',
nonce = as.character(as.numeric(as.POSIXct(Sys.time()))),
symbol = 'BTCUSD',
amount = '0.3',
price = '1000',
exchange = 'bitfinex',
side = 'sell',
type = 'exchange market'
)
The rest of the code will work without changes.
For list of payload parameters check the Bitfinex API docs, e.g. "New Order".

Resources