I need to read a source code for a reasearch and I can read the full text when I use a browser, but in R there is a hidden part. The code is replaced by a message saying that the content is allowed just for browsers which use cookies.
Based on the question
How to properly set cookies to get URL content using httr
I am using the following code:
library(httr)
url<-"https://www.ogol.com.br/player_results.php?id=5637"
r <- GET(url, query = list(a = 1))
cookies(r)
response<-GET(url,
set_cookies(`__cfduid` = "dde27d084f28a84488910bf48f22f5fa01530024956",
`FORCE_SITE_VERSION` = "desktop",
`FORCE_MODALIDADE` = "1",
`PHPSESSID` = "uou4jukkosdaafidp26857k8t3"))
player_code<-content(x = response,as = "text", encoding = "ISO-8859-1")
But it also hides a part of the code and returns the message:
"Este conteúdo apenas está disponível para browsers que aceitam cookies" (put the message just to identify if your help has the same result :) )
It means: The content is available just for browsers that accept cookies.
Am I using wrong cookie values or any other clue? Thanks in advance.
Related
Hopefully a quick question, I'm trying to connect to the KuCoin API, not super relevant as I think this is more an issue with how I'm using the POST function and how it sends JSON along
Here is my function that is supposed to place an order:
API.Order <- function(pair,buysell,price,size) {
path = "/api/v1/orders"
now = as.integer(Sys.time()) * 1000
json <- list(
clientOid = as.character(now),
side = buysell,
symbol=pair,
type="limit",
price=price,
size=size
)
json=toJSON(json, auto_unbox = TRUE)
str_to_sign = (paste0(as.character(now), 'POST', path, json))
signature = as.character(base64Encode(hmac(api_secret,str_to_sign,"sha256", raw=TRUE)))
passphrase=as.character(base64Encode(hmac(api_secret,api_passphrase,"sha256", raw=TRUE)))
response=content(POST(url=url,
path=path,
body=json,
encode="json",
config = add_headers("KC-API-SIGN"=signature,
"KC-API-TIMESTAMP"=as.character(now),
"KC-API-KEY"=api_key,
"KC-API-PASSPHRASE"=passphrase,
"KC-API-KEY-VERSION"="2")
),
"text",encoding = "UTF-8")
response
data.table(fromJSON(response)$data)
}
API.Order(pair,"sell",1.42,1.0)
And everything works, except I get the following response:
"{\"code\":\"415000\",\"msg\":\"Unsupported Media Type\"}"
Which is puzzling to me. Everything else checks out (the signature and other auth headers), and I set the encode to "json" in the POST.. I also can put it as standard "application/json" and neither works. I've been staring at this for hours now and I can't see what (likely very little) thing I got wrong?
Thanks
I would like to connect to COINAPI resources. They provide two types of authorization. https://docs.coinapi.io/#authorization
Custom authorization header named X-CoinAPI-Key
Query string parameter named apikey
When I am using the first method, it is working with basic requests. But respond with an error in more advanced.
endpoint<-"/v1/exchangerate/BTC?apikey="
But when I specify endpoint like this:
endpoint <- "/v1/trades/BITSTAMP_SPOT_BTC_USD/history?time_start=2016-01-01T00:00:00/?apikey="
I got error 401.
The second method is not working so far, I do not really understand how can I specify custom header name here.
I need to get data from here:
https://rest.coinapi.io/v1/ohlcv/BTC/USD/history?period_id=1DAY&time_start=2017-01-02T00:00:00.0000000Z&time_end=2019-01-02T00:00:00.0000000Z&limit=10000&include_empty_items=TRUE
I would appreciate any help on this issue.
1. method (working)
library(httr)
library(jsonlite)
base <- "https://rest.coinapi.io"
endpoint <- "/v1/exchangerate/BTC?apikey="
api_key <- <KEY>
call <- paste0(base, endpoint, api_key)
call
get_prices <- GET(call)
http_status(get_prices)
class(get_prices)
get_prices_text <- content(get_prices, "text", encoding = 'UTF-8')
get_prices_json <- fromJSON(get_prices_text, flatten = TRUE)
names(get_prices_json)
get_prices_json$asset_id_base
head(get_prices_json$rates)
data<-as.data.frame(get_prices_json)
2. method (not working)
key<-<KEY>
GET(
url = sprintf("https://rest.coinapi.io/v1/exchangerate/BTC"),
add_headers(`Authorization` = sprintf("X-CoinAPI-Key: ", key))
) -> res
http_status(res)
From reading the examples in the documentation, it looks like it's just looking for a simple header, not an "Authorization" header specifically. Try this
GET(
url = sprintf("https://rest.coinapi.io/v1/exchangerate/BTC"),
add_headers(`X-CoinAPI-Key` = key)
) -> res
http_status(res)
I'm trying to access the US Census Geocoder batch address API, found here: http://geocoding.geo.census.gov/geocoder/
I've also gone through the documentation for the API here: http://geocoding.geo.census.gov/geocoder/Geocoding_Services_API.pdf
I'm trying to use the httr package in R to post a batch csv file of formatted addresses using this format: Unique ID, Street address, City, State, ZIP
I've tried the single address request version using getURL from RCurl and that works fine, but postForm doesn't seem to submit the file in the right way. The code I'm using right now seems to post the request correctly, but I'm not getting any geocoded data back.
curlhandle <- handle("http://geocoding.geo.census.gov/geocoder/geographies/addressbatch",
cookies = TRUE)
# using fileUpload from RCurl instead of upload_file from httr
upload3 <- fileUpload(contents = address100, contentType = "file")
test <- POST(url="http://geocoding.geo.census.gov/geocoder/geographies/addressbatch",
body = list(addressFile = upload3,
benchmark = "Public_AR_Census2010",
vintage="Census2010_Census2010"),
encode = "multipart",
handle = curlhandle,
followLocation = TRUE,
verbose = TRUE)
Is my request missing something? I'm not sure if I should be using writefunction & writedata in this case. Any help would be appreciated!
You seem to have a weird mix of RCurl and httr. Here's how I'd write the request:
req <- POST(
"http://geocoding.geo.census.gov/geocoder/geographies/addressbatch",
body = list(
addressFile = upload_file(address100),
benchmark = "Public_AR_Census2010",
vintage = "Census2010_Census2010"
),
encode = "multipart",
verbose())
stop_for_status(req)
content(req)
(also "file" is not a valid mime type)
I'm trying to use RAmazonS3 to upload a local file to S3 storage, but I keep getting a broken pipe error.
require(RAmazonS3)
options(AmazonS3 = c('xxx' = "xxx")) #login and secret
setwd('[local directory]/reports') #set working directory to location of "polarity.png"
addFile("polarity.png", "umusergen", "destination.png",type="image/png",meta = c(foo = 123, author = "Duncan Temple Lang"))
Send failure: Broken Pipe
It works fine if I just try to upload content
addFile(I("This is a test"), "umusergen", "destination.png",type="text",meta = c(foo = 123, author = "Duncan Temple Lang"))
addFile() is quite simplistic and is focused on text content unless told otherwise.
Use
content = readBin("polarity.png", raw(), file.info("polarity.png")[1, "size"])
and
addFile(content, "umusergen", "destination.png", type = "image/png")
I'll update the addFile() function to allow one to indicate this is binary or content
or to use the (MIME) type.
I want to send an email using arabic text as subject line.
The code piece converts the special characters into arabic text properly for message body but fails to do so for message subject.
I would like to know what I am missing ?
Set objCDOSYS = Server.CreateObject("CDO.Message")
Set objCDOConf = CreateObject("CDO.Configuration")
Set objCDOFields = objCDOConf.Fields
objCDOFields("http://schemas.microsoft.com/cdo/configuration/sendusing") = 2
objCDOFields("http://schemas.microsoft.com/cdo/configuration/smtpserver") = "127.0.0.1"
objCDOFields.Update
Set objCDOSYS.Configuration = objCDOConf
objCDOSYS.MimeFormatted=True
objCDOSYS.BodyPart.Charset = "Windows-1256"
objCDOSYS.From = Trim(Request.Form("frmSender"))
objCDOSYS.To = Trim(Request.Form("frmRecipient"))
objCDOSYS.Subject =Request.Form("frmSubject")
objCDOSYS.HTMLBody = Trim(Request.Form("frmMessage"))
objCDOSYS.HTMLBodyPart.charset = "Windows-1256"
objCDOSYS.Fields.update
objCDOSYS.Send
Set objCDOFields = Nothing
Set objCDOConf = Nothing
Set objCDOSYS = Nothing
Changing to the UTF-8 charset is worth a stab:-
objCDOSYS.HTMLBodyPart.charset = "UTF-8"
I think that will result in the kind of encoding Jirapong was trying but CDOSYS will do it for you. Unfortunately I know that it doesn't work for display names in the email addresses.
You may need to use '=?UTF-8?B?' in front of subject and the Arabic base64 encoded string.
objCDOSYS.Subject = "=?UTF-8?B?" + Base64Encode(Request.Form("frmSubject"))
The Base64Encode function can find at - http://nolovelust.com/post/classic-asp-base64-encoder-decoder.aspx
Note: I did try this myself yet. so not 100% sure.
For me the combination of these 4 items worked:
session.codepage=65001
Response.Charset = "utf-8"
objMessage.HTMLBodyPart.Charset = "utf-8"
objMessage.BodyPart.Charset = "utf-8"