I'm trying to use R to grab some web data that requires OAuth authentication. Searching on CRAN and RSeek.org for info on OAuth + R provides nothing. Any tips for accessing RESTful APIs with R using OAuth?
I'm considering using some Python/Perl/Ruby to grab the data, save it to a text file, then work on it with R. I'd prefer to stay totally in R, but it seems like OAuth is a barrier.
Turns out Jeff Gentry from TwitteR wrote his own OAuth R library.
http://cran.r-project.org/web/packages/ROAuth/index.html
Best approach now appears to be to use httr (https://github.com/r-lib/httr)
One option I stumbled on is called OAuth Proxy and it slips the OAuth headers in using a proxy. This is an interesting kludge.
Another possible, assuming you have access to the required crypto functions in R, you can implement OAuth completely as HTTP requests without any lib support etc.
While writing a complete oauth lib might be asking too much, if you just need a quick and dirty way to pull down data, it shouldn't be too difficult to hand craft the few functions you need to complete an oauth request.
Also, while R may not have an oauth lib, most of the difficult part of oauth is in the interaction between servers, not in creating the tokens/requests, so you should be able to look at another language's implementation and port the required logic to R. I would suggest looking at a simple lib from http://oauth.net/code/ for a starting point.
It's possible to do this with the RCurl library as well. The main thing is that you parse the access_tokens and authorization verifications correctly. But I've managed to pull lots of data off of facebook with R via getURL.
Related
I'm thinking about creating an API with Rscript doing all my stuff.
But, we know that R is single-thread, and plumber also is.
Someone know a way to create an API multi thread in R? I dont think that an single-thread can help me in my case, I may have many users using my model in prod, then im afraid to use plumber.
I already see people saying about using RServer and Java to create an API multi thread. (but i dont know how do this).
Any suggestion, or links about this discussion is welcome.
Thank you all!
Just to document things that are possible to other people that may have the same question, and dont know how do this, I will put here some links that i found that can be helpful.
I dont test any idea yet to say which one is faster, or cheaper, or optimized...
But, what i found is:
You can use https://restrserve.org/ that is an alternative to plumber, even when plumber was just single-thread, restrserve already was helpful to create APIs multi-thead.
8 days ago, plumber release the version 1.0.0, that can support APIs multi-thread. Link to release: https://www.rplumber.io/news/index.html#plumber-router
Even in the past, without plumber v1.0.0, or if you dont want use the RestRserve, you could create an API single-thread with the plumber (in the past), use the Docker to Build your API and then use Kubernetes to manage the requests, creating "copys" of your API, and then the Kubernetes manage the requests, choosing the copy_API that will be used.
An talk at RSTUDIO::CONF2020 about an model that has 1.000.000 acces per day, with an R API: https://rstudio.com/resources/rstudioconf-2020/we-re-hitting-r-a-million-times-a-day-so-we-made-a-talk-about-it/
I've developed a REST API back end using Endpoints-Proto-Datastore, which wraps the Cloud Endpoints Python API. I'm starting to look at Qt and trying to get an idea what will be involved in accessing my API from the Qt networking or other library. Might it be nearly as straightforward as is making the calls from the command line using the Python Client library, which even handles OAuth2 flows? This would be very nice. I might use PyQt if this makes things simpler.
Your Endpoints service can generate an OpenAPI specification file which describes the API. Once you do this, there are many OpenAPI-compatible packages which can generate client code for you.
I located this document which gives a pretty good overview for my purposes:
"The Google APIs Client Library for C++ will automatically take care of many of the tedious details for interpreting and complying with the discovery documents so that you can write simpler and familiar C++ code."
Now it's a matter of building and installing the C++ client and then figuring out how to generate the client library and access it from a Qt application. But that is beyond the scope of this question.
I'm pretty new to Apigee so apologies in advanced. We currently have an api which is accessed an Apigee proxy. From reading the Apigee best practices it looks like we should version the api in the URL like so
api/v1/endpoint/
How do I go about stripping the version out of the URL and using it to target the correct api?
Thanks
Rather than answering this with more theoretical content - I would suggest you try out the first sample - Add and configure your first API and you will be on your way to build your production API with good grip on how to use Apigee.
How do I go about stripping the version out of the URL and using it to target the correct api?
If you choose your base path to be "api/v1/endpoint/" like so:
<HTTPProxyConnection>
<BasePath>/api/v1/endpoint</BasePath>
<VirtualHost>default</VirtualHost>
</HTTPProxyConnection>
then only the part of the URL after /api/v1/endpoint will be sent to the target. In this way, you avoid passing along the version portion of the URL to your target.
From the question here, There are two external libraries to use for http operation. It seems that dispatch has more visibility while scalaj-http is easy to use as stated there. Thus, I am more inclined toward scalaj-http. I want to use the http library in google app engine, where there are restraints. For standard Java, there is a work around for it from here. I would like to get advice on what would be the best approach to use Scala in Google app engine(this is not for Lift framework).
I personally am very happy with Dispatch. There are several executors, including one for App Engine, dispatch-gae.
I'm writing a web app in Scala using the Play framework. I'd like to be able to push some binary data to my web server from another machine I'm using to do number crunching. I'd like to do this over http. Can anyone suggest the best way to do each side? Ideas that have occurred to me so far are:
Send the data up as a file upload via the usual play form processing. Nice on the (web) server side, but I'm not sure what libraries to use for pushing the data up from the (number crunching) client. In C/C++ I'd consider using Curl.
Send the data up as raw POST with the binary attached and encoded appropriately. Not sure how to do either side.
I've done each of the above on several occasions in Python and C++ (although not recently enough to remember how!), but am not a web dev (but a more general sw engineer) and have only ever had control of one side before - so have no idea what the best way to do this is.
Any thoughts appreciated.
Alex
It depends what platform (and language) you're already using for the number-crunching client part. If that 'client' is also using the Play framework (or at least has access to the libraries), then there are some very helpful tools for accessing web services; (see here also).