Access the Betfair Exchange API using Julia
I've been using Julia for about 2mths now, and have recently been trying to use Julia to access the Betfair API.
Note about this service are here.
https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Getting+Started
Whilst I can get the Python example working (& I have an appKey & sessionToken though not shown), I've not been able to successfully translate this Python into Julia.
In example below I get a StatusError 400 response (which is the closest I've gotten). Other attempts indicated Bound issues probably from the Python example using {} and ' which Ive attempted to then translate.
I've looked at other Stackflow questions, but found they don't have the complexity associated with this example.
Wondering if anyone has any thoughts.
Thanks in advance
using HTTP
url="https://api.betfair.com/exchange/betting/json-rpc/v1"
header = "\"X-Application\" : \"appKey\", \"X-Authentication\" : \"sessionToken\" ,\"content-type\" : \"application/json\" "
jsonrpc_req="\"jsonrpc\": \"2.0\", \"method\": \"SportsAPING/v1.0/listEventTypes\", \"params\": {\"filter\":{ }}, \"id\": 1"
response = HTTP.post(url, data=[jsonrpc_req], headers=[header])
println(response.text)
Expected Results.
In Python, I get a summary of Betfair Sports and Market's.
{"jsonrpc":"2.0","result":[{"eventType":{"id":"1","name":"Soccer"},"marketCount":10668},{"eventType":{"id":"2","name":"Tennis"},"marketCount":4590},{"eventType":{"id":"3","name":"Golf"},"marketCount":43},{"eventType":{"id":"4","name":"Cricket"},"marketCount":394},{"eventType":{"id":"5","name":"Rugby Union"},"marketCount":37},{"eventType":{"id":"1477","name":"Rugby League"},"marketCount":24},{"eventType":{"id":"6","name":"Boxing"},"marketCount":27},{"eventType"
...etc...
Currently get
HTTP.ExceptionRequest.StatusError(400, HTTP.Messages.Response:
400 Bad Request.
While the interaction with a particular REST service is a problem-specific issue here are the general guidelines.
Firstly, you need to properly format headers - HTTP.jl manual reads: "headers can be any collection where [string(k) => string(v) for (k,v) in headers] yields Vector{Pair}."
Since we do not have Betfair API key let's have a look on a more generic example using https://postman-echo.com/ which is a free simple API testing that simply returns as JSON whatever it gets as the input.
using HTTP
using JSON
headers = (("X-Application","appKey"),("X-Authentication","sessionToken"),
("content-type","application/json"))
url="https://postman-echo.com/post"
req = Dict("jsonrpc" => "2.0", "params" => Dict("filet" => Dict()))
response = HTTP.post(url, headers, JSON.json(req))
response_text = String(response.body)
json_obj = JSON.parse()
Now let us parse the output from postman-echo.com:
julia> display(JSON.parse(response_text))
Dict{String,Any} with 7 entries:
"headers" => Dict{String,Any}("x-forwarded-port"=>"443","host"=>"postman-echo.com","x-application"=>"appKey","content-type"… "json" => Dict{String,Any}("params"=>Dict{String,Any}("filet"=>Dict{String,Any}()),"jsonrpc"=>"2.0")
"files" => Dict{String,Any}()
"args" => Dict{String,Any}()
"data" => Dict{String,Any}("params"=>Dict{String,Any}("filet"=>Dict{String,Any}()),"jsonrpc"=>"2.0")
"url" => "https://postman-echo.com/post"
"form" => Dict{String,Any}()
You can easily adopt the above code to any RESTful JSON API.
Thanks Przemyslaw Szufel for your response. After a few more days of frustration, I managed to get the first part of the API working using the Excel/VBA sample here: https://github.com/betfair/API-NG-Excel-Toolkit (my translations of the Python examples did not work).
Your comment helped in terms of understanding how to structure multiple headers, and using Dict( =>) rather than string for the manipulations I attempted above.
using HTTP
using JSON
const ListEventTypesMethod = "listEventTypes"
const AppKey = "appKey"
const Session = "sessionToken"
function SendRequest(Url, AppKey, Session, Data)
headers = (("X-Application", AppKey),
("content-type", "application/json"),
("Accept", "application/json"),
("X-Authentication", Session))
HTTP.get(Url,headers,JSON.json(Data))
end
function ParseJsonRpcResponseToCollection(Response)
ParseJsonRpcResponseToCollection = JSON.parse(Response)
end
function GetJsonRpcUrl()
GetJsonRpcUrl = "https://api.betfair.com/exchange/betting/json-rpc/v1/"
end
function MakeJsonRpcRequestString(Method, RequestString)
#MakeJsonRpcRequestString = "{""jsonrpc"": ""2.0"", ""method"": ""SportsAPING/v1.0/" & Method & """, ""params"": " & RequestString & ", ""id"": 1}"
MakeJsonRpcRequestString = Dict("jsonrpc" => "2.0", "method" =>"SportsAPING/v1.0/"*Method, "params" => RequestString, "id"=> 1 )
end
function GetListEventTypesRequestString()
#GetListEventTypesRequestString = "{""filter"":{}}"
GetListEventTypesRequestString=Dict("filter" =>Dict())
end
Request = MakeJsonRpcRequestString(ListEventTypesMethod, GetListEventTypesRequestString())
ListEventTypesResponse = SendRequest(GetJsonRpcUrl(), AppKey, Session, Request)
Response
HTTP/1.1 200 OK
Date: Sun, 17 Feb 2019 17:28:08 GMT
Server: Cougar - 4.4.2 (Cougar 2 mode)
Cache-Control: no-cache
Content-Type: application/json
Vary: Accept-Encoding, User-Agent
Content-Length: 1850
{"jsonrpc":"2.0","result":[{"eventType":{"id":"1","name":"Soccer"},"marketCount":6553},{"eventType":{"id":"2","name":"Tennis"},"marketCount":5511},{"eventType":{"id":"3","name":"Golf"},"marketCount":34}
etc...
Hope this helps others as well.
I have a controller action that returns a CSV document produced asynchronously by a data source. I subscribed to an observable data stream and I write to the http response with Response.WriteAsync(). It seems to work apart of the fact that the terminating (empty) chunk is missing at the end of the response (The missing data in hex is 30 0d 0a 0d 0a for the terminating 0\r\n\r\n).
I have few questions:
- is it okay to use WriteAsync in this scenario - call it multiple times as the data comes in - i certainly do not want to buffer the whole dataset and return it at once as it can grow huge
- whose responsibility it is to write the terminating chunk (should it write it explicitly in OnComplete handler with WriteAsync as well?)
example response from an action that returns a json object (note the terminator at the end of the response):
HTTP/1.1 200 OK
Date: Tue, 05 Feb 2019 09:25:23 GMT
Server: Kestrel
Transfer-Encoding: chunked
21
{"results":[{"statement_id":0}]}
0
Response from my action:
HTTP/1.1 200 OK
Date: Tue, 05 Feb 2019 09:27:42 GMT
Content-Type: text/csv
Server: Kestrel
Transfer-Encoding: chunked
47
ID,Timestamp,BadQualityCount,MAX_value,MEAN_value,MIN_value,TotalPoints
65
...
5d
SwitchStatus_On_a797c2c2-de78-4fe8-9e4b-5d64f51d1e00,2019-0204T11:52:17.3680000Z,0,0,0,0,1
Controller code:
using (var disposable = observableContent.Subscribe(
async current =>
await Response.WriteAsync(current),
() =>
{
//Response.WriteAsync(Environment.NewLine);
//Response.WriteAsync("0");
Response.Body.Flush();
completed = true;
}
))
SpinWait.SpinUntil(() => completed);
return Ok();
It turned out the terminator is written correctly if the action signature is changed from Task to Task.
I'm trying to write rspec tests. Some of them have to stub calls to external service.
Some of these calls send nested data, and this data never seems to be processed by Webmock correctly.
describe 'calc' do
before do
stub_request(:any, url).with(body: hash_including(operation: 'calc'))
end
it 'works' do
request_data = { sale_type: 'money', cover_type: 'money',
region: 'rf', period: 9,
adults: 600, children: 750, mice: 500 }
# This thing makes a HTTP request:
MiteService.new(login_params).calc(request_data)
expected_body = { operation: 'calc', product: 'mite3',
sessionID: session, data: request_data }
expect(WebMock).to have_requested(:post, url).with(body: expected_body)
end
end
This test is expected to pass. The http call made by service seems to be correct, but Webmock is unable to read any nested data (which is in data part of body in this case).
1) MiteService API calls calc works
Failure/Error: expect(WebMock).to have_requested(:post, url).with(body: expected_body)
The request POST http://example.com/api with body {"data"=>{"sale_type"=>"money", "cover_type"=>"money", "region"=>"rf", "period"=>9, "adults"=>600, "children"=>750, "mice"=>500}, "operation"=>"calc", "product"=>"mite3", "sessionID"=>"123"} was expected to execute 1 time but it executed 0 times
The following requests were made:
<...>
POST http://example.com/api with body 'operation=calc&product=mite3&sessionID=123&data=%7B%3Asale_type%3D%3E%22money%22%2C+%3Acover_type%3D%3E%22money%22%2C+%3Aregion%3D%3E%22rf%22%2C+%3Aperiod%3D%3E9%2C+%3Aadults%3D%3E600%2C+%3Achildren%3D%3E750%2C+%3Amice%3D%3E500%7D' with headers {'Accept'=>'*/*', 'Content-Type'=>'application/x-www-form-urlencoded', 'Date'=>'Tue, 25 Dec 2018 08:20:32 GMT', 'User-Agent'=>'HTTPClient/1.0 (2.8.3, ruby 2.4.1 (2017-03-22))'} was made 1 time
Made it work for this example by converting fields into json in example spec.
expect(WebMock).to have_requested(:post, url).with(body: expected_body.to_json)
This is my first day to have fun with Symfony and drupal 8, so please excuse me if my question is very obvious.
With drupal 7:
drupal_json_output(array('products' => array_values($products)));
exit;
the json output is clean:
{"products":["item_1","item_2",....]}
With drupal 8:
use Symfony\Component\HttpFoundation\JsonResponse;
// some process
print new JsonResponse(array('products' => array_values($products)));
exit;
It outputs with the headers:
HTTP/1.0 200 OK
Cache-Control: no-cache
Content-Type: application/json
Date: Wed, 18 Jul 2012 07:53:26 GMT
{"products":["item_1","item_2",....]}
How do you get rid of those headers?
I am stuck to read the reference here.
Any hint is very much appreciated.
You can get only the "content" of a response by calling $response->getContent().
In your case you could do
use Symfony\Component\HttpFoundation\JsonResponse;
// some process
$response = new JsonResponse(array('products' => array_values($products)));
print $response->getContent();
exit;
However, be aware that this would be a bad practice because you would lose the response headers in the process, and wouldn't tell for example, what the content-type of you response is (in this case: "application/json") etc ...
I do not know how to do this properly with drupal, any tips is appreciated.
I am trying to convert my node.js HTTP server to Go. Here's what I want to happen:
I have a resource that gets generated intermittently (say every second or so), and I want all requests for this resource to wait until the next time it is generated. This way clients can poll and be guaranteed to get only the up-to-date resource. I am using web.go to remove a lot of the complexity of running an HTTP server.
Here is a brief version of my code:
package main
import (
"time"
"web"
"fmt"
vector "container/vector"
)
var listeners vector.Vector;
func resource (ctx *web.Context) {
c := make(chan int)
listeners.Push(c)
go func() {
<-c
go func() {
ctx.WriteString("Cool")
fmt.Println("Wrote something")
}()
}()
}
func resourceLoop() {
time.Sleep(5 * 1000 * 1000000) // sleep for 5 seconds
for ; listeners.Len() > 0 ; {
c := listeners.Pop().(chan int)
c <- 1
}
go resourceLoop()
}
func main() {
web.Get("/resource", resource)
go resourceLoop()
web.Run("localhost:4000")
}
I would expect there to be a Context.End() or similar function, but it doesn't seem to have one. I read the source for web.go, but I couldn't figure out where it was ending the response (web.go:268 is where my resource() is called). In node.js this is trivial, you can call a ServerResponse.end().
When I kill the server while running the script in Chrome, I get this output (seems to be correct, except that the response isn't ending):
4
Cool
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Date: Sat, 16 Apr 2011 00:37:58 GMT
Server: web.go
Transfer-Encoding: chunked
0
Is this a problem with the web.go framework or am I doing something wrong? If it's a problem with the framework, I'll file an issue with him.
I'm pretty new to Go, so I could be going about this completely wrong.
I've never used web.go, but it seems that your example is entirely too complicated. Why do you need a goroutine to spawn a goroutine? I would assume the framework itself would take care of concurrency and just write this:
func resource (ctx *web.Context, val string) string {
c := make(chan int)
listeners.Push(c)
<-c
return "Cool"
}
Otherwise, it looks like it is doing exactly what you want and you just need to close the connection if you're truly done with it:
ctx.Close()