Google OAuth 2.0 Server to Server: Bad request - plsql

I've been banging my head against a wall for 3 days now trying to get this to work.
POST /oauth2/v3/token HTTP/1.1
Host: www.googleapis.com
Content-length: 495
Content-type: application/x-www-form-urlencoded
Authorization: Bearer ya29.cgEcY6meBrvaH6oe0nD_PtsFyMVqskiUYi7iJxapKHeEgPoIw8gMt0BJdIvRn1MfcEgzTS3_gTwI1w
grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiI5MDgyOTgxNjA1NTktc2R1bGFpbWhsaGpxOTY5M2s1Z2E4c25pZjhh%0D%0ANzhlZ3BAZGV2ZWxvcGVyLmdzZXJ2aWNlYWNjb3VudC5jb20iLCJzY29wZSI6Imh0%0D%0AdHBzOi8vd3d3Lmdvb2dsZWFwaXMuY29tL2F1dGgvYW5hbHl0aWNzLnJlYWRvbmx5%0D%0AIiwiYXVkIjoiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tL28vb2F1dGgyL3Rv%0D%0Aa2VuIiwiZXhwIjoxNDMxNTE0MDUyLCJpYXQiOjE0MzE1MTEwNTJ9.[Cert]
HTTP/1.1 400 Bad Request
Content-length: 67
X-xss-protection: 1; mode=block
X-content-type-options: nosniff
Expires: Wed, 13 May 2015 10:08:00 GMT
Vary: Origin,X-Origin
Server: GSE
Cache-control: private, max-age=0
Date: Wed, 13 May 2015 10:08:00 GMT
X-frame-options: SAMEORIGIN
Content-type: application/json; charset=UTF-8
{
"error_description": "Bad Request",
"error": "invalid_grant"
}
So, I'm creating this in PL/SQL. Oracle 12c. I've managed to get the JWT header and and JWT Claim set to produce an output the same as the google documentation. When i create the cert I think is were the problem occurs.
Do I need the "[-----BEGIN PRIVATE KEY-----" and -----END PRIVATE KEY-----\n] as part of the SHA-256 encryption. Should I do anything with the "New lines"? \n Should I surround it with brackets?
At what point should I be URL encoding?
Is the code in the following example sufficient for the encryption?: http://jastraub.blogspot.co.uk/2009/07/hmacsha256-in-plsql.html
I've attached the a function below to see if you can identify any issues?
Thanks for your help!
FUNCTION get_JWT (p_token_id ga_app_user.ID_TOKEN%TYPE)
RETURN VARCHAR2
IS
--Plain text
baseJWTheader VARCHAR2 (20000);
baseclaimSet VARCHAR2 (20000);
baseSigKey VARCHAR2 (20000);
--Seconds
sysSeconds NUMBER;
--Base64 Encoded
JWTheader VARCHAR2 (20000);
claimSet VARCHAR2 (20000);
sigKey VARCHAR2 (20000);
sigContent VARCHAR2 (20000);
--Returned value
output RAW (20000);
BEGIN
SELECT JWT_HEADER, JWT_CLAIM_SET, SIGNATURE
INTO baseJWTheader, baseclaimSet, baseSigKey
FROM dwman.ga_app_user au
WHERE AU.ID_TOKEN = p_token_id;
--DBMS_OUTPUT.PUT_LINE ('Base claim Set ' || baseclaimSet);
JWTheader :=
TRANSLATE (
UTL_RAW.cast_to_varchar2 (
UTL_ENCODE.BASE64_ENCODE (UTL_RAW.CAST_TO_RAW (baseJWTheader))),
'+/',
'-_');
SELECT ( SYSDATE
- TO_DATE ('01-01-1970 00:00:00', 'DD-MM-YYYY HH24:MI:SS'))
* 24
* 60
* 60
INTO sysSeconds
FROM DUAL;
baseclaimSet :=
REPLACE (baseclaimSet, '#EXPIRE#', ROUND (sysSeconds + 3000));
baseclaimSet := REPLACE (baseclaimSet, '#START#', ROUND (sysSeconds));
--DBMS_OUTPUT.PUT_LINE ('Claim Set ' || baseclaimSet);
claimSet := UTL_RAW.cast_to_varchar2 (
UTL_ENCODE.BASE64_ENCODE (UTL_RAW.CAST_TO_RAW (baseclaimSet)));
sigKey := baseSigKey;
sigContent := JWTheader || '.' || claimSet;
--DBMS_OUTPUT.PUT_LINE('Sig Content '||sigContent);
sigContent := REPLACE (sigContent, CHR (10), '');
sigContent := REPLACE (sigContent, CHR (13), '');
/*
FOR V_TR in 1..length(sigContent)
LOOP
DBMS_OUTPUT.PUT_LINE (substr(sigContent,V_TR,1)||'='||to_char(ASCII(substr(sigContent,V_TR,1))));
END LOOP;
*/
sigContent :=
sigContent
|| '.'
|| google_signature (sigContent, sigKey);
RETURN UTL_URL.ESCAPE(sigContent, TRUE, 'UTF-8');
END get_JWT;

Your code snippet might only be a portion of what you did, but it seemed to be lacking much of the OAUTH steps required by Google to connect.
You can get more detail on what those steps are by looking at this URL:
https://developers.google.com/accounts/docs/OAuth2WebServer#offline
For the remainder of this answer, I have described my own experiences, doing something similar (downloading data from GA, and uploading it to a database using SQL Statements).
Start by getting a consumerKey and consumerSecret for your Google project. You'll need to have a URL that Google can redirect to, both when you request your consumer Key, but also to supply to Google during your OAuth calls. They have to match.
The next step is to send a GET request to Google. Here is an example in c# which you can build using SQL string concatenation.
String URL_AUTH_FIRST = "https://accounts.google.com/o/oauth2/auth";
String URL_TOKEN_ENDPOINT_SECOND = "https://accounts.google.com/o/oauth2/token";
String url = String.Format(
"{0}?client_id={1}&redirect_uri={2}&access_type=offline&scope={3}&response_type=code&state={4}&approval_prompt=force",
URL_AUTH_FIRST,_consumerKey_web_app,redir_url,scope,state);
You will need to have an embedded browser to do this. Google.com will redirect that browser to a site that's under their control so that the user has to be able to login or refuse to authorize your app. After Google has the information they need, they redirect back to your embedded browser. You can do some of the steps using copy/paste in your own browser, but at some point (described below) you have to POST back some data which I wouldn't know if you can do that from a browser application.
Google will respond by redirecting your embedded browser to a url. The URL has data on it. You have to parse the parameters on the URL and look for the parameter "code". If you get a url with "code" as a parameter, you have to POST a format in a certain format back to Google.
WebClient client = get_WebClient(); // proprietary to include things like proxy info
try {
NameValueCollection values = new NameValueCollection();
values.Add("client_id", _consumerKey_web_app);
values.Add("client_secret", _consumerSecret_web_app_offline);
values.Add("grant_type", "authorization_code");
values.Add("redirect_uri", URL_GOOGLE_REDIRECTS_TO_THIS_URL_AFTER_URL_AUTH);
values.Add("code", authorization_code);
Byte[] responseBytes = client.UploadValues(URL_TOKEN_ENDPOINT_SECOND, values);
}
Google will return "responseBytes" which would be a json formatted string, resembling:
{
"access_token":"1/fFAGRNJru1FTz70BzhT3Zg",
"expires_in":3920,
"token_type":"Bearer",
"refresh_token":"1/6BMfW9j53gdGImsixUH6kU5RsR4zwI9lUVX-tqf8JXQ"
}
The access_token gets appended to your REST API calls.
You can supply a GA Query via REST API, get the data back, and upload it into your database using SQL statements. That's what my App does.
You can save that refresh_token and supply it in future connections. Indeed, this whole sequence needs to be done with a browser or browser control, interactively with a user logging in. After it's done, and you've received the refresh_token, then your SQL can use and re-use the refresh_token basically in-definitely, at least until the user's password changes.
Google will also return an error 401 on a regular basis. This just means you have to re-request your access token by Posting a new set of values to Google:
NameValueCollection values = new NameValueCollection();
values.Add("client_id", _consumerKey_web_app);
values.Add("client_secret", _consumerSecret_web_app_offline);
values.Add("refresh_token", refresh_token);
values.Add("grant_type", "refresh_token");
Byte[] responseBytes = client.UploadValues(URL_TOKEN_ENDPOINT_SECOND, values);

Related

Uploading a bitmap with idHTTP from a Delphi 10.3 multidevice app

I have read many related posts about sending data with idHTTP but still I can't manage it.
I use this code :
updated
procedure TTabbedForm.SpeedButton1Click(Sender: TObject);
var
fName : string;
mStream : TMemoryStream;
begin
fName := 'image.jpg';
mStream := TMemoryStream.Create;
myImage.Bitmap.SaveToStream(mStream);
mStream.Position := 0;
try
IdHTTP1.Request.ContentType := 'application/octet-stream';
IdHTTP1.PUT('http://www.example.com/'+fName, mStream);
finally
mStream.free;
end;
end;
but i receive the error "Method not allowed".
What i'm doing wrong, please ?
For uploads to Google Drive, some additional steps are required. For example, the HTTP POST request must include a auth token which in turn is provided to you only after authentication (log in with a Google account). For Google Drive you must also use secure connections (https) which require SSL libraries such as OpenSSL.
Example from the API docs:
POST https://www.googleapis.com/upload/drive/v3/files?uploadType=media HTTP/1.1
Content-Type: image/jpeg
Content-Length: [NUMBER_OF_BYTES_IN_FILE]
Authorization: Bearer [YOUR_AUTH_TOKEN]
[JPEG_DATA]
The file simple upload API for Google Drive is documented here:
https://developers.google.com/drive/api/v3/simple-upload
Update
Try this example, it requires a valid auth token:
procedure TDriveAPITest.Run;
var
PostData: TStream;
Response: string;
begin
PostData := TFileStream.Create('test.png', fmOpenRead or fmShareDenyWrite);
try
IdHTTP := TIdHTTP.Create;
try
IdHTTP.HTTPOptions := IdHTTP.HTTPOptions + [hoNoProtocolErrorException];
IdHTTP.Request.CustomHeaders.Values['Authorization'] := 'Bearer [YOUR_AUTH_TOKEN]';
Response := IdHTTP.Post('https://www.googleapis.com/upload/drive/v3/files?uploadType=media', PostData);
if IdHTTP.ResponseCode = 200 then begin
WriteLn('Response: ' + Response);
end else begin
WriteLn('Error: ' + IdHTTP.ResponseText);
end;
finally
IdHTTP.Free;
end;
finally
PostData.Free;
end;
end;
Output:
Error: HTTP/1.0 401 Unauthorized

How to access an API via Julia HTTP

Access the Betfair Exchange API using Julia
I've been using Julia for about 2mths now, and have recently been trying to use Julia to access the Betfair API.
Note about this service are here.
https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Getting+Started
Whilst I can get the Python example working (& I have an appKey & sessionToken though not shown), I've not been able to successfully translate this Python into Julia.
In example below I get a StatusError 400 response (which is the closest I've gotten). Other attempts indicated Bound issues probably from the Python example using {} and ' which Ive attempted to then translate.
I've looked at other Stackflow questions, but found they don't have the complexity associated with this example.
Wondering if anyone has any thoughts.
Thanks in advance
using HTTP
url="https://api.betfair.com/exchange/betting/json-rpc/v1"
header = "\"X-Application\" : \"appKey\", \"X-Authentication\" : \"sessionToken\" ,\"content-type\" : \"application/json\" "
jsonrpc_req="\"jsonrpc\": \"2.0\", \"method\": \"SportsAPING/v1.0/listEventTypes\", \"params\": {\"filter\":{ }}, \"id\": 1"
response = HTTP.post(url, data=[jsonrpc_req], headers=[header])
println(response.text)
Expected Results.
In Python, I get a summary of Betfair Sports and Market's.
{"jsonrpc":"2.0","result":[{"eventType":{"id":"1","name":"Soccer"},"marketCount":10668},{"eventType":{"id":"2","name":"Tennis"},"marketCount":4590},{"eventType":{"id":"3","name":"Golf"},"marketCount":43},{"eventType":{"id":"4","name":"Cricket"},"marketCount":394},{"eventType":{"id":"5","name":"Rugby Union"},"marketCount":37},{"eventType":{"id":"1477","name":"Rugby League"},"marketCount":24},{"eventType":{"id":"6","name":"Boxing"},"marketCount":27},{"eventType"
...etc...
Currently get
HTTP.ExceptionRequest.StatusError(400, HTTP.Messages.Response:
400 Bad Request.
While the interaction with a particular REST service is a problem-specific issue here are the general guidelines.
Firstly, you need to properly format headers - HTTP.jl manual reads: "headers can be any collection where [string(k) => string(v) for (k,v) in headers] yields Vector{Pair}."
Since we do not have Betfair API key let's have a look on a more generic example using https://postman-echo.com/ which is a free simple API testing that simply returns as JSON whatever it gets as the input.
using HTTP
using JSON
headers = (("X-Application","appKey"),("X-Authentication","sessionToken"),
("content-type","application/json"))
url="https://postman-echo.com/post"
req = Dict("jsonrpc" => "2.0", "params" => Dict("filet" => Dict()))
response = HTTP.post(url, headers, JSON.json(req))
response_text = String(response.body)
json_obj = JSON.parse()
Now let us parse the output from postman-echo.com:
julia> display(JSON.parse(response_text))
Dict{String,Any} with 7 entries:
"headers" => Dict{String,Any}("x-forwarded-port"=>"443","host"=>"postman-echo.com","x-application"=>"appKey","content-type"… "json" => Dict{String,Any}("params"=>Dict{String,Any}("filet"=>Dict{String,Any}()),"jsonrpc"=>"2.0")
"files" => Dict{String,Any}()
"args" => Dict{String,Any}()
"data" => Dict{String,Any}("params"=>Dict{String,Any}("filet"=>Dict{String,Any}()),"jsonrpc"=>"2.0")
"url" => "https://postman-echo.com/post"
"form" => Dict{String,Any}()
You can easily adopt the above code to any RESTful JSON API.
Thanks Przemyslaw Szufel for your response. After a few more days of frustration, I managed to get the first part of the API working using the Excel/VBA sample here: https://github.com/betfair/API-NG-Excel-Toolkit (my translations of the Python examples did not work).
Your comment helped in terms of understanding how to structure multiple headers, and using Dict( =>) rather than string for the manipulations I attempted above.
using HTTP
using JSON
const ListEventTypesMethod = "listEventTypes"
const AppKey = "appKey"
const Session = "sessionToken"
function SendRequest(Url, AppKey, Session, Data)
headers = (("X-Application", AppKey),
("content-type", "application/json"),
("Accept", "application/json"),
("X-Authentication", Session))
HTTP.get(Url,headers,JSON.json(Data))
end
function ParseJsonRpcResponseToCollection(Response)
ParseJsonRpcResponseToCollection = JSON.parse(Response)
end
function GetJsonRpcUrl()
GetJsonRpcUrl = "https://api.betfair.com/exchange/betting/json-rpc/v1/"
end
function MakeJsonRpcRequestString(Method, RequestString)
#MakeJsonRpcRequestString = "{""jsonrpc"": ""2.0"", ""method"": ""SportsAPING/v1.0/" & Method & """, ""params"": " & RequestString & ", ""id"": 1}"
MakeJsonRpcRequestString = Dict("jsonrpc" => "2.0", "method" =>"SportsAPING/v1.0/"*Method, "params" => RequestString, "id"=> 1 )
end
function GetListEventTypesRequestString()
#GetListEventTypesRequestString = "{""filter"":{}}"
GetListEventTypesRequestString=Dict("filter" =>Dict())
end
Request = MakeJsonRpcRequestString(ListEventTypesMethod, GetListEventTypesRequestString())
ListEventTypesResponse = SendRequest(GetJsonRpcUrl(), AppKey, Session, Request)
Response
HTTP/1.1 200 OK
Date: Sun, 17 Feb 2019 17:28:08 GMT
Server: Cougar - 4.4.2 (Cougar 2 mode)
Cache-Control: no-cache
Content-Type: application/json
Vary: Accept-Encoding, User-Agent
Content-Length: 1850
{"jsonrpc":"2.0","result":[{"eventType":{"id":"1","name":"Soccer"},"marketCount":6553},{"eventType":{"id":"2","name":"Tennis"},"marketCount":5511},{"eventType":{"id":"3","name":"Golf"},"marketCount":34}
etc...
Hope this helps others as well.

Flutter http MultipartRequest field can not be Chinese Charaters?

I try to post a form to server and here is the code:
ar request = new http.MultipartRequest("POST", _uri);
request.fields['user_acc'] = _userAcc;
// this issue should be solve
request.fields['user_nick_name'] = '中文名字';
request.fields['user_password'] = _password;
But the server side in the user_nick_name field always got null, note that is always, but I change it into English the server can receive that. I test on postman, the server can got Chinese correctly, so it's MultipartRequest issue on this problem.
My question is: Why the Dart or Flutter team so careless on this so important basic library? They even not consider about this simply issue. I opened a issue on github but no-one response, I think the team is done. So I ask the develop communit here, how to solve this problem anyway?
[UPDATE]
As kindly people suggested, I update my golang server now, if anyone else got this problem, you may wonna answer and suggestions too.
func HandleUserRegister(context *gin.Context) {
userAcc := context.PostForm("user_acc")
userAvatar := context.PostForm("user_avatar")
userNickName := context.PostForm("user_nick_name")
userPassword := context.PostForm("user_password")
userPhone := context.PostForm("user_phone")
userEmail := context.PostForm("user_email")
userGender := context.PostForm("user_gender")
userSign := context.PostForm("user_sign")
userType := context.PostForm("user_type")
userTypeInt, _ := strconv.Atoi(userType)
log.Infof("userAcc: %s, userNickName: %s, userPassword: %s", userAcc, userNickName, userPassword)}
This is based on gin, and this function is the api solver. If anyone wanna help, please help me figure it out.
OK! I update the question now, because it's really weird!. I did those test:
Post multiform via Flutter to Django server, it receives Chinese filed correctly;
Post multiform data via Postman, the golang(gin) server gots Chinese correctly;
Post multiform data via Flutter to golang(gin) server gots Chinese field null;
For more detail, I log the headers from my server for both postman(normal) and flutter (abnormal):
Postman:
request header: map[Content-Type:[multipart/form-data; boundary=--------------------------022341683711652813100488] Postman-Token:[855646d7-5bea-4b8f-b8df-81366226cd49] User-Agent:[PostmanRuntime/7.1.1] Content-Length:[422] Connection:[keep-alive] Cache-Control:[no-cache] Accept:[*/*] Accept-Encoding:[gzip, deflate]]
Flutter:
request header: map[User-Agent:[Dart/2.0 (dart:io)] Content-Type:[multipart/form-data; boundary=dart-http-boundary-.XUeYeqXpg4Yfyh8QhH1T5JB4zi_f3WxX9t7Taxhw91EFqhyki4] Accept-Encoding:[gzip] Content-Length:[574]]
Does anyone can notice the difference and let me know how to change the it make server can receive the Chinese Characters?
#DannyTuppeny is correct. This is a server problem.
When asked to include a non-ASCII field into a multi-part request, the Dart library correctly wraps this with a binary content-transfer-encoding.
String _headerForField(String name, String value) {
var header =
'content-disposition: form-data; name="${_browserEncode(name)}"';
if (!isPlainAscii(value)) {
header = '$header\r\n'
'content-type: text/plain; charset=utf-8\r\n'
'content-transfer-encoding: binary';
}
return '$header\r\n\r\n';
}
(Postman does not and simply sends the utf8 encoded string without any headers.)
Dart/ASCII looks like this:
--dart-http-boundary-HjDS88CmQicdgd8VaHSwPqJK8iR4H6rTG3LovSZy-QXGpU7pAB0
content-disposition: form-data; name="test"
stackover
--dart-http-boundary-HjDS88CmQicdgd8VaHSwPqJK8iR4H6rTG3LovSZy-QXGpU7pAB0
Dart/non-ASCII looks like this:
First boundary: --dart-http-boundary-58NU6u6_Fo22xjH8H7yPCtKuoKgB+A8+RTJ82iIK1gs3nnGMLlp\r\n
Encapsulated multipart part: (text/plain)
content-disposition: form-data; name="test"\r\n
content-type: text/plain; charset=utf-8\r\n
content-transfer-encoding: binary\r\n\r\n
Line-based text data: text/plain
\344\270\255\346\226\207\345\220\215\345\255\227
Boundary: \r\n--dart-http-boundary-58NU6u6_Fo22xjH8H7yPCtKuoKgB+A8+RTJ82iIK1gs3nnGMLlp\r\n
So the problem is that the server is unable to unwrap the value from the encapsulation.
EDIT
Here's the Postman trace I captured yesterday. It's multi-form, but fails to add the content-type-encoding header despite the field being non-ASCII.
MIME Multipart Media Encapsulation, Type: multipart/form-data, Boundary: "--------------------------595246000077585285134204"
[Type: multipart/form-data]
First boundary: ----------------------------595246000077585285134204\r\n
Encapsulated multipart part:
Content-Disposition: form-data; name="name"\r\n\r\n
Data (12 bytes)
0000 e4 b8 ad e6 96 87 e5 90 8d e5 ad 97 ............
Data: e4b8ade69687e5908de5ad97
[Length: 12]
Last boundary: \r\n----------------------------595246000077585285134204--\r\n
I tested by posting to httpbin and the response suggests that the characters were posted correctly:
"user_nick_name":"\u4e2d\u6587\u540d\u5b57"
I tried with both the Stable v1 SDK and a v2 SDK from Flutter. Is it possible the issue is on the server? Have you tried using something like Fiddler to capture what's actually being sent?
Edit: My guess is that your server side code is not correctly reading the data as MultipartForm data (eg. you should be using ParseMultipartForm and reading from MultipartForm).
The problem, it appears, is in formdata.go part of multipart. Go assumes that any multipart part with an Content-Type header is a file (not a field). However, knowing this you can change your server code as follows:
func main() {
r := gin.Default()
r.POST("/sotest", func(c *gin.Context) {
formValue := c.PostForm("form_value")
if formValue == "" {
formFile, _ := c.FormFile("form_value")
file, _ := formFile.Open()
b1 := make([]byte, formFile.Size)
file.Read(b1)
formValue = string(b1)
}
c.JSON(200, gin.H{
"status": "posted",
"formValue": formValue,
})
})
r.Run() // listen and serve on 0.0.0.0:8080
}
When you detect that PostForm returns the empty string, you know that Go has treated the field as a file, in which case you can Open and Read the 'file' and decode it as the utf-8 string that we know it is. Obviously, you could encapsulate the "try as PostForm and if that's empty, try as FormFile" test into a function.
If you don't want to have to test for empty string at the server, you could change your Dart end code to always utf-8 encode even non-ascii strings with
request.files.add(
new http.MultipartFile.fromBytes(
'some_form_value_name',
utf8.encode('the string value'),
contentType: new MediaType('text', 'plain', {'charset': 'utf-8'}),
),
);
and read them at the server with the Open/Read/string method.
I have now solved this. Thanks to Richard and Danny for their help.
1. Reason for this
No matter what happens but this really not only one-side problem, we can not say it's Flutter or Go wrong. But the combination, Flutter + Go server just may be got this issue. The behind reason I still not quit sure, but it must some head not right set (postman can do it right);
2. Solution
We don't only need know why but also how to solve it. Here is what I do:
Do not use the official http package. Using dio, which is a extension Dart package. link: https://pub.dartlang.org/packages/dio
It's more clean and easy to use, so my code becomes to:
FormData _formData = new FormData.from({
"user_acc": _userAcc,
"user_nick_name": _userNickName,
'user_password': _password,
});
Dio dio = new Dio();
Response response = await dio.post(usersUrl, data: _formData);
print(response.data);
I can not post the none-English words now:
INFO[0668] userAcc: ww, userNickName: 小鹿叮叮婴儿湿巾手口专用80抽湿纸巾婴儿湿巾婴儿100抽带盖批发【原价】34.90元【券后】9.9元【省】25元【复制此信息打开手机淘宝即可查看并下单】¥Tnsx0E77pFs¥【必买理由】新品预售80抽*3仙女联盟,更多优惠fd.loliloli.pro , userPassword: ww
INFO[0671] user exist.

Indy HTTP only sending numbers and stops at letters

I'm using Indy with Lazarus
Here is my code:
IdHTTP1.Request.ContentType := 'text/plain' ;
IdHTTP1.Response.ContentType := 'text/plain' ;
IdHTTP1.Response.Charset := 'ISO-8859-1,utf-8;q=0.7,*;q=0.3' ;
IdHTTP1.Request.CharSet:= 'ISO-8859-1,utf-8;q=0.7,*;q=0.3 ' ;
IdHTTP1.HTTPOptions := IdHTTP1.HTTPOptions + [hoNoProtocolErrorException];
IdHTTP1.Get('http://192.168.25.965:8541/rest/SearchCard('+MYCARD+')',Stream) ;
If I start MYCARD with a letter, the server is picking up the full string. However, if I start with a number, it stops at the first letter.
MYCARD:= '12366854'; //works
MYCARD:= 'A125ASD555'; //Works
MYCARD:= '123YH963'; // The server only sees 123
What am I doing wrong?
First off, the two Request properties you are setting are meaningless in a GET request, and you should not be setting any Response properties at all.
// get rid of these assignments
//IdHTTP1.Request.ContentType := 'text/plain' ;
//IdHTTP1.Response.ContentType := 'text/plain' ;
//IdHTTP1.Response.Charset := 'ISO-8859-1,utf-8;q=0.7,*;q=0.3' ;
//IdHTTP1.Request.CharSet:= 'ISO-8859-1,utf-8;q=0.7,*;q=0.3 ' ;
IdHTTP1.HTTPOptions := IdHTTP1.HTTPOptions + [hoNoProtocolErrorException];
IdHTTP1.Get('http://192.168.25.965:8541/rest/SearchCard('+MYCARD+')', Stream);
Second, using the current version of Indy, I cannot reproduce your issue. TIdHTTP.Get() sends the specified URL as-is, it makes no assumptions about the characters in it (you are responsible for URL encoding). In my testing, 123YH963 works just fine. Here is the actual HTP request being sent:
GET /rest/SearchCard(123YH963) HTTP/1.1
Host: 192.168.25.965:8541
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/3.0 (compatible; Indy Library)
As you can see, the full MYCARD text is in the requested resource, as expected. So any truncation must be happening on the server side, not in TIdHTTP itself.
Are you sure you are formatting the URL correctly to begin with? Are you sure it should actually be sent like this:
/rest/SearchCard(123YH963)
And not something more like these instead?
/rest/SearchCard%28123YH963%29
/rest/SearchCard/123YH963
/rest/SearchCard?param=123YH963

ArangoDB can't send request with curl

I can't unserstand what I am doing wrong, but when I am sending next request with curl, I am getting error:
echo {"id":1,"question":"aaa"},{"id":2,"question":"bbb?"} | curl -X POST --data-binary #- --dump - http://localhost:8529/_db/otest/_api/document/?collection=sitetestanswers
HTTP/1.1 100 (Continue)
HTTP/1.1 400 Bad Request
Server: ArangoDB
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Content-Length: 100
{"error":true,"errorMessage":"failed to parse json object: expecting EOF","code":400,"errorNum":600}
Any ideas? I tied wrap it's to [...]. Nothing do not help.
With [...] validator mark this as valid
Same with D. Here is my code:
void sendQuestionsToArangoDB(Json questions)
{
string collectionUrl = "http://localhost:8529/_db/otest/_api/document/?collection=sitetestanswers";
auto rq = Request();
rq.verbosity = 2;
string s = `{"id":"1","question":"foo?"},{"id":2}`;
auto rs = rq.post(collectionUrl, s, "application/json");
writeln("SENDED");
}
--
POST /_db/otest/_api/document/?collection=sitetestanswers HTTP/1.1
Content-Length: 37
Connection: Close
Host: localhost:8529
Content-Type: application/json
HTTP/1.1 400 Bad Request
Server: ArangoDB
Connection: Close
Content-Type: application/json; charset=utf-8
Content-Length: 100
100 bytes of body received
For D I use this lib: https://github.com/ikod/dlang-requests
Same issue with vibed.
ArangoDB do not understand JSON if it's come ass array like [...]. It should be passed as key-value. So if you need pass array it should have key mykey : [].
Here is working code:
import std.stdio;
import requests.http;
void main(string[] args)
{
string collectionUrl = "http://localhost:8529/_db/otest/_api/document?collection=sitetestanswers";
auto rq = Request();
rq.verbosity = 2;
string s = `{"some_data":[{"id":1, "question":"aaa"},{"id":2, "question":"bbb"}]}`;
auto rs = rq.post(collectionUrl, s, "application/json");
writeln("SENDED");
}
otest - DB name
sitetestanswers - collection name (should be created in DB)
echo '[{"id":1,"question":"aaa"},{"id":2,"question":"bbb?"}]'
should do the trick. You need to put ticks around the JSON. The array brackets are necessary otherwise this is not valid JSON.
You are trying to send multiple documents. The data in the original question separates the documents by comma ({"id":1,"question":"aaa"},{"id":2,"question":"bbb?"}) which is invalid JSON. Thus the failed to parse json object answer from ArangoDB.
Putting the documents into angular brackets ([ ... ]) as some of the commentors suggested will make the request payload valid JSON again.
However, you're sending the data to a server endpoint that handles a single document. The API for POST /_api/document/?collection=... currently accepts a single document at a time. It does not work with multiple documents in a single request. It expects a JSON object, and whenever it is sent something different it will respond with an error code.
If you're looking for batch inserts, please try the API POST /_api/import, described in the manual here: https://docs.arangodb.com/HttpBulkImports/ImportingSelfContained.html
This will work with multiple documents in a single request. ArangoDB 3.0 will also allow sending multiple documents to the POST /_api/document?collection=... API, but this version is not yet released. A technical preview will be available soon however.

Resources