Unable to decode form field data in karate framework [duplicate] - decode

This question already has an answer here:
Karate: Query Param values are getting encoded
(1 answer)
Closed 1 year ago.
I have recently learnt about Karate and it's awesome experience working on this.
I'm stuck at one problem and looked out for a solution from various websites but it didn't help
Scenario:
Given url "https://test.payu.in/_payment"
And form field surl = '/payu/web-response/17703721?appVersion=null&clientId=web-client/1.0'
When method POST
Then status 302
When we hit the above request the form field value is getting encoded as 'surl=%2Fpayu%2Fweb-response%2F17703721%3FappVersion%3Dnull%26clientId%3Dweb-client%2F1.0'
due to this request is failing
If you can provide me the solution it would be really helpfull
Solutions tried :
java.net.URLDecoder.decode('/payu/web-response/17703721?appVersion=null&clientId=web-client/1.0', 'UTF-8')
But no luck

Karate is doing the right thing. You can see for yourself using this 3-line Karate test:
* url 'https://httpbin.org/anything'
* form field foo = 'one/two?three=four'
* method post
You can see that the "raw" request is:
1 > POST https://httpbin.org/anything
1 > Content-Type: application/x-www-form-urlencoded
1 > Content-Length: 28
1 > Host: httpbin.org
1 > Connection: Keep-Alive
1 > User-Agent: Apache-HttpClient/4.5.13 (Java/11.0.11)
1 > Accept-Encoding: gzip,deflate
foo=one%2Ftwo%3Fthree%3Dfour
But in the response, you can see the server handled it correctly:
1 < 200
1 < Date: Wed, 06 Oct 2021 12:06:10 GMT
1 < Content-Type: application/json
1 < Content-Length: 513
1 < Connection: keep-alive
1 < Server: gunicorn/19.9.0
1 < Access-Control-Allow-Origin: *
1 < Access-Control-Allow-Credentials: true
{
"args": {},
"data": "",
"files": {},
"form": {
"foo": "one/two?three=four"
},
"headers": {
"Accept-Encoding": "gzip,deflate",
"Content-Length": "28",
"Content-Type": "application/x-www-form-urlencoded",
"Host": "httpbin.org",
"User-Agent": "Apache-HttpClient/4.5.13 (Java/11.0.11)",
"X-Amzn-Trace-Id": "Root=1-615d9132-260dcbf96f57a6992b6273dc"
},
"json": null,
"method": "POST",
"origin": "122.179.54.225",
"url": "https://httpbin.org/anything"
}
See the JSON field called form in the response.
I think you should do some research, or talk to your "server" team. Who knows, maybe Karate has found a bug for you.

Related

Azure Cosmos Table API Rest API $filter InvalidInput

I am trying to query a Cosmosdb with a Table API with Postman and use OData.
I have a query which works for $top and $select - but returns Invalid Input when I try a filter.
so for the query.
https://{{databaseAccount}}.table.cosmos.azure.com:443/Rdm()?$filter=PartitionKey eq '726'
I get returned
{
"odata.error": {
"code": "InvalidInput",
"message": {
"lang": "en-us",
"value": "Request url is invalid.\r\nActivityId: 3cf6769d-fe56-40ca-832d-305820ef78e1, documentdb-dotnet-sdk/2.14.0 Host/64-bit MicrosoftWindowsNT/10.0.19041.0\nRequestID:3cf6769d-fe56-40ca-832d-305820ef78e1\n"
}
}
}
Though the following will return me results:
https://{{databaseAccount}}.table.cosmos.azure.com:443/Rdm()?$top=1&$select=PartitionKey
Gives:
{
"value": [
{
"odata.etag": "W/\"datetime'2022-01-01T14%3A07%3A54.8052493Z'\"",
"PartitionKey": "8"
}
]
}
Is there some weird little header that is required to do filtering?
Clearly Authorisation is working, otherwise headers sent are:-
Accept: application/json
x-ms-version: 2021-06-08
x-ms-date: Fri, 06 May 2022 17:14:30 GMT
Authorization: SharedKey {{databaseAccount}}:gargbagebutitdoesappeeartotherwisework
User-Agent: PostmanRuntime/7.29.0
Cache-Control: no-cache
Postman-Token: 0e09a9e8-210d-4b6f-a258-efe6963bff39
Host: {{databaseAccount}}.table.cosmos.azure.com:443
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
The signature is composed of:
verb + \n + \n + \n + dateUtc + \n + /{{databaseAccount}}/{{table}}
where table in this case = Rdm()
Turns out I needed to add the header - MaxDataServiceVersion: 3.0;NetFx
Eventually found the answer in GitHub https://github.com/Azure/azure-sdk-for-go/issues/1020
It's also quite clearly stated in their documentation - doh.
https://learn.microsoft.com/en-us/rest/api/storageservices/setting-the-odata-data-service-version-headers
When you call one of these operations, you must specify the OData data
service version, using one of the following request headers:
MaxDataServiceVersion, to specify the maximum data service version
DataServiceVersion, to specify the exact data service version
If both headers are present, precedence is given to
MaxDataServiceVersion

CouchDB request with JWT causes error "No DB shards could be opened"

I have a couchdb server, which at this moment is for development and it has just one node running in docker.
I would like to authenticate through JWT. I have build my token.
GET https://comp010:6984/_session
Accept: application/json
Content-Type: application/json; charset=utf-8
Authorization: Bearer <JWT token>
I get proper answer (or at least I think so):
{
"ok": true,
"userCtx": {
"name": "uaru",
"roles": "admin"
},
"info": {
"authentication_handlers": [
"jwt",
"cookie",
"default"
],
"authenticated": "jwt"
}
}
When I send request to get actual object from the database
GET https://comp010:6984/db_userspaces/xxxx3
Accept: application/json
Content-Type: application/json; charset=utf-8
I get "unauthorized" exception. This is ok, I did not authenticated this request. So I add the same authorization header:
GET https://comp010:6984/db_userspaces/xxxx3
Accept: application/json
Content-Type: application/json; charset=utf-8
Authorization: Bearer <JWT token>
And I get
{
"error": "internal_server_error",
"reason": "No DB shards could be opened.",
"ref": 179462285
}
But if I switch off the authorization ( [chttpd] require_valid_user = false), and send the same request without Authorization header,
GET https://comp010:6984/db_userspaces/xxxx3
Accept: application/json
Content-Type: application/json; charset=utf-8
I get proper response.
Server: CouchDB/3.2.1 (Erlang OTP/23)
X-Couch-Request-ID: 02c628ce15
X-CouchDB-Body-Time: 0
{
"_id": "xxxx3",
"_rev": "1-a11f390ffa77a03c557ffbbc7c5fda75",
"x": "1"
}
How JWT can relate to shards? I am puzzled and I cannot find anything related.
There are no errors with Fauxton.
Thank you in advance for any suggestions.
Here is the log when the request took place
couchdb-server_1 | [error] 2022-03-09T04:52:34.662593Z nonode#nohost <0.6234.1> 82a6b79f38 rexi_server: from: nonode#nohost(<0.6134.1>) mfa: fabric_rpc:open_shard/2 error:function_clause [{lists,usort,[<<"admin">>],[{file,"lists.erl"},{line,1063}]},{couch_db,check_security,3,[{file,"src/couch_db.erl"},{line,713}]},{couch_db,is_authorized,2,[{file,"src/couch_db.erl"},{line,705}]},{couch_db,is_member,1,[{file,"src/couch_db.erl"},{line,685}]},{couch_db,check_is_member,1,[{file,"src/couch_db.erl"},{line,671}]},{couch_db,open,2,[{file,"src/couch_db.erl"},{line,166}]},{mem3_util,get_or_create_db,2,[{file,"src/mem3_util.erl"},{line,549}]},{fabric_rpc,open_shard,2,[{file,"src/fabric_rpc.erl"},{line,307}]}]
couchdb-server_1 | [error] 2022-03-09T04:52:34.662982Z nonode#nohost <0.6236.1> 82a6b79f38 rexi_server: from: nonode#nohost(<0.6134.1>) mfa: fabric_rpc:open_shard/2 error:function_clause [{lists,usort,[<<"admin">>],[{file,"lists.erl"},{line,1063}]},{couch_db,check_security,3,[{file,"src/couch_db.erl"},{line,713}]},{couch_db,is_authorized,2,[{file,"src/couch_db.erl"},{line,705}]},{couch_db,is_member,1,[{file,"src/couch_db.erl"},{line,685}]},{couch_db,check_is_member,1,[{file,"src/couch_db.erl"},{line,671}]},{couch_db,open,2,[{file,"src/couch_db.erl"},{line,166}]},{mem3_util,get_or_create_db,2,[{file,"src/mem3_util.erl"},{line,549}]},{fabric_rpc,open_shard,2,[{file,"src/fabric_rpc.erl"},{line,307}]}]
couchdb-server_1 | [error] 2022-03-09T04:52:34.663440Z nonode#nohost <0.6134.1> 82a6b79f38 req_err(179462285) internal_server_error : No DB shards could be opened.
couchdb-server_1 | [<<"fabric_util:get_shard/4 L118">>,<<"fabric_util:get_shard/4 L132">>,<<"fabric:get_security/2 L183">>,<<"chttpd_auth_request:db_authorization_check/1 L112">>,<<"chttpd_auth_request:authorize_request/1 L19">>,<<"chttpd:handle_req_after_auth/2 L325">>,<<"chttpd:process_request/1 L310">>,<<"chttpd:handle_request_int/1 L249">>]
couchdb-server_1 | [notice] 2022-03-09T04:52:34.663753Z nonode#nohost <0.6134.1> 82a6b79f38 comp010:6984 ::ffff:150.26.121.46 uaru GET /db_userspaces/xxxx3 500 ok 2
In the payload to be turned into JWT, roles MUST BE an array.
{
:sub => username,
:'_couchdb.roles' => roles,
:exp => ...,
}
In my case, roles was not. But it means that should be error 400 Bad Request.
The whole problem has nothing to do with shards configuration, etc. The error message was misleading.
Thanks to people in CouchDb slack channel for guiding me in the right direction.

Google Analytics API call frequently returns expected data but occasionally the same call returns error 500

we have been running Google Analytics(GA) API for several accounts .
It works well for all the accounts except for one which occasionally(not always) fails and returns the error 500.
Regarding the account with the problem, the error 500 started to happen since last year(2020/DEC).
The occurrence of this error(500) shows a tendency to happen when the API CALL is made to fetch reports for periods over 3 days (However the chance of this error for periods shorter than 3 days so far has not been zero.)
Regarding the explanation above↑ the problem seems to be in GA side.
May we ask your assistance to check up the situation and help us to solve the problem.
We deeply appreciate all your help and kindness.
Below, As an example I paste the returned response(error 500) for the API call we made in 2021/JULY/8.
Best regards,
Shahin
I, [2021-07-09T18:11:44.314945 #13000] INFO -- : get https://analytics.googleapis.com/analytics/v3/data/ga?dimensions=ga%3Adate%2Cga%3AadDistributionNetwork%2Cga%3AsourceMedium%2Cga%3AadKeywordMatchType%2Cga%3Akeyword%2Cga%3AadContent%2Cga%3Acampaign&end-date=2021-07-08&ids=ga%3A61390358&max-results=1000&metrics=ga%3Agoal11Completions%2Cga%3Agoal12Completions%2Cga%3Agoal13Completions%2Cga%3Agoal14Completions%2Cga%3Agoal15Completions%2Cga%3Agoal16Completions%2Cga%3Agoal17Completions%2Cga%3Agoal18Completions%2Cga%3Agoal19Completions%2Cga%3Agoal20Completions&samplingLevel=HIGHER_PRECISION&start-date=2021-07-04&start-index=2001
D, [2021-07-09T18:11:48.813256 #13000] DEBUG -- request: User-Agent: "analytics/v3 google-api-ruby-client/0.8.6 Linux/2.6.32-431.el6.x86_64\n (gzip)"
Accept-Encoding: "gzip"
Content-Type: ""
Authorization: "Bearer ya29.a0ARrdaM9m7nl7eZ40UtyNPMLo-quMyknO5gC6d60ucF2zhN3WgO_YcsHtldGyqpjuTsHUJAFurvcC3cryutiB2U2P_Wf-gM6DtnhkAqEjoPfCKwsWeuz8wrFgXfa7M3QknRcFDFrWO0CNrEKeDG7caf0B1R49Rws"
Cache-Control: "no-store"
D, [2021-07-09T18:11:48.813315 #13000] DEBUG -- request:
I, [2021-07-09T18:12:48.790970 #13000] INFO -- Status: 500
D, [2021-07-09T18:12:48.791044 #13000] DEBUG -- response: vary: "Origin, X-Origin, Referer"
content-type: "application/json; charset=UTF-8"
date: "Fri, 09 Jul 2021 09:12:48 GMT"
server: "ESF"
cache-control: "private"
x-xss-protection: "0"
x-frame-options: "SAMEORIGIN"
x-content-type-options: "nosniff"
alt-svc: "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-T051=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\""
connection: "close"
transfer-encoding: "chunked"
D, [2021-07-09T18:12:48.791065 #13000] DEBUG -- response: {
"error": {
"code": 500,
"message": "There was an internal error.",
"errors": [
{
"message": "There was an internal error.",
"domain": "global",
"reason": "internalError"
}
]
}
}

How to get session token when authenticating to JSON REST API (in R)

I'm trying to access JSON data (in R) from a REST API.
To authenticate myself, I need to use a POST method in https://dashboard.server.eu/login. The data that needs to be sent are email and password:
library(httr)
login <- list(
email = "my#email.com",
password = "mypass"
)
res <- POST("https://dashboard.server.eu/login", body = login, encode = "form", verbose())
When executing the above, I get this output:
-> POST /login HTTP/1.1
-> Host: dashboard.server.eu
-> User-Agent: libcurl/7.59.0 r-curl/3.3 httr/1.4.1
-> Accept-Encoding: gzip, deflate
-> Cookie: session=10kq9qv1udf0107F4C70RY14fsum41sq50
-> Accept: application/json, text/xml, application/xml, */*
-> Content-Type: application/x-www-form-urlencoded
-> Content-Length: 53
->
>> email=my%40email.com&password=mypass
<- HTTP/1.1 200 OK
<- access-control-allow-headers: Accept, Authorization, Content-Type, If-None-Match
<- access-control-allow-methods: HEAD, GET, POST, PUT, DELETE
<- cache-control: no-cache
<- content-encoding: gzip
<- content-type: application/json; charset=utf-8
<- date: Mon, 09 Mar 2020 14:58:31 GMT
<- set-cookie: session=10kq9qv1udf0107F4C70RY14fsum41sq50; HttpOnly; SameSite=Strict; Path=/
<- vary: origin,accept-encoding
<- x-microserv: NS4yNi4xODQuMjE3
<- x-poweredby: Poetry
<- Content-Length: 2346
<- Connection: keep-alive
The doc of the site says that, in case of success, a JSON res is returned and contains a string token in res.data._id.
I don't find it... even looking at every list (and sub-lists) of res.
How am I supposed to find the token?
Following the doc, and an example in AngularJS, I'm then supposed to do:
// Create JSON Object with your token
let authorizeObject = {
'Authorization': 'Session ' + token,
'content-type': 'application/json;charset=UTF-8',
'accept': 'application/json,text/plain',
};
// Create header from the previous JSON Object
let header = {'headers':authorizeObject};
// Use the header in your http request...
$http.get('https://dashboard.server.eu/', header)
Any hint on making this dream become true?
UPDATE -- With cURL, I could check that there is a _id key/value returned…
With the command:
curl -k -X POST "https://dashboard.server.eu/login" \
-d '{ "email" : "my#email.com", "password" : "mypass" }' \
-H "Content-Type: application/json"
I get the output:
{
"_id": "697v2on4ll0107F4C70RYhosfgtmhfug",
"isAuthenticated": true,
"user": {
"_id": "5dd57868d83cfc000ebbb273",
"firstName": "me",
"lastName": "Me",
...
So, the session token is indeed somewhere...
Does this help to help me?
Looking at the image of res in your question, the message is there, under content - it's just that the content is stored as a vector of raw bytes, which is why you didn't recognise it as json.
Since any file type can be sent by http, the contents in an httr response object are stored in raw format rather than a character string for various reasons - perhaps most importantly because many binary files will contain a 0x00 byte, which isn't allowed in a character string in R.
In your case, we can not only tell that res$content is text, but that it is your "missing" json. The first six bytes of res$content are shown in your image, and are 7b, 22, 5f, 69, 64, 22. We can convert these to a character string in R by doing:
rawToChar(as.raw(c(0x7b, 0x22, 0x5f, 0x69, 0x64, 0x22)))
[1] "{\"_id\""
This matches the first six characters of your expected json string.
Therefore if you do:
httr::content(res, "text")
or
rawToChar(res$content)
You will get your json as a character string.

Language detection always returns 1 result

I'm having troubles with getting Language Detection to work. According to the documentation on Microsoft's website I should be able to get multiple languages when the service is not sure about which language the input string was. However, I am always receiving only one language. In my example below I used "Hallo" which is "Hello" in both German and Dutch. Also with other words like "Bier" which is "Beer" in German and Dutch, the service gives only one result, despite the numberOfLanguages Query parameter value of 5.
Am I missing something?
Is the service malfunctioning?
Can someone provide an example which delivers multiple languages as a result?
Thank you for all your help.
Request:
POST https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/languages?numberOfLanguagesToDetect=5 HTTP/1.1
Content-Type: application/json
Host: westus.api.cognitive.microsoft.com
Ocp-Apim-Subscription-Key: ••••••••••••••••••••••••••••••••
{
"documents": [
{
"id": "Test",
"text": "Hallo"
}
]
}
With this response:
Transfer-Encoding: chunked
x-ms-transaction-count: 0
x-aml-ta-request-id: 4bb2e01e-59a1-4f66-bb31-8ca2c32d262b
X-Content-Type-Options: nosniff
apim-request-id: 65a8d5f5-1394-43cf-89e1-83d7188cb81a
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Date: Tue, 25 Apr 2017 15:18:34 GMT
Content-Type: application/json; charset=utf-8
{
"documents": [
{
"id": "Test",
"detectedLanguages": [
{
"name": "German",
"iso6391Name": "de",
"score": 1.0
}
]
}
],
"errors": []
}
The documentation has been updated to explain this.
numberOfLanguagesToDetect: Format - int32. (Optional. Deprecated) Number of languages to detect. Set to 1 by default. Irrespective of
the value, the language with the highest score is returned.

Resources