How to Parse complex json in Robot framework - robotframework

I am loading a JSON directly from a file and I need to validate that json for number of attributes .
I am using below json file having data arrays and I need to traverse through all the arrays and retrieve the values (The array is dynamic so cant hardcore the loop for 2 times )
How can I get the number of data arrays present in the response so that I can loop through those and get thevalues.
Json response :
{
"total": 863,
"data": [
{
"id": 6154616,
"categories": {
"total": 0,
"data": []
},
"isAnonymized": false,
"jobOrders": {
"total": 0,
"data": []
},
"jobSubmissions": {
"total": 0,
"data": []
},
"lastName": "u engineering",
"leads": {
"total": 0,
"data": []
},
"timeZoneOffsetEST": 10
},
{
"id": 85456552,
"categories": {
"total": 0,
"data": []
},
"isAnonymized": false,
"jobOrders": {
"total": 0,
"data": []
},
"jobSubmissions": {
"total": 0,
"data": []
},
"lastName": "Engineering",
"leads": {
"total": 0,
"data": []
},
"timeZoneOffsetEST": 10
}
]
}
I have written below Robot code but it is not working.
*** Settings ***
Library RequestsLibrary
Library JSONLibrary
*** Test Cases ***
check_response
${response_data}= Load JSON From File ${response}
Log To Console ${cnf_response}
${no_of_arrays}= get Length $.data
Log To Console ${no_of_arrays}

You are using variables not defined in the example - they are probably done so in some other place, but overall that's not how you address a dictionary key in python, e.g. Robot Framework.
Presuming the ${response_data} variable is the parsed json file as a dictionary, this is how to address the key/get the length of the underlying list:
${no_of_arrays}= Get Length $response_data['data']

Related

Generata jsondata robotframework

i got this response :
{
"id": 1,
"name": "me",
"sku": "one"
},
{
"id": 2,
"name": "you",
"sku": "two"
},
{
"id": 3,
"name": "us",
"sku": "three"
}
]
and i want to create new jsondata like this in robotframework as bodydata in another request:
{
"users": [
{
"name": "me",
"id": 1,
},
{
"name": "you",
"id": 2,
}
],
}
how can i do that?
I do not know your implementation, but this works for your example and should give you clue how to solve your problem in general. I have used robotframework-jsonlibrary and ended up with this code. I have put your example response in original.json file and then loaded. You can omit this line, because you already have json loaded in some variable.
*** Settings ***
Library JSONLibrary
*** Test Cases ***
test
${json}= Load JSON From File original.json
${json}= Delete Object From Json ${json} $.[-1]
${json}= Delete Object From Json ${json} $..sku
${new_json}= Convert String to JSON { "users": [] }
${json}= Update Value To Json ${new_json} $.users ${json}
log to console ${json}
JsonLibrary:
https://robotframework-thailand.github.io/robotframework-jsonlibrary/JSONLibrary.html#Load%20JSON%20From%20File

SOAPUI did not match jsonpath in multi dim array

I tried to match jsonpath assertion in SOAPUI 5.5.0 with the following statement:
$.root[*][?(#.id==1)].updates
expected = 2
response to test:
{
"root": [
[
{
"id": 2,
"title": "hello world",
"kind": "post",
"updates": 3,
"comments": [...]
},
{
"id": 3,
"title": "how to best practices",
"kind": "post",
"updates": 0,
"comments": [...]
}
],
[
{
"id": 1,
"title": "Release notes...",
"kind": "newsletter",
"updates": 2,
"forks": [...]
}
]
]
}
soapui seems not to find my nested entity with id=1.
There are array of arrays in the root.
You can use below JSONpath to get the desired output
$..[?(#.id==1)].updates[0]

Not able to retrieve results from Open TSDB via HTTP

I have set up open TSDB using docker image. I am able to push data to database via HTTP post request.
[{
"metric": "sys.cpu.nice",
"timestamp": 1567764102,
"value": 18,
"tags": {
"host": "web01",
"dc": "lga"
}
},
{
"metric": "sys.cpu.abc",
"timestamp": 1567764602,
"value": 9,
"tags": {
"host": "web02",
"dc": "lga"
}
}
]
Same data is also visible on the portal but I want to retrieve this data form TSDB by API.
I used this but it's not returning actual data.
http://localhost:4242/api/search/lookup?m=sys.cpu.nice{host=*}
{
"type": "LOOKUP",
"metric": "sys.cpu.nice",
"tags": [{
"key": "host",
"value": "*"
}],
"limit": 25,
"time": 6.0,
"results": [],
"startIndex": 0,
"totalResults": 0
}
Could you please help to get data back from the database. I need to send this data to some other system.
Try using the /api/query endpoint
In your case you could try one of the following GET requests:
http://localhost:4242/api/query?m=sum:cmp.sys.db{host=server01,tenant01}&start=1h-ago
http://localhost:4242/api/query?m=zimsum:cmp.sys.db{host=server01,tenant01}&start=1h-ago

Error when calling the Anomaly Detection API

I've been messing with the Anomaly Detection API as part of the Microsoft Cognitive Services Labs. However, when I try to POST data to get a response, it always comes back as
400: TypeError('string indices must be integers'
Thinking this is an issue with the format of the data I'm giving it, I double check it with what they have as a sample:
{
"Period": 7,
"Points": [
{
"Timestamp": "2018-03-01T00:00:00Z",
"Value": 32858923
},
{
"Timestamp": "2018-03-02T00:00:00Z",
"Value": 29615278
},
{
"Timestamp": "2018-03-03T00:00:00Z",
"Value": 22839355
},
]
}
And here is some of the data that I'm sending to it that returns the error:
{
"Period": 1,
"Points":[
{
"Timestamp": "1962-01-01T00:00:00Z",
"Value": 589
},
{
"Timestamp": "1962-02-01T00:00:00Z",
"Value": 561
},
{
"Timestamp": "1962-03-01T00:00:00Z",
"Value": 640
}
]
}
Using Python to call the API with the requests package:
request = json.dumps(output)
config = json.load(open("config.json"))
url = "https://api.labs.cognitive.microsoft.com/anomalyfinder/v1.0/anomalydetection"
headers = {"Ocp-Apim-Subscription-Key": config["apiKey"]}
response = requests.post(url, headers=headers, json=request)
The output object is a dictionary that looks like this:
{'Period': 1,
'Points': [{'Timestamp': '1962-01-01T00:00:00Z', 'Value': 589},
{'Timestamp': '1962-02-01T00:00:00Z', 'Value': 561},
{'Timestamp': '1962-03-01T00:00:00Z', 'Value': 640}]}
Which was derived from the monthly milk production data set.
Is there something wrong with my data that I'm missing to cause it to return back that error?

How can I Extract a Large json Array in U-SQL and Include meta-data?

I am trying to extract a large amount of Data from a json document. There are about 1500 nodes per json document. When I attempt to load the body node I get the 128KB limit error. I have found a way to load the node but I have to go all the way down to the array list. JsonExtractor("body.nprobe.items[*]"); The issue I am having is that I cannot access any other part of the json document, I need to get the meta data, like: Id, SerialNumber etc. Should the json file be changed in some way? The data I need is 3 levels down. The json has be obfuscated and shortened, the real file is about 33K lines of formatted json about 1500 items with n-20 fields in each.
{
"headers": {
"pin": "12345",
"Type": "URJ201W-GNZGK",
"RTicks": "3345",
"SD": "211",
"Jov": "juju",
"Market": "Dal",
"Drst": "derre",
"Model": "qw22",
"DNum": "de34",
"API": "34f",
"Id": "821402444150002501"
},
"Id": "db5aacae3778",
"ModelType": "URJ",
"body": {
"uHeader": {
"ID": "821402444150002501",
"SerialNo": "ee028861",
"ServerName": "BRXTTY123"
},
"header": {
"form": 4,
"app": 0,
"Flg": 1,
"Version": 11056,
"uploadID": 1,
"uDate": "2016-04-14T18:29"
},
"nprobe": {
"items": [{
"purchaseDate": "2016-04-14T18:21:09",
"storeLoc": {
"latitude": 135.052335,
"longitude": 77.167005
},
"sr": {
"ticks": 3822,
"SkuId": "24",
"Data": {
"X": "0.00068",
"Y": "0.07246",
}
}
},
{
"purchaseDate": "2016-04-14T18:21:09",
"storeLoc": {
"latitude": 135.052335,
"longitude": 77.167005
},
"sr": {
"ticks": 3823,
"SkuId": "25",
"Data": {
"X": "0",
"Y": "2",
}
}
}]
}
},
"Messages": []
}
Thanks.
You'll have to use CROSS APPLY:
https://msdn.microsoft.com/en-us/library/azure/mt621307.aspx
and EXPLODE:
https://msdn.microsoft.com/en-us/library/azure/mt621306.aspx
See a worked out solution here:
https://github.com/algattik/USQLHackathon/blob/master/VS-Solution/USQLApplication/ciam-to-sqldw.usql
https://github.com/algattik/USQLHackathon/blob/master/Samples/Customer/customers%202016-08-10.json
-- IMPROVED ANSWER: --
As this solution will not work for you since your inner JSON is too large to fit in a string, you can parse the input twice:
DECLARE #input string = #"/so.json";
REFERENCE ASSEMBLY [Newtonsoft.Json];
REFERENCE ASSEMBLY [Microsoft.Analytics.Samples.Formats];
#meta =
EXTRACT Id string
FROM #input
USING new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor();
#items =
EXTRACT purchaseDate string
FROM #input
USING new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor("body.nprobe.items[*]");
#itemsFull =
SELECT Id,
purchaseDate
FROM #meta
CROSS JOIN #items;
OUTPUT #itemsFull
TO "/items_full.csv"
USING Outputters.Csv();

Resources