Related
Suppose we have a structure:
{
"nested_items": [
{
"nested_sample0": "1",
"nested_sample1": "test",
"nested_sample2": "test",
"nested_sample3": {
"type": "type"
},
"nested_sample": null
},
{
"nested_sample0": "1",
"nested_sample1": "test",
"nested_sample2": "test",
"nested_sample3": {
"type": "type"
},
"nested_sample1": null
},
...
],
"sample1": 1233,
"id": "ed68ca34-6b59-4687-a557-bdefc9ec2f4b",
"sample2": "",
"sample3": "test",
"sample4": "test",
"_ts": 1656503348
}
I want to retrieve documents by id by with limit of "nested_items" field .As I know limit and offset not supported in sub queries. Any way to do this except of divide into two queries? Maybe some udf or else?
You can use the function ARRAY_SLICE assuming the array is ordered.
Example data:
{
"name": "John",
"details": [
{
"id": 1
},
{
"id": 2
},
{
"id": 3
}
]
}
Example queries
-- First 2 items from nested array
SELECT c.name, ARRAY_SLICE(c.details, 0, 2) as details
FROM c
-- Last 2 items from nested array
SELECT c.name, ARRAY_SLICE(c.details, ARRAY_LENGTH(c.details) - 2, 2) as details
FROM c
I have a deserialized object that I want to dynamically loop through to return the related results. The response package looks like so:
{"RatingResponse":
{"Success":"true",
"Message":"",
"QuoteID":"57451",
"LoadNum":"57451",
"Rates":
{"Rate":
[
{"SCAC":"test1",
"CarrierName":"TEST1",
"TransitTime":"1",
"ServiceLevel":"D",
"TotalCost":"1,031.82",
"ThirdPartyCharge":"1,031.82",
"Accessorials":
{"Accessorial":
[
{"Code":"400",
"Cost":"1,655.55",
"Description":"Freight"
},
{"Code":"DSC",
"Cost":"-952.77",
"Description":"Discount"
},
{"Code":"FUE",
"Cost":"329.04",
"Description":"Fuel Surcharge"
}
]
},
"QuoteNumber":""
},
{"SCAC":"test2",
"CarrierName":"TEST2",
"TransitTime":"1",
"ServiceLevel":"D",
"TotalCost":"1,031.82",
"ThirdPartyCharge":"1,031.82",
"Accessorials":
{"Accessorial":
[
{"Code":"400",
"Cost":"1,655.55",
"Description":"Freight"
},
{"Code":"DSC",
"Cost":"-952.77",
"Description":"Discount"
},
{"Code":"FUE",
"Cost":"329.04",
"Description":"Fuel Surcharge"
}
]
},
"QuoteNumber":""
}
]
},
"AverageTotalCost":"1,031.82"
}
}
I have parsed the response data so that there is less information to work with, especially since I only need the Accessorial Costs. The parsed response looks like
[
{
"SCAC": "test1",
"CarrierName": "TEST1",
"TransitTime": "1",
"ServiceLevel": "D",
"TotalCost": "1,031.82",
"ThirdPartyCharge": "1,031.82",
"Accessorials": {
"Accessorial": [
{
"Code": "400",
"Cost": "1,655.55",
"Description": "Freight"
},
{
"Code": "DSC",
"Cost": "-952.77",
"Description": "Discount"
},
{
"Code": "FUE",
"Cost": "329.04",
"Description": "Fuel Surcharge"
}
]
},
"QuoteNumber": ""
},
{
"SCAC": "test2",
"CarrierName": "TEST2",
"TransitTime": "1",
"ServiceLevel": "D",
"TotalCost": "1,031.82",
"ThirdPartyCharge": "1,031.82",
"Accessorials": {
"Accessorial": [
{
"Code": "400",
"Cost": "1,655.55",
"Description": "Freight"
},
{
"Code": "DSC",
"Cost": "-952.77",
"Description": "Discount"
},
{
"Code": "FUE",
"Cost": "329.04",
"Description": "Fuel Surcharge"
}
]
},
"QuoteNumber": ""
}
]
The problem I am facing is that I will never know how many Rate items will come back in the response data, nor will I know the exact amount of Accessorial Costs. I'm hoping to capture the Rate child node counts and the Accessorial child node counts per Rate. Here's what I have so far.
Root rootObject = Newtonsoft.Json.JsonConvert.DeserializeObject<Root>(responseFromServer);
//rate stores the parsed response data
JArray rate = (JArray)JObject.Parse(responseFromServer)["RatingResponse"]["Rates"]["Rate"];
var rate2 = rate.ToString();
//this for loop works as expected. it grabs the number of Rate nodes (in this example, 2)
for (int i = 0; i < rate.Count(); i++)
{
dynamic test2 = rate[i];
//this is where I'm struggling
dynamic em = (JArray)JObject.Parse(test2)["Accessorials"]["Accessorial"].Count();
for (int j = 0; j < em; j++)
{
string test3 = test2.Accessorials.Accessorial[j].Cost;
System.IO.File.AppendAllText(logPath, Environment.NewLine + test3 + Environment.NewLine);
}
}
I apologize in advance for the bad formatting and odd variable names - I'm obviously still testing the functionality, so I've been using random variables.
Where I'm struggling (as notated above) is getting to the Accessorial node to count how many items are in its array. I was thinking I could parse the first array (starting with SCAC data) and extend down to the Accessorial node, but I'm not having any luck.
Any help is GREATLY appreciated, especially since I am new to this type of code and have spent the majority of the day trying to resolve this.
you can try this
var rates = (JArray)JObject.Parse(json)["RatingResponse"]["Rates"]["Rate"];
var costs = rates.Select(r => new
{
CarrierName = r["CarrierName"],
Costs = ((JArray)((JObject)r["Accessorials"])["Accessorial"])
.Where(r => (string)r["Description"] != "Discount")
.Select(r => (double)r["Cost"]).Sum()
}).ToList();
result
[
{
"CarrierName": "TEST1",
"Costs": 1984.59
},
{
"CarrierName": "TEST2",
"Costs": 1984.59
}
]
I have a dictionary dct with two sets set1 and set2 of different lengths.
dct ={
"id": "1234",
"set1": [
{
"sub_id": "1234a",
"details": [
{
"sum": "10",
"label": "pattern1"
}
],
},
{
"sub_id": "1234b",
"details": [
{
"sum": "10",
"label": "pattern3"
}
],
}
],
"set2": [
{
"sub_id": "3463a",
"details": [
{
"sum": "20",
"label": "pattern1"
}
],
},
{
"sub_id": "3463b",
"details": [
{
"sum": "100",
"label": "pattern2"
}
],
},
{
"sub_id": "3463c",
"details": [
{
"sum": "100",
"label": "pattern3"
}
],
}
]
}
I need to check for each label if the corresponding sum has changed, and if yes, subtract these.
pairs1=[]
pairs2=[]
for d in dct['set1']:
for dd in d['details']:
pairs1.append((dd['label'],dd['sum']))
for d in dct['set2']:
for dd in d['details']:
pairs2.append((dd['label'],dd['sum']))
result={}
for p in pairs1:
for pp in pairs2:
if p[0] == pp[0]:
result[p[0]]= int(pp[1])-int(p[1])
result
Output something like:
{'pattern1': 10, 'pattern3': 90}
Is there a better way to iterate through the nested dictionary?
I'm trying to flatten and filter my json data that is in a CosmosDB. The data looks like below and I would like to flatten everything in the array Variables. I need to show the id, amount and the array of orderidentifiers( only the orderId field)
my json:
{
"id": "b71687be180da8116208cbb9a40e7e5e630e6cd595f3e09040a155978a2169f3",
"amount": 5354.39,
"orderIdentifiers": [
{
"orderId": "16520328183646646587",
"itemIds": [
"90420839-2769-3acc-a686-3171386190a7"
]
},
{
"orderId": "45288779686596595261",
"itemIds": [
"fb662e41-1c7e-3f36-8cfd-ef2f3c7f0752",
"0b97371c-4eb9-3ec5-8ab9-bb65a9c9efe1"
]
},
{
"orderId": "859986489484974993023",
"itemIds": [
"5c16bb0d-f1a4-3289-bda5-28d0b09a2a56",
"ffb5dc2a-6e43-321d-97cb-3279ddfd1e39",
"781953ad-83e0-30da-a563-e69ed2a752c6",
"1936e885-c41a-3bd1-a3e0-80e7e3089fe4"
]
},
{
"orderId": "80455639909013091834",
"itemIds": [
"147d4a11-6c05-3fe3-8e4c-bcda3d238845",
"6c586585-6355-393d-a6f4-6fa6c665f3b8"
]
}
]
}
when I run this query :
select c.id, c.amount, oi.orderId
from finalcategorysales c
JOIN oi IN c.orderIdentifiers
where c.id='b71687be180da8116208cbb9a40e7e5e630e6cd595f3e09040a155978a2169f3'
the result is showing me only one element in orderIdentifiers array
{ "id": "b71687be180da8116208cbb9a40e7e5e630e6cd595f3e09040a155978a2169f3",
"amount": 5354.39,
"orderId": "16520328183646646587"
},
You can try something like this SQL:
SELECT
c.id, c.amount, Array(SELECT oi.orderId FROM c JOIN oi IN c.orderIdentifiers) AS orderIds
FROM c
WHERE c.id='b71687be180da8116208cbb9a40e7e5e630e6cd595f3e09040a155978a2169f3'
Result:
[
{
"id": "b71687be180da8116208cbb9a40e7e5e630e6cd595f3e09040a155978a2169f3",
"amount": 5354.39,
"orderIds": [
{
"orderId": "16520328183646646587"
},
{
"orderId": "45288779686596595261"
},
{
"orderId": "859986489484974993023"
},
{
"orderId": "80455639909013091834"
}
]
}
]
Reporting API v4
I am a developer. I have my clients google adwords and analytics. I have been using adwords and analytics report API for almost a year now.
I am also using https://ga-dev-tools.appspot.com/query-explorer/. The query builder. For comparing if I have retrieve the right amount of data.
I don't know if its an error or not but its acting weird.
Try number 1 using https://ga-dev-tools.appspot.com/query-explorer/
I tried to add 2 metrics and 7 dimensions. This Account ID, contains 1 million data in only 1 month. I know this because I retrieved 1 million in a range of july 25, 2018 - august 16, 2018.
Then, here's the interesting part. I run the query again with the same parameters, it retrieves 5999 results. I did it again it returns 1 million. The results keep changing. I thought its the error in my code but its also happening in the query builder.
What do you guys think? is it a bug or not?
You can try this if you have more than a million data.
I know its not related to coding. But Google Analytics doesn't have forums just like Adwords.
Try number 2 using this link https://developers.google.com/analytics/devguides/reporting/core/v4/rest/v4/reports/batchGet
this is my request
{
"reportRequests": [
{
"dateRanges": [
{
"endDate": "2018-08-16",
"startDate": "2018-07-16"
}
],
"dimensions": [
{
"name": "ga:dimension2"
},
{
"name": "ga:dimension3"
},
{
"name": "ga:dimension1"
},
{
"name": "ga:adPlacementDomain"
}
],
"pageSize": 5,
"viewId": "********",
"samplingLevel": "LARGE",
"metrics": [
{
"expression": "ga:entrances"
},
{
"expression": "ga:newUsers"
}
],
"includeEmptyRows": true
}
]
}
The return of rowCount is sometimes 2111 and then 1000000.
This my response json with 1million result:
{
"reports": [
{
"columnHeader": {
"dimensions": [
"ga:dimension2",
"ga:dimension3",
"ga:dimension1",
"ga:adPlacementDomain"
],
"metricHeader": {
"metricHeaderEntries": [
{
"name": "ga:entrances",
"type": "INTEGER"
},
{
"name": "ga:newUsers",
"type": "INTEGER"
}
]
}
},
"data": {
"rows": [
{
"dimensions": [
"(other)",
"(other)",
"(other)",
"(other)"
],
"metrics": [
{
"values": [
"120834",
"68730"
]
}
]
},
{
"dimensions": [
"1000025873.1532426892",
"1532426891790.o9z84x",
"2018-07-24T11:08:15.449+01:00",
"unknown"
],
"metrics": [
{
"values": [
"0",
"0"
]
}
]
},
{
"dimensions": [
"1000025873.1532426892",
"1532426891790.o9z84x",
"2018-07-24T11:08:17.589+01:00",
"unknown"
],
"metrics": [
{
"values": [
"0",
"0"
]
}
]
},
{
"dimensions": [
"1000025873.1532426892",
"1532426891790.o9z84x",
"2018-07-24T11:08:31.809+01:00",
"unknown"
],
"metrics": [
{
"values": [
"0",
"0"
]
}
]
},
{
"dimensions": [
"1000025873.1532426892",
"1532427045552.p38pk78",
"2018-07-24T11:09:06.43+01:00",
"unknown"
],
"metrics": [
{
"values": [
"0",
"0"
]
}
]
}
],
"totals": [
{
"values": [
"158626",
"90225"
]
}
],
"rowCount": 1000000,
"minimums": [
{
"values": [
"0",
"0"
]
}
],
"maximums": [
{
"values": [
"120834",
"68730"
]
}
],
"isDataGolden": true
},
"nextPageToken": "5"
}
]
}
another response example when i have less 1million results:
{
"reports": [
{
"columnHeader": {
"dimensions": [
"ga:dimension2",
"ga:dimension3",
"ga:dimension1",
"ga:adPlacementDomain"
],
"metricHeader": {
"metricHeaderEntries": [
{
"name": "ga:entrances",
"type": "INTEGER"
},
{
"name": "ga:newUsers",
"type": "INTEGER"
}
]
}
},
"data": {
"rows": [
{
"dimensions": [
"1002211166.1531434756",
"1531762918308.fjnj7pa6",
"2018-07-16T18:41:58.307+01:00",
"mobileapp::2-com.forsbit.spider"
],
"metrics": [
{
"values": [
"1",
"0"
]
}
]
},
{
"dimensions": [
"1002211166.1531434756",
"1531771001486.jawfrpz8",
"2018-07-16T20:56:41.482+01:00",
"mobileapp::2-com.forsbit.spider"
],
"metrics": [
{
"values": [
"1",
"0"
]
}
]
},
{
"dimensions": [
"1002211166.1531434756",
"1531772475507.7n4w2qzb",
"2018-07-16T21:21:15.503+01:00",
"mobileapp::2-com.forsbit.spider"
],
"metrics": [
{
"values": [
"1",
"0"
]
}
]
},
{
"dimensions": [
"1002211166.1531434756",
"1531859165986.zl7we6a5",
"2018-07-17T21:26:05.977+01:00",
"mobileapp::2-com.forsbit.spider"
],
"metrics": [
{
"values": [
"1",
"0"
]
}
]
},
{
"dimensions": [
"1002211166.1531434756",
"1531859632678.dz7hccsa",
"2018-07-17T21:33:52.673+01:00",
"mobileapp::2-com.forsbit.spider"
],
"metrics": [
{
"values": [
"1",
"0"
]
}
]
},
{
"dimensions": [
"1002211166.1531434756",
"1531861026792.kw71ngx9",
"2018-07-17T21:42:31.667+01:00",
"mobileapp::2-com.forsbit.spider"
],
"metrics": [
{
"values": [
"1",
"0"
]
}
]
}
],
"totals": [
{
"values": [
"2111",
"233"
]
}
],
"rowCount": 2112,
"minimums": [
{
"values": [
"0",
"0"
]
}
],
"maximums": [
{
"values": [
"1",
"1"
]
}
],
"isDataGolden": true
},
"nextPageToken": "6"
}
]
}
I am assuming that you have kept all the queries intact. Double check just to make sure.
Second step would be to check for sampling. Check the field samplingSpaceSizes and samplesReadCounts in the response for sampling. If these fields were not defined that means no sampling was introduced.