I have a XLS file in the following format-- XLS input file. The first column is just for explanation of JSON elements. I want to convert the column headers into main elements and then the next 4 rows into nested JSON elements. The desired JSON file will look like this-- JSON output file. I have tried string substitution in python which makes it easier but is there any way to do this in R?
XLS file -- XLS input file Excel link
Desired JSON output--
{
"MRI ID": {
"Description": "MRI ID"
},
"Participant ID": {
"Description": "Participant ID"
},
"Group": {
"Levels": {
"Case": "",
"Control": "",
"Cohort": ""
}
},
"Gender": {
"Levels": {
"Male": "",
"Female": "",
"Other": ""
}
},
"Age": {
"Units": "Years"
},
"Vocab Raw Score WASI": {
"Description": "Wechsler Abbreviated Scale Intellegence",
"Units": "Raw Score"
},
"Handedness": {
"Levels": {
"Left": "",
"Right": "",
"Ambidextrous": ""
}
},
"Last 60 Days Number of Drinks TLFB": {
"Description": "Alcohol Timeline Followback",
"Units": "Drinks"
},
"Last 60 Days Drinking Days TLFB": {
"Description": "Alcohol Timeline Followback",
"Units": "Days"
},
"Last 60 Days Average Drinks Per Drinking Day TLFB": {
"Description": "Alcohol Timeline Followback",
"Units": "Average Drinks Per Drinking Day"
},
"Last 60 Days Number of Non-Drinking Days TLFB": {
"Description": "Alcohol Timeline Followback",
"Units": "Days"
},
"Last 60 Days Max Number of Drinks TLFB": {
"Description": "Alcohol Timeline Followback",
"Units": "Days"
},
"Last 60 Days Number of Heavy Drinking Days TLFB": {
"Description": "Alcohol Timeline Followback",
"Units": "Days"
},
"Last 60 Days Number of Cigarette Smoking Days TLFB": {
"Description": "Nicotine Timeline Followback",
"Units": "Days"
},
"Last 60 Days Average Number of Cigarettes on Smoking Days TLFB": {
"Description": "Nicotine Timeline Followback",
"Units": "Average Cigarettes"
},
"Last 60 Days Number of Non-Cigarette Smoking Days TLFB": {
"Description": "Nicotine Timeline Followback"
},
"Last 60 Days Max Number of Cigarettes TLFB": {
"Description": "Nicotine Timeline Followback",
"Units": "Cigarettes"
},
"Last 60 Days Number of Marajuana Smoking Days TLFB": {
"Description": "Cannabis Timeline Followback",
"Units": "Days"
},
"Last 60 Days Number of Non-Marajuana Smoking Days TLFB": {
"Description": "Cannabis Timeline Followback",
"Units": "Days"
},
"Matrix Reasoning Raw Score WASI": {
"Description": "Wechsler Abbreviated Scale Intellegence",
"Units": "Raw Score"
}
}
I want to get the boardgame rank (value) from this nested array in Cosmos DB.
{
"name": "Alpha",
"statistics": {
"numberOfUserRatingVotes": 4155,
"averageRating": 7.26201,
"baysianAverageRating": 6.71377,
"ratingStandardDeviation": 1.18993,
"ratingMedian": 0,
"rankings": [
{
"id": 1,
"name": "boardgame",
"friendlyName": "Board Game Rank",
"type": "subtype",
"value": 746
},
{
"id": 4664,
"name": "wargames",
"friendlyName": "War Game Rank",
"type": "family",
"value": 140
},
{
"id": 5497,
"name": "strategygames",
"friendlyName": "Strategy Game Rank",
"type": "family",
"value": 434
}
],
"numberOfComments": 1067,
"weight": 2.3386,
"numberOfWeightVotes": 127
},
}
So I want:
{
"name": "Alpha",
"rank": 746
}
Using this query:
SELECT g.name, r
FROM Games g
JOIN r IN g.statistics.rankings
WHERE r.name = 'boardgame'
I get this (so close!):
{
"name": "Alpha",
"r": {
"id": 1,
"name": "boardgame",
"friendlyName": "Board Game Rank",
"type": "subtype",
"value": 746
}
},
But extending the query to this:
SELECT g.name, r.value as rank
FROM Games g
JOIN r IN g.statistics.rankings
WHERE r.name = 'boardgame'
I get this error:
Failed to query item for container Games:
Message: {"errors":[{"severity":"Error","location":{"start":21,"end":26},"code":"SC1001","message":"Syntax error, incorrect syntax near 'value'."}]}
ActivityId: 0a0cb394-2fc3-4a67-b54c-4d02085b6878, Microsoft.Azure.Documents.Common/2.14.0
I don't understand why this doesn't work? I don't understand what the syntax error is. I tried adding square braces but that didn't help. Can some help me understand why I get this error and also how to achieve the output I'm looking for?
This should work,
SELECT g.name, r["value"] as rank
FROM Games g
JOIN r IN g.statistics.rankings
WHERE r.name = 'boardgame'
FOllowing is the input that i have
[{
"date": " "
},
{
"date": "2022-01-21"
},{
"date": " "
},{
"date": " "
},{
"date": " "
}]
And the required output is as follows
[
{
"Date": "2022-01-21"
},
{
"Date": "2021-01-21"
},
{
"Date": "2020-01-21"
},
{
"Date": "2019-01-21"
},
{
"Date": "2018-01-21"
}
]
Thanks in advance!!
It should be obvious that there is no buil-in function in DataWeave to achieve this result. You can create a custom function for it. For example using a recursive function we can get the expected output from your input:
%dw 2.0
output application/json
fun decDates(a, nextDate)=
[{date: nextDate}]
++
if (sizeOf(a) > 1) decDates(dw::core::Arrays::drop(a, 1), nextDate - |P1Y|) else []
---
decDates(payload, |2022-03-21|)
Output:
[
{
"date": "2022-03-21"
},
{
"date": "2021-03-21"
},
{
"date": "2020-03-21"
},
{
"date": "2019-03-21"
},
{
"date": "2018-03-21"
}
]
I have this json format I'm making an API using ASP.NET.
{
"0": {
"order_id": 11748,
"complete_date": "2021-04-19 14:48:41",
"shipping_code": "aramex.aramex",
"awbs": [
{
"aramex_id": "1314",
"order_id": "11748",
"awb_number": "46572146154",
"reference_number": "11748",
"date_added": "2021-03-04 03:46:58"
}
],
"payment": {
"method": {
"name": "الدفع عند الاستلام",
"code": "cod"
},
"invoice": [
{
"code": "sub_total",
"value": "120.8700",
"value_string": "120.8700 SAR",
"title": "الاجمالي"
},
{
"code": "shipping",
"value": "0.0000",
"value_string": "0.0000 SAR",
"title": "ارمكس"
},
{
"code": "coupon",
"value": "-13.9000",
"value_string": "-13.9000 SAR",
"title": "قسيمة التخفيض(RMP425)"
},
{
"code": "cashon_delivery_fee",
"value": "5.0000",
"value_string": "5.0000 SAR",
"title": "رسوم الدفع عند الاستلام"
},
{
"code": "tax",
"value": "18.1300",
"value_string": "18.1300 SAR",
"title": " ضريبة القيمة المضافة (15%)"
},
{
"code": "total",
"value": "130.1000",
"value_string": "130.1000 SAR",
"title": "الاجمالي النهائي"
}
]
},
"product": [
{
"id": 69,
"name": "مخلط 4 أو دو بيرفيوم للجنسين - 100 مل",
"sku": "45678643230",
"weight": "0.50000000",
"quantity": 1,
"productDiscount": "",
"images": []
}
]
}
}
How can I reach order_id? I made an object let's say its name is obj1 I tried foreach obj1 and storing into a variable obj1.order_id;
It stored null in the variable. the {"0"} is the numbering of orders starts 0-1-2 etc.
You can deserialize that json to Dictionary<string,dynamic> without creating a new class as following:
var values = JsonConvert.DeserializeObject<Dictionary<string, dynamic>>(json);
var orderId = values["0"]["order_id"].ToString();
This will give you 11748 as a result.
I have this file with JSON formatted data, but need this into a dataframe. Ultimately I would like to plot the geolocations onto a map, but can't seem to get this data into a df first.
json_to_df <- function(file){
file <- lapply(file, function(x) {
x[sapply(x, is.null)] <- NA
unlist(x)
})
df <- do.call("rbind", file)
return(df)
}
But I get only this error:
Error in fromJSON(file) :
STRING_ELT() can only be applied to a 'character vector', not a 'list'
The file structure looks like this (this is only part of the data):
{
"results": [
{
"utc_offset": 7200000,
"venue": {
"country": "nl",
"localized_country_name": "Netherlands",
"city": "Bergen",
"address_1": "16 Notweg",
"name": "FitClub Bergen",
"lon": 4.699218,
"id": 24632049,
"lat": 52.673046,
"repinned": false
},
"headcount": 0,
"distance": 22.46796989440918,
"visibility": "public",
"waitlist_count": 0,
"created": 1467149834000,
"rating": {
"count": 0,
"average": 0
},
"maybe_rsvp_count": 0,
"description": "<p>Start your week off right with a Monday Morning Bootcamp!!! The fresh air and peaceful dunes provide the perfect setting for a total body workout. Whether you are a beginner with brand spankin' new health goals and in need of some direction, or training for a race or competition, we're the trainers for you!!! See you at 8:50 for sign-in!</p>",
"event_url": "https://www.meetup.com/FitClubBergen/events/234936736/",
"yes_rsvp_count": 3,
"duration": 3600000,
"name": "Free Bootcamp in the Bergen Dunes",
"id": "glzqvlyvnbgc",
"time": 1477292400000,
"updated": 1477297999000,
"group": {
"join_mode": "open",
"created": 1441658286000,
"name": "FitClub Bergen Free Bootcamp in the Dunes",
"group_lon": 4.710000038146973,
"id": 18908751,
"urlname": "FitClubBergen",
"group_lat": 52.66999816894531,
"who": "FitClubbers"
},
"status": "past"
},
{
"utc_offset": 7200000,
"venue": {
"country": "nl",
"localized_country_name": "Netherlands",
"city": "Bergen",
"address_1": "16 Notweg",
"name": "FitClub Bergen",
"lon": 4.699218,
"id": 24632049,
"lat": 52.673046,
"repinned": false
},
"headcount": 0,
"distance": 22.46796989440918,
"visibility": "public",
"waitlist_count": 0,
"created": 1467149834000,
"rating": {
"count": 0,
"average": 0
},
"maybe_rsvp_count": 0,
"description": "<p>Start your week off right with a Monday Morning Bootcamp!!! The fresh air and peaceful dunes provide the perfect setting for a total body workout. Whether you are a beginner with brand spankin' new health goals and in need of some direction, or training for a race or competition, we're the trainers for you!!! See you at 8:50 for sign-in!</p> <p>ALWAYS FREE</p> <p>FOR ALL LEVELS OF FITNESS</p> <p>BRING: water bottle and energy</p>",
"event_url": "https://www.meetup.com/FitClubBergen/events/234936737/",
"yes_rsvp_count": 3,
"name": "Monday Morning Bootcamp in the Bergen Dunes",
"id": "flzqvlyvnbgc",
"time": 1477292400000,
"updated": 1477303926000,
"group": {
"join_mode": "open",
"created": 1441658286000,
"name": "FitClub Bergen Free Bootcamp in the Dunes",
"group_lon": 4.710000038146973,
"id": 18908751,
"urlname": "FitClubBergen",
"group_lat": 52.66999816894531,
"who": "FitClubbers"
},
"status": "past"
},
{
"utc_offset": 7200000,
"venue": {
"country": "nl",
"localized_country_name": "Netherlands",
"city": "Amsterdam",
"phone": "020 4275777",
"address_1": "Dijksgracht 2",
"address_2": "1019 BS ",
"name": "Klimmuur Central",
"lon": 4.91284,
"id": 1143381,
"lat": 52.376626,
"repinned": false
},
"headcount": 0,
"distance": 1.0689502954483032,
"visibility": "public",
"waitlist_count": 0,
"created": 1477215767000,
"rating": {
"count": 0,
"average": 0
},
"maybe_rsvp_count": 0,
"description": "<p>Climbing Right After Work: RAW.<br/>Quiet hall, pretty much every rope available; no rope chasing necessary. And.. still some time left to do other things later that evening. Take you gear and an extra sandwich to work and join me afterwards pulling some plastic.<br/>Some notes:<br/>- This events starts #17:00. If you can't make it that early, please comment the time you can.<br/>- Please fill in your belaying skills in your profile. If you've never climbed before or don't have belaying skills: follow an introduction course a the gym first! Safety above all!</p>",
"event_url": "https://www.meetup.com/The-Amsterdam-indoor-rockclimbing/events/235054729/",
"yes_rsvp_count": 3,
"name": "Monday's RAW Climb",
"id": "235054729",
"time": 1477321200000,
"updated": 1477334279000,
"group": {
"join_mode": "approval",
"created": 1358348565000,
"name": "The Amsterdam indoor rockclimbing",
"group_lon": 4.889999866485596,
"id": 6689952,
"urlname": "The-Amsterdam-indoor-rockclimbing",
"group_lat": 52.369998931884766,
"who": "Climbers"
},
"status": "past"
},
{
"utc_offset": 7200000,
"venue": {
"country": "nl",
"localized_country_name": "Netherlands",
"city": "Amstelveen",
"address_1": "Langs de Akker 3",
"name": "Emergohal",
"lon": 4.87967,
"id": 23816542,
"lat": 52.290199,
"repinned": false
},
"rsvp_limit": 12,
"headcount": 0,
"distance": 5.541957378387451,
"visibility": "public",
"waitlist_count": 0,
"created": 1474452073000,
"fee": {
"amount": 5.5,
"accepts": "cash",
"description": "per person",
"currency": "EUR",
"label": "price",
"required": "0"
},
"rating": {
"count": 0,
"average": 0
},
"maybe_rsvp_count": 0,
"description": "<p>We will play the Whole Season indoor soccer on Mondays from 18:00 - 19:00 starting 5 September until May 2017 in the Emergohal Amstelveen.</p> <p>Preferred payment is with Paypal EUR 5.50 (in advance)<br/>If this is not possible you may pay cash but then I will ask EUR 6,-<br/>(Please have the exact cash with you)</p> <p>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</p> <p>A couple of Unisys (ex)colleagues and football lovers are playing every Monday in the Emergohal Amstelveen at 6PM on a reasonable good level. We are looking for a compact group of players who are willing/able to play (almost) every Monday playing 5v5 (or 6v6).<br/>We are playing with the FIFA Futsal rules in mind:<br/>http://www.fifa.com/mm/document/footballdevelopment/refereeing/51/44/50/lawsofthegamefutsal2014_15_eneu_neutral.pdf</p> <p>The Emergohal has dressing rooms and a nice bar for after the game.</p> <p>Hope to see you on Mondays</p> <p>Cheers Jeroen</p> <p>For questions you may call me on[masked], send a text message (SMS) or leave a message on this meetup group.</p>",
"event_url": "https://www.meetup.com/Futsal_Emergohal_Monday_18-00/events/234290812/",
"yes_rsvp_count": 11,
"duration": 4500000,
"name": "Futsal",
"id": "234290812",
"time": 1477323900000,
"updated": 1477330559000,
"group": {
"join_mode": "approval",
"created": 1474445066000,
"name": "Futsal_Emergohal_Monday_18.00",
"group_lon": 4.860000133514404,
"id": 20450096,
"urlname": "Futsal_Emergohal_Monday_18-00",
"group_lat": 52.31999969482422,
"who": "Players"
},
"status": "past"
}],
"meta": {
"next": "https://api.meetup.com/2/open_events?and_text=False&offset=1&city=Amsterdam&sign=True&format=json&lon=4.88999986649&limited_events=False&photo-host=public&page=20&time=-24m%2C&radius=25.0&lat=52.3699989319&status=past&desc=False",
"method": "OpenEvents",
"total_count": 643,
"link": "https://api.meetup.com/2/open_events",
"count": 20,
"description": "Searches for recent and upcoming public events hosted by Meetup groups. Its search window is the past one month through the next three months, and is subject to change. Open Events is optimized to search for current events by location, category, topic, or text, and only lists Meetups that have **3 or more RSVPs**. The number or results returned with each request is not guaranteed to be the same as the page size due to secondary filtering. If you're looking for a particular event or events within a particular group, use the standard [Events](/meetup_api/docs/2/events/) method.",
"lon": ,
"title": "Meetup Open Events v2",
"url": "",
"signed_url": "{signed_url}",
"id": "",
"updated": 1479988687055,
"lat":
}
}
So I was wondering how I would put this in a dataframe or csv even to be able to extract geolocations later?
There is no need to write a parser yourself, there are a number of packages that can read JSON formatted data. The one I use, and #hrbrmstr linked, is jsonlite. This package provides a fromJSON function which can parse JSON into a data.frame:
fromJSON('file.json', flatten = TRUE)
note that the flatten argument here ensures the json is flattended into a nice data.frame.