I'm building a json query to pass to a mongodb database in R.
In one scenario, I have a vector of dates and I want to query the database to return all records which have a date in the relevant field that matches a date in my vector of dates.
The second scenario is the same as the first, but this time I have a vector of character strings (IDs) and need to return all the records with matching IDs.
I understood the correct way to do this in a json query is to use the $in operator, and then put my vector in an array.
However, when I pass the query to my mongodb database, the exportLogId returns NULL. I'm quite sure that the problem is something to do with how I am representing the $in operator in the final query, since I have very similarly structured queries without the $in operator and they are all working. If I look for just one of my target dates or character strings, I get the desired result.
I followed the mongodb manual here to construct my query, and the only issue I can see is that the $in operator in the output of jsonlite::toJSON() is enclosed in double quotes; whereas I think it might need to be in single quotes (or no quotes at all, but I don't know how to write the syntax for that).
I'm creating my query in two steps:
Create the query as a series of nested lists
Convert the list object to json with jsonlite::toJSON()
Here is my code:
# Load libraries:
library(jsonlite)
# Create list of example dates to query in mongodb format:
sampledates <- c("2022-08-11T00:00:00.000Z",
"2022-08-15T00:00:00.000Z",
"2022-08-16T00:00:00.000Z",
"2022-08-17T00:00:00.000Z",
"2022-08-19T00:00:00.000Z")
# Create query as a list object:
query_list_l <- list(filter =
# Add where clause:
list(where =
# Filter results by list of sample dates:
list(dateSampleTaken = list('$in' = sampledates),
# Define format of column names and values:
useDbColumns = "true",
dontTranslateValues = "true",
jsonReplaceUndefinedWithNull = "true"),
# Define columns to return:
fields = c("id",
"updatedAt",
"person.visualId",
"labName",
"sampleIdentifier",
"dateSampleTaken",
"sequence.hasSequence")))
# Convert list object to JSON:
query_json = jsonlite::toJSON(x = query_list_l,
pretty = TRUE,
auto_unbox = TRUE)
The JSON query now looks like this:
> query_json
{
"filter": {
"where": {
"dateSampleTaken": {
"$in": ["2022-08-11T00:00:00.000Z", "2022-08-15T00:00:00.000Z", "2022-08-16T00:00:00.000Z", "2022-08-17T00:00:00.000Z", "2022-08-19T00:00:00.000Z"]
},
"useDbColumns": "true",
"dontTranslateValues": "true",
"jsonReplaceUndefinedWithNull": "true"
},
"fields": ["id", "updatedAt", "person.visualId", "labName", "sampleIdentifier", "dateSampleTaken", "sequence.hasSequence"]
}
}
As you can see, $in is now enclosed in double quotes, even though I put it in single quotes when I created the query as a list object. I have tried replacing with sprintf() but that just adds a lot of backslashes to my query. I also tried:
query_fixed <- gsub(pattern = "\\"\\$\\in\\"",
replacement = "\\'$in\\'",
x = query_json)
... but this fails with an error.
I would be very grateful to know if:
The syntax problem that is preventing $in from working is actually the double quotes?
If double quotes is the problem, how do I replace them with single quotes without messing up the JSON format?
UPDATE:
The issue seems to occur when R is passing the query to the database, but I still can't work out exactly why.
If I try the query out in loopback explorer in the database, it works and using the export log ID produced, I can then fetch the results with httr::GET() in R. Example query results are shown below (sorry for the hashes - the main point is you can see the format of the returned values):
[1] "[{\"_id\":\"e59953b6-a106-4b69-9e25-1c54eef5264a\",\"updatedAt\":\"2022-09-12T20:08:39.554Z\",\"dateSampleTaken\":\"2022-08-16T00:00:00.000Z\",\"labName\":\"LNG_REFERENCE_DATA_CATEGORY_LAB_NAME_LAB_A\",\"sampleIdentifier\":\"LS0044-SCV2-PCR\",\"sequence\":{\"hasSequence\":false},\"person\":{\"visualId\":\"C-2022-0002\"}},{\"_id\":\"af5cd9cc-4813-4194-b60b-7d130bae47bc\",\"updatedAt\":\"2022-09-12T20:11:07.467Z\",\"dateSampleTaken\":\"2022-08-17T00:00:00.000Z\",\"labName\":\"LNG_REFERENCE_DATA_CATEGORY_LAB_NAME_LAB_A\",\"sampleIdentifier\":\"LS0061-SCV2-PCR\",\"sequence\":{\"hasSequence\":false},\"person\":{\"visualId\":\"C-2022-0003\"}},{\"_id\":\"b5930079-8d57-43a8-85c0-c95f7e0338d9\",\"updatedAt\":\"2022-09-12T20:13:54.378Z\",\"dateSampleTaken\":\"2022-08-16T00:00:00.000Z\",\"labName\":\"LNG_REFERENCE_DATA_CATEGORY_LAB_NAME_LAB_A\",\"sampleIdentifier\":\"LS0043-SCV2-PCR\",\"sequence\":{\"hasSequence\":false},\"person\":{\"visualId\":\"C-2022-0004\"}}]"
I have a txt file, that contains a list of json objects, with 1 of the keys containing an emoji representation in JavaCode format.
I want to parse the json file and convert it to CSV, with the column called emoji, and the emoji to be kept in its JavaCode representation for eg. "\uD83D\uDEB6\u200D\u2642\uFE0F".
Is there any way to do that? Currently, I'm using jsonlite to read it in but it seems to auto-convert the emoji data to be like this (which I assume is its Unicode representation): "<U+0001F6B6><U+200D><U+2640><U+FE0F>"
This is the code to read it in:
df <- fromJSON(data_file)
head(df)
Edited the question to include a sample json object from the list:
{
"x": [
{
"y": "a",
"emoji": "\uD83D\uDEB6\u200D\u2642\uFE0F"
}
],
"z": "b"
}
I have json in which for a key there is HTML code. In that html code there is special char ($ - dollar).
{
"output": {"agreementText": "test data",
"edgeTerms": [
{
"edgeDeviceName": "Apple® iPhone® 6 Plus 16GB in Gold",
"deviceLabel": null,
"edgeTermsAndCondition": "<h3>RETAIL INSTALLMENT SALE AGREEMENT / RETAIL INSTALLMENT OBLIGATION — SUBJECT TO STATE REGULATION</h3> <p>Price $59 and discount $2</p> "
}
]
}
}
When i am trying to paint json data in html using triple curly braces for HTML parsing like
{{{edgeTermsAndCondition}}}
page goes blank but when i using double curly brace it is painting fine but HTML tags as string, which i don't want.
Please help.
I want to export data to an Excel file via HTTP Request. I have a class implementing the request handler interface.
I set the response body to something like header1 \t header2 \t \n content1 \t content2 \t \n
The content-type is application/msexcel; charset=iso-8859-2, and the Content-Dispositionis set to attachment; filename=Excel.xls
This approach works nice for a server-side Javascript application, but when doing the same in ABAP the created Excel File has no columns or rows at all, the whole excel-formatted response body string is inserted in one cell.
Does anyone know, what is the difference between JS and ABAP?
Thanks and best Regards!
I use this sample:
* list1 is an internal table
loop at list1 INTO wa .
CONCATENATE output WA-weekday WA-x WA-DTEXT
WA-SCHKZ WA-BEGTM WA-ENDTM
CL_ABAP_CHAR_UTILITIES=>CR_LF
INTO output SEPARATED BY TAB .
endloop.
* utf-16le , format for excel file
app_type1 = 'APPLICATION/MSEXCEL;charset=utf-16le'.
* convert string to xstring
CALL FUNCTION 'SCMS_STRING_TO_XSTRING'
EXPORTING
text = output
mimetype = 'APPLICATION/MSEXCEL;charset=utf-16le'
IMPORTING
buffer = l_xstring.
CONCATENATE cl_abap_char_utilities=>byte_order_mark_little
l_xstring
INTO l_xstring IN BYTE MODE.
CALL METHOD cl_bsp_utility=>download
EXPORTING
object_s = l_xstring
content_type = app_type
content_disposition = 'attachment;filename=webforms.xls'
response = response
navigation = navigation.
You can also see this thread in sdn:
http://scn.sap.com/people/thomas.jung/blog/2004/08/09/bsp-download-to-excel-in-unicode-format
I allow user input from TinyMCE on client and store it as a JSON string, then pass it to server ASP.NET C#.
The JSON String looks like this: { "mcfn2" : ";lt;p;gt;Trước đ& oacute;, việc tung ra t& ecirc;n miền lần đầu ti& ecirc;n được sự đồng & yacute; của ICANN - tổ chức quản l& yacute; t& ecirc;n miền quốc tế" } (JSON string contains Vietnamese accent)
But when process on server, I received error "Unterminated string. Expected delimiter: ". Line 1, position ...." (It looks like the error happened because of đ& oacute;). (In this page, I seperate & with character after it by a space, because it will automacally converted to a Vietnamese if there are no space)
There are no error if user input is English text (no Vietnamese accent).
Please guide me how to fix this error.
I know at this time this will probably not be useful for you, but maybe it can help another person.
You should convert your string to UTF8 to deal with accents (Vietnamese and many other languages) before serializing it to JSON. For that you can use this function:
private string ConvertToUtf8(string textOriginal)
{
if (!string.IsNullOrEmpty(textOriginal))
{
byte[] bytes = Encoding.Default.GetBytes(textOriginal);
return Encoding.UTF8.GetString(bytes);
}
return string.Empty;
}