Kusto extract text between string - azure-data-explorer

Am trying to use regex to extract a string between a set of strings. But Kusto complains about the regex expression as invalid.
Am trying to replicate the expression from this link in my kusto query.
traces
| where ....
| project extract_all(#"(?<=This is)(.*)(?=sentence)", message)
Kusto would complain this error..
Relop semantic error: 'extractall' has the following semantic error: SEM0420: Regex pattern is ill-formed: (?<=This is)(.*)(?=sentence).
If issue persists, please open a support ticket. Request id: 01b819be-2e11-4983-9312-30f946c07afb
Could someone please help me fix the syntax of the above regex for kusto?

as the error message suggests, the regular expression you're specifying as the argument to the extract_all() method is invalid.
from the documentation: https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/re2
The regular expression syntax supported by Kusto is that of the re2 library.
If you were to provide a sample input and the matching expected output, it'd be easier to provide you with a functional and efficient solution.

Related

Valid JSONPath is throwing EOF in RestAssured JsonPath

The JSONPath that is normally working is not working when used in rest-assured-JSONPath
tenants=jpath.getList("$..aa[?(#.name=='tid')].value");
is throwing
Script1.groovy: 1: expecting EOF, found '[' # line 1, column 59.
untime-response'.$..aa[?(#.name=
$..aa[?(#.name=='tid')].value is working when I validated using https://jsonpath.com/
what am i doing incorrectly? please help. thanks in advance.
As mentioned in the docs, the jsonpath library in rest-assured uses Groovy GPath syntax and NOT Jayway's jsonpath syntax which is what you're trying. If you post a sample json you're working with and what you're trying to extract, I can try to help you come up with the correct expression.

infomatica unconnected lookup invalid symbol reference error

I am getting invalid symbol reference for below expression in informatica cloud
decode(true,STAGE_NAME_CODE='S1',:LKP.Lkp_SFDC_description(CODE),0)
in the unconnected lookup i have three fields code,code_type and description, I want to return description for stage_name_code from source file from the unconnected lookup field Code
but he onlytime the expression is valid is when its written like this..
decode(true,STAGE_NAME_CODE='S1',:LKP.Lkp_SFDC_description(STAGE_NAME_CODE))
Rather than using DECODE, you can use IIF function.
IIF(STAGE_NAME_CODE='S1',:LKP.Lkp_SFDC_description(STAGE_NAME_CODE))
This provides you an exact result.
To have fun on ETL, play with https://etlinfromatica.wordpress.com/

Firebase validate a zip code

I tried the following regular expression
newData.val().matches(/(^\d{5}$)|(^\d{5}-\d{4}$)/)
I get an error that the escape character \d isn't recognized. I removed and cleared the errors but I still get a permission denied error.
You must use double slashes \\.
It will look like
newData.val().matches(/(^\\d{5}$)|(^\\d{5}-\\d{4}$)/)
Take some time observing some of the examples that you can find on the documentation.

R error: regular expression is invalid in this locale

I am trying to gather all instances of "Walloni\xeb" within a data-frame column in order to remove "\" using the grep function. However, I'm getting the following error message as shown below:
grep("Walloni\xeb", InvoAndinfo2$Regio)
Error in grep("Walloni\xeb", InvoAndinfo2$Regio) :
regular expression is invalid in this locale
Does anyone know what to do to resolve this?
The backslash is a special character in regexp, if you want to look for a string that has a backslash, you should escape it by adding another backslah in front of it.
Try:
grep("Walloni\\xeb", InvoAndinfo2$Regio)

Bracket-escaped table names with dplyr

I'm programmatically fetching a bunch of datasets, many of them having silly names that begin with numbers and have special characters like minus signs in them. Because none of the datasets are particularly large, and I wanted the benefit R making its best guess about data types, I'm (ab)using dplyr to dump these tables into SQLite.
I am using square brackets to escape the horrible table names, but this doesn't seem to work. For example:
data(iris)
foo.db <- src_sqlite("foo.sqlite3", create = TRUE)
copy_to(foo.db, df=iris, name="[14m3-n4m3]")
This results in the error message:
Error in sqliteSendQuery(conn, statement, bind.data) : error in statement: no such table: 14m3-n4m3
This works if I choose a sensible name. However, due to a variety of reasons, I'd really like to keep the cumbersome names. I am also able to create such a badly-named table directly from sqlite:
sqlite> create table [14m3-n4m3](foo,bar,baz);
sqlite> .tables
14m3-n4m3
Without cracking into things too deeply, this looks like dplyr is handling the square brackets in some way that I cannot figure out. My suspicion is that this is a bug, but I wanted to check here first to make sure I wasn't missing something.
EDIT: I forgot to mention the case where I just pass the janky name directly to dplyr. This errors out as follows:
library(dplyr)
data(iris)
foo.db <- src_sqlite("foo.sqlite3", create = TRUE)
copy_to(foo.db, df=iris, name="14M3-N4M3")
Error in sqliteSendQuery(conn, statement, bind.data) :
error in statement: unrecognized token: "14M3"
This is a bug in dplyr. It's still there in the current github master. As #hadley indicates, he has tried to escape things like table names in dplyr to prevent this issue. The current problem you're having arises from lack of escaping in two functions. Table creation works fine when providing the table name unescaped (and is done with dplyr::db_create_table). However, the insertion of data to the table is done using DBI::dbWriteTable which doesn't support odd table names. If the table name is provided to this function escaped, it fails to find it in the list of tables (the first error you report). If it is provided escaped, then the SQL to do the insertion is not synatactically valid.
The second issue comes when the table is updated. The code to get the field names, this time actually in dplyr, again fails to escape the table name because it uses paste0 rather than build_sql.
I've fixed both errors at a fork of dplyr. I've also put in a pull request to #hadley and made a note on the issue https://github.com/hadley/dplyr/issues/926. In the meantime, if you wanted to you could use devtools::install_github("NikNakk/dplyr", ref = "sqlite-escape") and then revert to the master version once it's been fixed.
Incidentally, the correct SQL-99 way to escape table names (and other identifiers) in SQL is with double quotes (see SQL standard to escape column names?). MS Access uses square brackets, while MySQL defaults to backticks. dplyr uses double quotes, per the standard.
Finally, the proposal from #RichardScriven wouldn't work universally. For example, select is a perfectly valid name in R, but is not a syntactically valid table name in SQL. The same would be true for other reserved words.

Resources