I want all records where ParaName matches with tagName. i have tried Length, LEFT and SUBSTRING function, but i think Length,LEFT and SUBSTRING functions are not possible in U-SQL. If possible, what is the syntax??
#var=
SELECT * FROM Table
WHERE ParaName LIKE tagName+"%";
U-SQL emphasizes the use of C# Expressions and methods on .NET types to handle many common cases that SQL achieves with functions.
In this case your type is string (System.String) so methods like StartsWith() and Contains() can be used among many others.
Example: ParmeterName that begins with tagName
WHERE ParameterName.StartsWith( tagName )
Example: ParmeterName that contains with tagName
WHERE ParameterName.Contains( tagName )
There are many examples of using various c# expressions and methods at the official reference site - U-SQL Language Reference. Look under the sub-topic Built-in C# Functions and Operators (U-SQL).
Related
I want to perform dynamic combination (AND OR) search on the basis of the provided parameter by the user.
Search Combination Example:
( (title = "United States" or isbn = "2345371242192") and author ="Jhon" )
In the above query each parameter will look on their XPATH e.g. (item/tigroup/title, item/isbn), XPATH not provided by the user, i have to generate XPATH dynamically with search combination
How Combination query can be formed dynamically to pass it to the BaseX?
User can perform any kind of AND OR search, their can be multiple AND OR criteria
Any suggestions much appreciated
With xquery:eval, strings can be evaluated as XQuery expression (see the documentation for more examples):
declare variable $QUERY := 'text()';
db:open('db')//*[xquery:eval($QUERY, map { '': . })]
Please note that it’s very risky to evaluate arbitrary strings as XQuery code. If the string contains user input, malicious strings may be passed on that do unexpected things. In the example above, a malicious string could be a file operation (e.g., file:delete(.)), or a query that runs very long and blocks your system.
So, I faced an interview recently with a well known company on Marklogic. He has asked me a question which I couldn't answer. There is an XML example data as below shown.
He asked me how can you get only employee id whose zipcode is 12345 and state is california using search? like cts:search
The thing which came into my mind is write XPath like below but since he asked me using search I couldn't answer
let $x :=//employee/officeAddress[zipCode="38023"]/../employeeId/string()
return $x
xml dataset:
<employees>
<employee>
<employeeId>30004</employeeId>
<firstName>crazy</firstName>
<lastName>carol</lastName>
<designation>Director</designation>
<homeAddress>
<address>900 clean ln</address>
<street>quarky st</street>
<city>San Jose</city>
<state>California</state>
<zipCode>22222</zipCode>
</homeAddress>
<officeAddress>
<address>000 washington ave</address>
<street>bonaza st</street>
<city>San Francisco</city>
<state>California</state>
<zipCode>12345</zipCode>
</officeAddress>
<employee>
</employees>
Using XPath is a natural initial thought for many familiar with XML technologies and starting with MarkLogic. It was what I first started to do when I was just starting out.
Some XPath expressions can be optimized by the database and perform fast and efficiently, but there are also others that cannot and may not perform well.
Using cts:search and the built-in query constructs allows for optimized expressions that will leverage indexes, and allows you to further tune by analyzing xdmp:plan, xdmp:query-meters, and xdmp:query-trace.
An equivalent cts:search expression for the XPath, specifying the path to /employees/employee in the first $path parameter and combining cts:element-value-query with cts:and-query in the second $query parameter would be:
cts:search(/employees/employee,
cts:and-query((
cts:element-value-query(xs:QName("zipCode"), "12345"),
cts:element-value-query(xs:QName("state"), "California") )))/employeeId
You could also use a more generic $path to search against all documents and use an xdmp:element-query() to surround the cts:element-value-query criteria to restrict the search to descendants of the employee element and then XPath into the resulting document(s):
cts:search(doc(),
cts:element-query(xs:QName("employee"),
cts:and-query((
cts:element-value-query(xs:QName("zipCode"), "12345"),
cts:element-value-query(xs:QName("state"), "California") ))
)
)/employees/employee/employeeId
xpath I would have tried (not tested):
/employees/employee[officeAddress/zipCode = '38023' and officeAddress/state = 'California']/employeeId/string()
Note that you can use xdmp:plan on xpath too; it's interesting to see how it works vs cts:search.
In general you're better off putting as much into cts:search as possible vs xpath (and I like xpath!).
The question is a little ambiguous. Are there many employees in one document? Or many employees documents? Both?
Also, don't forget to add the appropriate position indexes, or you won't get much unfiltered help. Look at the plan before and after adding the indexes.
See also https://help.marklogic.com/Knowledgebase/Article/View/queries-constrained-to-elements
I have created a virtual table and am executing the following query that returns nothing:
SELECT * FROM table_search WHERE name MATCH 'Test'
If I change the MATCH to '=' or 'LIKE' then the query returns a row of data.
As the documentation explains:
The MATCH operator is a special syntax for the match()
application-defined function. The default match() function
implementation raises an exception and is not really useful for
anything. But extensions can override the match() function with more
helpful logic.
If you haven't defined such a function, then it will select no rows. If you have defined such a function, then explain that in your question. Otherwise, stick with like, =, or regexp.
I am wondering what changed since Gordon Linoff's answer, because he is obviously quite the SQL expert and I'm not. Just talking about my own experience:
Don't specify the column but the table name for FTS queries:
SELECT * FROM table_search WHERE table_search MATCH 'Test'
I did the same mistake and even tried the implementation of a custom MATCH function from this repository after reading the answer from Gordon Linoff and the doc.
The MATCH operator is a special syntax for the match() application-defined function. The default match() function implementation raises an exception and is not really useful for anything. But extensions can override the match() function with more helpful logic.
Contrary, I found out the MATCH function works out of box (using sqlite 3.39) if I used the table name, and an asterix 'Test*' but that was just because of my specific dataset and a wrong assumption.
The custom function is not being called by the MATCH, but it is called if I applied it to a column like
SELECT match(name) FROM table_search WHERE table_search MATCH 'Test'
which is a very different intention.
I'm trying to perform a SQLite FTS query with untrusted user input. I do not want to give the user access to the query syntax, that is they will not be able to perform a match query like foo OR bar AND cats. If they tried to query with that string I would want to interpret it as something more like foo \OR bar \AND cats.
There doesn't seem to be anything built in to SQLite for this, so I'll probably end up building my own escaping function, but this seems dangerous and error-prone. Is there a preferred way to do this?
The FTS MATCH syntax is its own little language. For FTS5, verbatim string literals are well defined:
Within an FTS expression a string may be specified in one of two ways:
By enclosing it in double quotes ("). Within a string, any embedded double quote characters may be escaped SQL-style - by adding a second double-quote character.
(redacted special case)
It turns out that correctly escaping a string for an FTS query is simple enough to implement completely and reliably: Replace " with "" and enclose the result in " on both ends.
In my case it then works perfectly when I put it into a prepared statement such as SELECT stuff FROM fts_table WHERE fts_table MATCH ?. I would then .bind(fts_escape(user_input)) where fts_escape is the function I described above.
OK I've investigated further, and with some heavy magic you can access the actual tokenizer used by SQLite's FTS. The "simple" tokenizer takes your string, separates it on any character that is not in [A-Za-z0-0], and lowercases the remaining. If you perform this same operation you will get a nicely "escaped" string suitable for FTS.
You can write your own, but you can access SQLite's internal one as well. See this question for details on that: Automatic OR queries using SQLite FTS4
SQLite doesn't really have date columns. You can store your dates as ISO-8601 strings, or as the integer number of seconds since the epoch, or as Julian day numbers. In the table I'm using, I want my dates to be human-readable, so I've chosen to use ISO-8601 strings.
Suppose I want to query all the records with dates after today. The ISO-8601 strings will sort properly, so I should be able to use string comparison with the ISO-8601 string for today's date.
However, I see no way to do the comparison using the F# SqlProvider type provider. I'm hoping that this is just a reflection of my lack of knowledge of F# query expressions.
For instance, I can't do:
query {
for calendarEntry in dataContext.``[main].[calendar_entries]`` do
where (calendarEntry.date >= System.DateTime.Today.ToString("yyyy-MM-dd hh:mm:ss"))
... }
I get:
The binary operator GreaterThanOrEqual is not defined for the types 'System.String' and 'System.String'.
I also can't do any variation of:
query {
for calendarEntry in dataContext.``[main].[calendar_entries]`` do
where (calendarEntry.date.CompareTo(System.DateTime.Today.ToString("yyyy-MM-dd hh:mm:ss")) >= 0)
... }
I get:
Unsupported expression. Ensure all server-side objects appear on the left hand side of predicates. The In and Not In operators only support the inline array syntax.
Anyone know how I might do string comparisons in the where clause? It seems that my only option for filtering inside the query is to store seconds-since-epoch in the database and use integer comparisons.
This was a temporary bug with old SQLProvider version and it should be working now. If not, please open a new issue to the GitHub repository: https://github.com/fsprojects/SQLProvider