XQuery : Search for a combination of different letters - xquery

Question

I have following xml structure
<image>
<id>88091942</id>
<imageType>Primary</imageType>
<format>pdf</format>
<status timestamp="2019-11-20T12:20:02.616Z">Accepted</status>
<size/>
<languageCode>
<val>eng</val>
</languageCode>
<comments/>
<effectiveDate>2013-01-01T00:00:00.000Z</effectiveDate>
<extractedText> Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, ABCDE remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including B.C.D.E versions of Lorem Ipsum.</extractedText>
</image>
I am trying to search ABCDE in extractedText. The text needs to be searched can be in any format for eg. BCDE , A.B.C.D.E. , Abcde , B.C.D.E or any combination. If any of the combinations are present then result should return text otherwise empty string
Below is the code snippet that I am trying to use:-
let $id := $image/cd:id/string()
let $text := $extractedText[contains(., "*ABCDE*" OR "*A.B.C.D.E.*&quot OR "ABCDE01"")]/string()
return fn:string-join(($id,$text),"!!!!")
I get the following result -
88091942
where as I should be getting -
88091942!!!!Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, ABCDE remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including B.C.D.E versions of Lorem Ipsum.
Any help is much appreciated.

Answer 1

The original requestor has mentioned that the content resides within MarkLogic. Therefore, I will give a MarkLogic-Centric response for the use-cases based on the information available. Tuning for a full implementation including case sensitivity may take more understanding of the underlying features of MarkLogic:
Case: "A.B C", "A B D.C" etc
Under the hood, all content is indexed. The word indexes tokenize content and apply rules about case sensitivity and diacritic sensitivity when storing the content. Word boundaries are based on a sane set of defaults related to whitespace and punctuation marks. That means that the data is already well prepared for the items with some separation of characters. We can see this my Analyzing a sample of the above:
xdmp:describe(cts:tokenize("A.B D.C"))
Shows that spaces are ignored and punctuation is understood based on the sample, results are:
(cts:word("A"), cts:punctuation("."), cts:word("B"), ...)
This means that we would need to simply take some more item into account the relation of each word to one-another in the text. For that, we ensure that the words (A,B,C,D) are near each-other. For this, the setting in the database called word positions may help for performance. For me, I left it off for my sample of 300k docs. Our query is then as simple as:
cts:search(doc(), cts:near-query(cts:word-query(("A", "B", "C", "D")), 1))
Breaking it down:
doc() is just the easiest searchable expression - for JS, you would not have this.
Then we ask for word queries on the words A,B,C,D. The inner working kings of cts:word-query() will already expand this into a list of or-queries.
This is all constrained by position - the word-query results should be within one position of each-other
MarkLogic has many features. For the above - where I had whitespace and puncuation available, I simply used the word search features OOTB.
Case: "ABCD"
This use-case is completely different and is related to a full word. I could start doing some heavy work by indexing the system down to single character wildcards. That would probably work, but be expensive. Instead, I think of it differently. The sample looks like there is a finite combination. If that is the case, the fastest solution would probably be to calculate the permutations of ABCD, ACBD, etc in advance and feed them to a word or value query.
You could also marry that approach with a lexicon on the element and expand terms to only those present in the system - and still just pass them as a sequence into a search.

Answer 2

contains(., "A OR B OR C") is searching for the literal string "A OR B OR C".
You want contains(., "A") or contains(., "B") or contains(., "C").
Alternatively, you can reformulate that as matches(., "A|B|C").
Or if the strings are related, like ABCDE , A.B.C.D.E. , Abcde , A.B.C.D.E, then you could try something like
contains(. => upper-case() => translate('.', ''), "ABCDE")

Answer 3

Your examples are not very clear but it looks like you want this type of RegExp in your XQuery:
let $id := /image/id
let $text := /image/extractedText[matches(.,'((a|A)\.)?(b|B)\.(c|C)\.(d|D)\.(e|E)')]
return fn:string-join(($id,$text),"!!!!")
Output:
88091942!!!! Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, ABCDE remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including B.C.D.E versions of Lorem Ipsum.
Test in here
If you want an XPath/XQuery' specific RegExp flavor, you might use flags as well like in matches(.,'(A\.)?B\.C\.D\.E', 'i') for ignoring case.