SQLite, text search FTS etc - sqlite

I am doing some updates and repairs on some old D5 business stuff (freebie for friend) with SQLite3 so have to use Zeos 6.6
I would like to speed up a text search which currently uses two BLOB_TEXT Fields, "Class" and "Methods"
if not(cbSearchMatchCase.Checked) then
SearchText:=UpperCase(SearchText);
while not(dm.tbl.EOF) do
begin
SearchData:=dm.tbl.FieldByName(fldClass).AsString+' '+
dm.tbl.FieldByName(fldMethods).AsString;
if not(cbSearchMatchCase.Checked) then
SearchData:=UpperCase(SearchData);
MatchFound:=AnsiPos(SearchText,SearchData) > 0;
dm.tbl.Edit;
dm.tbl.FieldByName(fldSearch).AsBoolean:=MatchFound;
dm.tbl.Post;
pbMain.StepIt;
dm.tbl.Next;
end;
Can anyone point me at some code for using the SQLite FTS extension or something quicker than the above?
I found this but can't get it to work for way too many reasons to define here. I am sure it works as Žarko Gajić stuff is always spot-on, but just not working this installation. :)
http://zarko-gajic.iz.hr/full-text-txt- ... g-project/
Just find this link here...
Does BLOB data types in SQLite supports FULL-text search?
That might explain why nothing seems to work, The BLOB_TEXT is being ignored? Yes? No?
Any snippets to speed this Text Search up?

Why not to create index on class / method? It can be expression index: https://www.sqlite.org/expridx.html or even complex one. I would start with lower case, and then re-filter if case sensitivity is needed.

Related

Is there a way to import multiple enumerands in IBM Rhapsody?

I have an enumerand of around 150 entries, which I need to get into IBM Rhapsody.
Doing this by hand is clearly lengthy and error prone. I have google extensively but found only things that tell me how to edit the generated code -- not go the other way.
The question is: How is this done? And if there is no way -- please someone post that as an answer.
David,
I would jump into the Java API (plugin subsystem) and do it that way. If you haven't learned how to use the API, there is a bit of a learning curve. There are two ways to go about it: Implement a Java (or your favorite JVM language--I use Scala) app that realizes the Rhapsody Plugin framework and then you choose to package it up and deploy it so that it gets loaded when you load your model, or, if it is a one off job, do everything up to the point of packaging it up and then run it from within your IDE and you are done. If you are comfortable with Scala, I can post some code.
So what I did in the end was I edited the relevant .sbs file, used a small python program to generate the items I required, and then update the length of the array accordingly.
all_the_literals = ["enum_name = 0x4e", enum_name2 = 0xF2", ... ,]
for field1, waste, field1_value in map(lambda x: x.split(" "),
all_the_literals):
literal_string = f""" {{ IEnumerationLiteral
- _id = GUID {uuid.uuid4()};
- _name = \"{field1}\";
- codeUpdateCGTime = 5.16.2022::19:24:18;
- _modifiedTimeWeak = 5.16.2022::19:24:18;
- _value = \"{field1_value}\";
}}"""
print(literal_string)
Note the above "code" snippet purely prints the items, which you then copy-paste into the relevant field in the sbs file. YMMV -- this was the correct format for an enum in Rhapsody (and note how I fudged the update time, but it worked successfully, so you'll need to do the same if you use this answer).
Also note it's probably better to use bauhaus9's answer, but I definitely didn't have time for it.

Custom Injections in Neovim Treesitter with Tagged Template Literals

I'm trying to write a custom injection for Neovim's Treesitter that will highlight tagged template literals as SQL code.
The code in question looks like this:
import sql from "postgres"
const query = sql` SELECT * FROM my_table`
I'd like to tell Treesitter to parse this tagged template literal as SQL code. This is possible using Treesitter's custom injections; in fact the library uses this same technique to inject syntax for GraphQL tagged template literals, which I believe happens in this file here.
I've written a configuration file at ~/.config/nvim/after/queries/ecma/injections.scm that's quite similar to the GraphQL query, and although Treesitter is recognizing the file, the injection isn't working at all. Does anyone have a solution to this? Ideally, Treesitter would see the sql tagged template literal and interpret the body of the function with the SQL parser.
to make tree sitter use injections in your after/ folder, your injections.scm file should contain ; extends as first line.
Took me many hours to figure it out, it's now documented right here
Similar to what #brotify-force said, but my own SQL injection for Python was not working until I realized what the issue was.
~/.config/nvim/after/queries/python/injections.scm
; extends
(assignment
left: (identifier) #_id (#match? #_id "sql")
right: (string) #sql
)
My original query used #id, but it seems like if you have any captures not starting with an underscore besides the language capture, the Neovim Treesitter machinery gets confused.
I've been trying to do something similar but for template strings that just happen to contain SQL. I was inspired by a VSCode plugin that syntax highlights any template string that has a comment like /*sql*/ in front of it. Here's what I came up with.
; extends
(
(comment) #_comment-text
(#match? #_comment-text "sql")
(template_string) #ql
)
I do wish there was a proper sql module for tree-sitter, and it looks like one is in the works, but the ql module will do for now.

Which tool of Progress Developer Studio for OpenEdge can I use for testing ABL queries?

I've just written following erroneous ABL query:
FOR EACH T1 (WHERE T1.some-date = ' ') AND
(integer(T1.num1) >= 100) AND
(integer(T1.num1) <= 200) AND
(T1.some-other-date = 01/12/2021) AND
(T2.Name = itsme),
EACH T2 WHERE T2.num2 = T1.num2
BY T1.num1
As you can see, this is wrong because I've put the first bracket in front of "WHERE" instead of behind it. In top of that, my name "itsme", is not put between quotes, so the ABL query will never work.
I've been looking in my development environment ("Tools" menu), but I couldn't find an ABL query tester. I've also checked the directory "C:\Progressx86\OpenEdge\bin", but being a newbie I didn't find anything.
I have downloaded the "DataDigger" application, which contains a so-called "MCF Query Tester", but this only works on single table and only checks criteria, not entire ABL queries.
Does anybody know where I can find an ABL query tester, first for syntax checking (the bracket in front of the "WHERE") and (if possible) for data testing (01/12/2021, is that January 12th or December 1st?)?
Thanks in advance
Dominique
Create a new OpenEdge project in Progress Developer Studio for Openedge. Create a new ABL procedure under the project with the necessary database connection. Copy the above ABL code into the procedure file and you should be able to see the errors and warnings in your procedure file.
The ABL Scratch Pad view of Progress Developer Studio allows to execute ad-hoc queries.
https://knowledgebase.progress.com/articles/Knowledge/000055088
You don't mention which editor you're using, but in general terms, the ABL COMPILE statement performs syntax checking. There's no separate/independent executable that compiles/checks syntax. Generally you'll need to write a .P to perform the compilation.
You can use the PCTCompile Ant task if you'd like to use an Ant- or Gradle-based build system.
As to date formats, I thought that was in the doc, but no. Dates always have a MM/DD/YYYY format when hard-coded; decimals always use a . decimal separator (ie 123.45) when hard-coded.
Another way to test a query you're working on would be to use query-prepare which accepts a string.
DEFINE QUERY q-Test FOR T1, T2.
QUERY q-test:HANDLE:QUERY-PREPARE("your query string here").
One of my colleagues showed me an incredible easy solution:
Copy the query in a separate procedure window and add the results you want to see, something like this:
FOR EACH T1 (WHERE T1.some-date = ' ') AND
(integer(T1.num1) >= 100) AND
(integer(T1.num1) <= 200) AND
(T1.some-other-date = 01/12/2021) AND
(T2.Name = itsme),
EACH T2 WHERE T2.num2 = T1.num2
BY T1.num1
/* To be added for viewing results */
DISPLAY T1.Field1 T1.Field2 T2.Field5
END.
And launch this inside the programming environment (F2).
In case of syntax mistakes, the compiler shows the errors. In case the syntax is correct, a pop-up window is started, showing the results.

Is there a way or a plugin to configure SQL Dialects as DQL in PhpStorm?

In PhpStorm, I can change global, project or directories settings with existing SQL Dialects, but is there a way to configure SQL Dialects as Symfony/DQL in PhpStorm or a way to detect that App:Panel is a valid entity, not a table? (App:Panel table name is te_panel)
I read this answer which explains that we have to add a Java plugin, because it's currently not possible to add a new SQL Dialect on PhpStorm.
As example, this is an error that PhpStorm is throwing:
The : between App and Panel is not understood. It cannot understand the table name provided (because I provide the name of the Symfony entity).
DQL is not supported.
https://youtrack.jetbrains.com/issue/WI-9948 -- watch this ticket (star/vote/comment) to get notified on any progress.
You may try and treat App:Panel as placeholder (similar to how it was described in that linked question). But I have no ideas if it will help (have not really worked with Symfony/DQL so cannot test it myself).
What I may suggest though -- threat the whole query as plain text. Yes, no syntax highlighting and stuff but will not show errors either.
How? One way by placing special comment just before the string, e.g.
->query(/** #lang text */'SELECT ...');
Or by disabling Language Injection rule for SQL altogether.
Alternatively try what has been suggested in this comment -- custom SQL detection syntax(?): https://gist.github.com/willemnviljoen/d20ad8ad0cc365a7e80744328246610f

SQL Search Statement like Google?

Is there some kind of SQL Statement that I can used to do a search on 1 Column in my table that is similar to what I am looking for.
Like if I am looking for something to do with a "Car" but I spell it as "Kar" it must return items of "car".
Or
If I am looking for "My Company" and I spell it as "MyCompany" it must still retun "My Company".
Select * from TableName where Column Like '%company%' will return "My Company" but I need more as the user will not always know how to spell. So I need something more advanced like some small Google App or something...
That feature is in the text services so if you build a full-text search index, you should be able to use the feature.
Have a look here:
http://msdn.microsoft.com/en-us/library/ms187384.aspx
http://msdn.microsoft.com/en-us/library/ms142571.aspx
This is quite an involved problem. The quick answer is to use the SQL Server soundex algorithm, but it's pretty hopeless. Try the suggestions on this SO answer. (Updated)
Read this blog post: http://googlesystem.blogspot.com/2007/04/simplified-version-of-googles-spell.html
This is something you could implement with SQL, but it's not a built in feature.
Another way to help you users find what they are looking for is to implement type-ahead on the search field. If the user type: "my" he will get "My Company" as a suggestion and likely go with that.
You can easily implement type ahead using jquery or other javascript libraries. Here's a list of type ahead plugins for jQuery: http://plugins.jquery.com/plugin-tags/typeahead
No. A full text index might get you closer, depending on your requirements (what exact features are you looking for?) One option would be roll you own .NET assembly with the desired features add it (CREATE ASSEMBLY) to sql server and use that to search.

Resources