The SQLite FTS5 docs say that search queries such as SELECT ... WHERE MATCH '<query1> NOT <query2>' are supported, but it looks like there's no support for the unary NOT operator.
For example, if I want to search for everything that doesn't match <query>, I cannot use MATCH 'NOT <query>'. I would have to use NOT MATCH '<query>', which is a completely different thing (the FTS5 module never gets to see the NOT operator, as it is outside the quotation marks). Only the text inside the quotation marks is the search query.
I need to find a way to use an unary NOT operator inside the search query. I can't use it outside, because I only get to control the search query text, and not the rest of the SQL statement.
A possible approach I've thought of would be to find a search query that matches anything, and do MATCH '<match_anything> NOT <query>'. However, I've found no way to match everything in a search query.
Can you think of a way to have the behaviour of the unary NOT operator inside the search query?
Try this ..
SELECT * FROM docs
WHERE ROWID NOT IN (
SELECT ROWID FROM docs WHERE content MATCH '<query>'
)
Related
Using RODBCext (and Teradata) my SQL query often need to be restricted and is done so with a where statement. However, this is not always required and it would be beneficial to not restrict, but I would like to use a single SQL query. (The actual query is more complex and has several instances of what I'm attempting to apply here)
In order to return all rows, using a wildcard seems like the next best option, but nothing appears to work correctly. For example, the sql query is:
SELECT *
FROM MY_DB.MY_TABLE
WHERE PROC_TYPE = ?
The following does work when passing in a string for proc_type:
sqlExecute(connHandle, getSQL(SQL_script_path), proc_type, fetch = TRUE)
In order to essentially bypass this filter, I would like to pass a wildcard so all records are returned.
I've tried proc_type set to '%', '*'. Also escaped both with backslashes and enclosed with double-quotes, but no rows are ever returned, nor are any errors produced.
You could use COALESCE to do this:
SELECT *
FROM MY_DB.MY_TABLE
WHERE PROC_TYPE = COALESCE(?, PROC_TYPE);
In the event that your parameter is NULL it will choose PROC_TYPE to compare to PROC_TYPE which will return everything.
As for your wildcard attempt you would have to switch over to an operator that can use a wildcard. Instead of =, LIKE for instance. I think you would end up with some oddball edge cases though depending on your searchterm and the data in that column, so the COALESCE() option is a better way to go.
I am using DB Browser for SQLite. The documentation for SQLite's fts3 says "FTS is primarily designed to support Boolean full-text queries". I built a virtual table using fts4 and successfully executed a few WHERE ... MATCH queries. But the following attempts give errors:
SELECT id FROM histsearch WHERE id MATCH ("-1456" IN BOOLEAN MODE);
SELECT id FROM histsearch WHERE NOT EXIST id MATCH ("1457");
Is the problem in DB Browser or in SQLite? How else can I write this query so it will work?
SQLite's full text service (fts3) basically offers Boolean Mode by default, no search modifier needed. DB Browser uses fts's standard query syntax, so NOT is not supported. To exclude a term, do something like
SELECT * FROM indexed WHERE indexed MATCH 'sqlite -database';
Edit: however, you cannot only exclude search terms in fulltext search:
An FTS query may not consist entirely of terms or term-prefix queries with unary "-" operators attached to them.
You'll have to use NOT LIKE for that.
When I execute the following query in a sqlite engine (android or sqlitebrowser) it throws an exception that says unable to use function MATCH in the requested context.
select
a.Body1,
b.Body2
from
tbl1_fts as a,
tbl2_fts as b
where
a.ID = b.ParentID and
(
a.Body1 match('value') or
b.Body2 match('value')
)
-Both tables have fts.
-Using And operator between two matches (instead of OR) runs normally.
How can I fix this or change the query to find rows with above condition?
you can not use OR Operation, just change your Match Keyword.
like
SELECT * FROM docs WHERE docs MATCH 'sqlite OR database';
OR maybe you can use union
SELECT docid FROM docs WHERE docs MATCH 'sqlite AND database'
UNION
SELECT docid FROM docs WHERE docs MATCH 'library';
MATCH as a function would have two parameters:
... WHERE match('value', SomeColumn) ...
However, the usual method of using MATCH is as an operator:
... WHERE SomeColumn MATCH 'value' ...
MATCH has to be used without parentheses.
The AND and OR operators must be in capital letters when used with FTS.
I'm trying to perform a SQLite FTS query with untrusted user input. I do not want to give the user access to the query syntax, that is they will not be able to perform a match query like foo OR bar AND cats. If they tried to query with that string I would want to interpret it as something more like foo \OR bar \AND cats.
There doesn't seem to be anything built in to SQLite for this, so I'll probably end up building my own escaping function, but this seems dangerous and error-prone. Is there a preferred way to do this?
The FTS MATCH syntax is its own little language. For FTS5, verbatim string literals are well defined:
Within an FTS expression a string may be specified in one of two ways:
By enclosing it in double quotes ("). Within a string, any embedded double quote characters may be escaped SQL-style - by adding a second double-quote character.
(redacted special case)
It turns out that correctly escaping a string for an FTS query is simple enough to implement completely and reliably: Replace " with "" and enclose the result in " on both ends.
In my case it then works perfectly when I put it into a prepared statement such as SELECT stuff FROM fts_table WHERE fts_table MATCH ?. I would then .bind(fts_escape(user_input)) where fts_escape is the function I described above.
OK I've investigated further, and with some heavy magic you can access the actual tokenizer used by SQLite's FTS. The "simple" tokenizer takes your string, separates it on any character that is not in [A-Za-z0-0], and lowercases the remaining. If you perform this same operation you will get a nicely "escaped" string suitable for FTS.
You can write your own, but you can access SQLite's internal one as well. See this question for details on that: Automatic OR queries using SQLite FTS4
Sqlite3's full text search facility - FTS3 - allows to use MATCH operator for fast full-text search:
SELECT ItemId FROM docs WHERE docs.text MATCH 'linux'
However, it does not support OR operator anywhere in an SQL query where there's a MATCH (source: 1, 2):
SELECT ItemId FROM docs WHERE docs.text MATCH 'linux' OR column=value
error: unable to use function MATCH in the requested context
(Not to be confused with OR operator in the FTS3 query itself, i.e. SELECT ItemId FROM docs WHERE docs.text MATCH 'linux OR unix'. This works fine.)
Is there a way to rewrite the query so that it works (even if it's somewhat slower)?
Rewriting the query using temporary views will work as expected:
CREATE TEMP VIEW view1 AS SELECT ItemId FROM docs WHERE docs.text MATCH 'linux'
SELECT * FROM docs WHERE ItemId IN view1 OR column=value
DROP VIEW view1
The speed will be comparable to that of a "direct" query (without the temporary view) if the temporary view is not "sweeping", i.e. it does not generate a lot of rows.