I've got a query builder that's been built in house which is using a full text index in order to perform description searches.
The query is built and parametrized and I was wondering the best way to encode the form field from the website in order to pass search strings such as:
Covered by
"red" near "yellow"
red" fish
Thanks
If you want to use full text search you should use where clause with other specific functions ( not just = or like ).
#param1 will still be a string (nvarchar eventually); see here:
Querying SQL Server Using Full-Text Search
for example, you query in this way (from MSDN):
USE AdventureWorks2008R2;
GO
DECLARE #SearchWord nvarchar(30)
SET #SearchWord = N'performance'
SELECT Description
FROM Production.ProductDescription
WHERE CONTAINS(Description, #SearchWord);
about special chars and escaping them, just have a look here: SQL Server Full Text Search Escape Characters?
Related
The SQLite FTS5 docs say that search queries such as SELECT ... WHERE MATCH '<query1> NOT <query2>' are supported, but it looks like there's no support for the unary NOT operator.
For example, if I want to search for everything that doesn't match <query>, I cannot use MATCH 'NOT <query>'. I would have to use NOT MATCH '<query>', which is a completely different thing (the FTS5 module never gets to see the NOT operator, as it is outside the quotation marks). Only the text inside the quotation marks is the search query.
I need to find a way to use an unary NOT operator inside the search query. I can't use it outside, because I only get to control the search query text, and not the rest of the SQL statement.
A possible approach I've thought of would be to find a search query that matches anything, and do MATCH '<match_anything> NOT <query>'. However, I've found no way to match everything in a search query.
Can you think of a way to have the behaviour of the unary NOT operator inside the search query?
Try this ..
SELECT * FROM docs
WHERE ROWID NOT IN (
SELECT ROWID FROM docs WHERE content MATCH '<query>'
)
I am not able to search the middle part of string using fulltext search index for eg:there was a string "I like music" i was not able to search for like which is in the middle part of string..
Try LIKE operator.
SELECT
*
FROM
YourTABLE
WHERE
ColumnName LIKE '%like%'
Use Like keyword in query as follows:
select * from tablename where col like '%like%'
Here is the tutorial of like in sql:
http://www.w3schools.com/sql/sql_like.asp
Here is the MSDN:
http://msdn.microsoft.com/en-us/library/ms179859.aspx
Hope its helpful.
Well, in order to still use the fulltext index and to avoid a table scan, try this
select *
from yourTable
where contains (ColumnName,'like')
and (ColumnName not like 'like%' or ColumnName like '_%like%')
However, the query optimizer might decide that a full table scan is more effective.
Check if statistics of your table out of date, especially check the statistics relates to your fulltext index. Since you are using SQl Server 2008, so, you can query DMV to get statistics information of your index or using DBCC to see the statistics detail information
DBCC SHOW_STATISTICS ("[schema].[table]",indexname);
Check the first return result, [Updated], [rows], [rows sampled], if the statistics out of date, or [rows sampled] far less than [rows], which many cause SQL Engine decided to use table scan instead of using your index.
I have an application that allow users to search on multiple columns (prod_name,prod_desc)
So I used full text search like below, but it does not return all the records, for excample I tried to find 'o' character in 2 columns (prod_name,prod_desc)but it can not find for some records.
Also when I do not use wildcard for the 'o' character it can not find any thing while contains means like %o%.
I am a bit confused about full text search.
Please help what is the problem.
CREATE FULLTEXT CATALOG catalog_crashcourse3;
CREATE FULLTEXT INDEX ON products(prod_name,prod_desc)
KEY INDEX pk_products ON catalog_crashcourse3;
SELECT prod_name, prod_desc
FROM products
WHERE CONTAINS((prod_name,prod_desc), '"*o*"');
SQL Server FTS is a word-based search process. When you create a full-text index on a column, the indexing engine crawls the content and breaks it into individual words in a process known as tokenization. The index then stored the word, the primary key of the row it was found in, and the word's position in the content (i.e. is is the first word in the field, the 57th word, or whatever).
When you specify a CONTAINS predicate such as
CONTAINS((prod_name,prod_desc), '"o"');
the SQL Server FTS engine looks for tokens (i.e. words) in its index that are "o". If your content does not have the word "o" in it (which is probably doesn't) then no matches will be found.
As you point out, you can do wildcard searches, where you try and matched patterns in the indexed word. For example, if you specify a predicate such as
CONTAINS((prod_name,prod_desc), '"o*"');
then the search will return all words in the indexed content that start with the letter "o"
FTS is best used when you want to search for groups of words in your indexed content. It can do sophisticated word stemming (such as searching for "ran" and "running" when you specify "run"). It also provides a ranking of the search result content so that you can find the best match. If you just want to search for a specified word in your content and your content is not too large, you may not need FTS. As MikeSmithDev pointed out in the comments, you may be able to just get away with a LIKE clause.
Note added: In response to your comment, if you have a table with 8 columns that you want to search using FTS, then you would create full-text indexes on each of these columns and search them as follows:
CONTAINS(*, '"Word"')
where the asterik indicates that all 8 indexed columns in the table should be included in the search.
You have two issues:
You are using a prefix wildcard *o which Sql Server FTS is
helpless with. It only works with suffix wildcards like word*.
You are using a single-character search term. Single character words
are excluded from the FT index by default, which is a good thing.
Unless specified otherwise, SQL Server associates the system
full-text stoplist by default when creating the index.
To see the default stoplist your database is using behind your back, use this query
Select SysStop.stopword, Langs.name
From sys.fulltext_system_stopwords SysStop
Inner Join sys.fulltext_languages Langs
On Langs.lcid = SysStop.language_id;
If you really want to search for single characters, you can drop and
recreate the FT index using the option WITH STOPLIST OFF, but be prepared
for a lot of noise. See Create FullText Index.
Sqlite3's full text search facility - FTS3 - allows to use MATCH operator for fast full-text search:
SELECT ItemId FROM docs WHERE docs.text MATCH 'linux'
However, it does not support OR operator anywhere in an SQL query where there's a MATCH (source: 1, 2):
SELECT ItemId FROM docs WHERE docs.text MATCH 'linux' OR column=value
error: unable to use function MATCH in the requested context
(Not to be confused with OR operator in the FTS3 query itself, i.e. SELECT ItemId FROM docs WHERE docs.text MATCH 'linux OR unix'. This works fine.)
Is there a way to rewrite the query so that it works (even if it's somewhat slower)?
Rewriting the query using temporary views will work as expected:
CREATE TEMP VIEW view1 AS SELECT ItemId FROM docs WHERE docs.text MATCH 'linux'
SELECT * FROM docs WHERE ItemId IN view1 OR column=value
DROP VIEW view1
The speed will be comparable to that of a "direct" query (without the temporary view) if the temporary view is not "sweeping", i.e. it does not generate a lot of rows.
I've got a search box that users can type terms into. I have a table setup with fulltext searching on a string column. Lets say a user types this: "word, office, microsoft" and clicks "search".
Is this the best way to deal with multiple search terms?
(pseudocode)
foreach (string searchWord in searchTerms){
select col1 from myTable where contains(fts_column, ‘searchWord’)
}
Is there a way of including the search terms in the sql and not iterating? I'm trying to reduce the amount of calls to the sql server.
FREETEXT might work for you. It will separate the string into individual words based on word boundaries (word-breaking). Then you'd only have a single SQL call.
MSDN -- FREETEXT
Well you could just build your SQL Query Dynamically...
string [] searchWords = searchTerm.Split(",");
string SQL = "SELECT col1 FROM myTable WHERE 1=2";
foreach (string word in searchWords)
{
SQL = string.Format("{0} OR contains(fts_column, '{1}')", SQL, word);
}
//EXEC SQL...
Obviously this comes with the usual warnings/disclaimers about SQL Injection etc... but the principal is that you would dynamically build up all your clauses and apply them in one query.
Depending on how your interacting with your DB, it might be feasible for you to pass the entire un-split search term into a SPROC and then split & build dynamic SQL inside the stored procedure.
You could do it similar to what you have there: just parse the search terms based on delimiter, and then make a call on each, joining the results together. Alternatively, you can do multiple CONTAINS:
SELECT Name FROM Products WHERE CONTAINS(Name, #Param1) OR CONTAINS(Name, #Param2) etc.
Maybe try both and see which is faster in your environment.
I use this class for Normalizing SQL Server Full-text Search Conditions