I'm implementing search for an Android app using an SQLite db, and am wanting to order the results from a search according to the number of matches in a TEXT column
For example let's say that my db table is called article, the TEXT column I want to search is called article_main_text, and that the the user searches for the word "friend". Then I want the db table rows which have the highest number of occurrences of "friend" in their article_main_text column to be shown first
I already know that I can provide custom SQLite statements in the sort order in my CursorLoader (for example supplying this to the CursorLoader constructor), so I'd like to ask: How can I write SQLite code that orders the results with the help of SQLite?
Counting just the occurrences of a word might not be the most accurate or most efficient way of doing it. What you really need is full text search. Which is supported in sqlite and available in Android.
Fortunately there is a tutorial on Android Developer and I am sure there would be a full working sample in their code collections.
Related
Situation: MS Access (happens to be 2010) using SQLite ODBC driver (0.997) to link to tables in a SQLite (3.x) database.
Problem: data values in all columns in all rows display as "#Deleted".
Solution: This is a "answer my own question" kind of post, with a solution, below.
Edited: to move solution to Answers section.
Earlier, I searched in stackoverflow, found a similar question (sqlite linked tables in Access give #deleted values) with a good answer that turns out to be inapplicable in my case. So I'm adding some info here.
Half of the problem is explained here: http://support.microsoft.com/kb/128809 '"#Deleted" errors with linked ODBC tables.'
The above link was no longer available in Jul-2021. However you may find a good explanation for '#DELETED# Records Reported by Access' in https://dev.mysql.com/doc/connector-odbc/en/connector-odbc-errors.html
This explains that Access (Jet) wants a table to have a unique index in order to be able to insert/update the table if necessary.
If your SQLite table doesn't have a unique index (or primary key), then Access will only allow read access to the table -- you can't edit the table's data in Access, but the data displays fine.
To make the table updateable you might revise your SQLite code (or using a SQLite tool) to add an index to the table.
If your PK/unique index happens to use a TEXT field, that's fine for SQLite. However, when you link to it in Access, Access will show the #Deleted indications.
The chain of events appears to be:
Access/Jet notices the unique index, and tries to use it. However, SQLite TEXT fields are variable length and possibly BLOBs. This apparently doesn't fulfill Access's requirements for a unique index field, hence the #Delete indication.
To avoid that problem, the index has to be a SQLite field type that Access will accept. I don't know the complete list of types that are acceptable, but INTEGER works.
Hope this helps someone.
We have a website which is running on DNN 7.1 with SQL server. We implemented full text search to show search results. We need to search several tables and show the results to the user. Right now the implementation is user enters search word(s) and clicks search, the code behind creates several threads to search different tables, and merge the data. Currently we are using contains predicate, issue with this is, there is no ranking and sometimes after the merge the results on the first page are not the best matches. I thought that I can use containstable and order the results by ranking but I read ranking doesn't have any meaning by itself, it merely tells which one best matches in the current resultset. But in my scenario I have multiple resultsets, how will I know which are best matches across multiple resultsets. Or am I going about this wrong way? What is a good way to handle this scenario? We need to improve the response time along with better results. Any help is greatly appreciated.
this is how we implemented full text searching across multiple tables:
1) create a new table to store the primary keys of the other tables in each column, another column to store the string concatenated values of all the search fields from each table, and another column to store the checksum value of the concatenated values.
2) implement the FTI on this new table, and create a job that regularly syncrhonizes/updates the concatenated search values only if the binary_checksum value is different
3) use the contains predicate on this new table and based on the results, join back to their corresponding tables based on the primary keys returned.
I am trying to run a search query on my SQLite db and am having problems with special characters that are stored.
I have a column called site_name which contains records like castle, chàteau, church. When someone uses chateau as their search term I want it to pull out the chàteau record.
Is there a method for handling this in SQLite?
Thanks
See here
The link references Android development, but it appears to answer your question.
I'm looking at implementing tags in my ASP.NET website. After looking at several algorithms, I'm leaning towards having a couple of database columns that contain one or more tag words. I will then use full-text search to locate rows with specified tags.
All of this seems pretty straight forward except for one thing: I need to be able to generate a list of available tags, which the user can select from.
I know I can write a C# program to build the list of available tags, and then run it once every week or so, but I was just wondering if there's any SQL-method for doing stuff like this more efficiently.
Also, I can't help but notice that the words will be extracted anyway as part of building the full-text index. I don't suppose there's any way to access that information?
This isn't how I'd choose to structure this but to answer the actual question...
In SQL Server 2008 you can query the sys.dm_fts_index_keywords and sys.dm_fts_index_keywords_by_document table valued functions to get the information that you want.
Why not to use separate table for tags with many-to-many relationship with tagged items table?
I mean something like that:
--Articles
ArticleId
Text
--Tags
TagId
Name
--TagsToArticles
ArticleRef
TagRef
Here's what I'm trying to do.
User (a):
Enters data in two fields (description-1) and (description-2).
User (b) Enters similar data in opposite fields.
User (a) or (b) search on both fields would find a match.
A good analogy would be a dating search. User (a) enters a description of themselves and the match they are looking for, and User (b) enters a description of themselves and the match they are looking for and Both would be able to do a search and find a match.
So in psuedo query english...
Select name from data where me = 'target' and target = 'me'
The catch would be, some of the words in the field would match, but not all.
This type of matching is hard no matter what the technology. You may have bitten off more than you can chew.
My recommendation to you is to read up on the Text Search data types in PostgreSQL.
PostgreSQL offers a flexible and powerful solution for full-text search, and it may do what you need, whereas SQLite probably won't.
Using the PostgreSQL tsquery and tsvector data types, you could turn one user's description into a form that queries the description of another user. Both tsquery and tsvector can be generated dynamically or saved in database columns and indexed.
If you still need to use SQLite, you need to learn about the various FTS virtual table types. These are all experimental and are not enabled by default. So you need to recompile SQLite, enabling FTS1, FTS2, or FTS3.
Documentation for these features is pretty sparse. Here's all I have found:
http://www.sqlite.org/cvstrac/wiki?p=FullTextIndex
http://www.sqlite.org/cvstrac/wiki?p=FtsUsage
http://www.sqlite.org/cvstrac/wiki?p=FtsOne
http://www.sqlite.org/cvstrac/wiki?p=FtsTwo
http://www.sqlite.org/cvstrac/wiki?p=CompilingFts
http://www.sqlite.org/cvstrac/wiki?p=CompilingFtsThree