SQLite full text search indexes - sqlite

I currently have a diagnosis table. I want to make the code and description fields searchable using FTS. As I understand it though, FTS tables don't support indexes and I need to be able to lookup Diagnosis by diagnosisID very quickly. Am I going to have to create a second virtual table with all of the data duplicated just for full text searching or am I missing a solution where I dont have to duplicate all of my diagnosis codes and descriptions?
CREATE TABLE Diagnosis (
diagnosisID INTEGER PRIMARY KEY NOT NULL,
code TEXT,
collect INTEGER NOT NULL,
description TEXT
);

Turns out an FTS table has a hidden rowid field, which you can populate when you are entering data:
sqlite> create virtual table test1 using fts3;
sqlite> insert into test1 values ("This is a document!");
sqlite> insert into test1(docid,content) values (5,"this is another document");
sqlite> select rowid,* from test1;
1|This is a document!
5|this is another document
You could create an integer field in your standard table that refers to the FTS table by rowid, and move the columns you wish to make text-searchable into the FTS table.
All the info you need here :)

Related

SQLite Copy Table Content from one Table to another Table and Update

I have two SQLite files, each of them has one table and the same table design. One Column is set as Primary Key. I want to copy all data from ItemsB into ItemsA. All data should be updated. The ItemsB Table is the newer one.
I've tried:
ATTACH DATABASE ItemsB AS ItemsB;
INSERT INTO ItemsA.PMItem (ItemID,VarID,Name1) SELECT ItemID,VarID,Name1 FROM ItemsB.PMItem;
Obviously this can't work due the Primary Key (which is the column VarID).
Then I tried it with ON CONFLICT:
ON CONFLICT (VarID) DO UPDATE SET Name1=excluded.Name1
But this won't work either.
Example Table:
CREATE TABLE PMItem (
ItemID INTEGER,
VarID INTEGER PRIMARY KEY,
Name1 TEXT
);
You need a WHERE clause with an always true condition, to overcome the ambiguity that is raised when ON CONFLICT is used after a SELECT statement:
INSERT INTO PMItem (ItemID,VarID,Name1)
SELECT ItemID,VarID,Name1
FROM ItemsB.PMItem
WHERE 1
ON CONFLICT(VarID) DO UPDATE
SET Name1 = EXCLUDED.Name1;

SQlite FTS table is not being populated

DROP TABLE IF EXISTS TEXTS;
CREATE TABLE TEXTS (text_id int PRIMARY KEY, text ntext,last_update datetime DEFAULT
CURRENT_TIMESTAMP);
DROP TABLE IF EXISTS FTSTEXTS;
CREATE VIRTUAL TABLE FTSTEXTS USING fts5(text,content="TEXTS",content_rowid="text_id",tokenize=unicode61);
Insert into texts (text_id,text) values (1,'first'),(2,'second'),(3,'third'),(4,'fourth');
Insert into ftstexts (rowid,text) values (1,'first'),(2,'second'),(3,'third'),(4,'fourth');
update texts set text='5555' where text_id=2;
update ftstexts set text='5555' where rowid=2;
select text from texts where text_id=(select rowid from ftstexts where text match('second') limit 1);
result is "5555"
After creating table and updating the value of row 2 to "5555" the query
select rowid from ftstexts where text match('second') limit 1
returns 2:
How can the FTS table be populated to exclude the term "second" from fts search ?
The query was run on SQLITEStudio 3.2.1
External content FTS5 tables don't support updates. You have to delete the old contents of a row and insert the new contents as separate steps.
The documentation includes some sample triggers to automate keeping such a table in sync with the table that holds the actual data.
You might make an update trigger something along the lines of
CREATE TRIGGER texts_au AFTER UPDATE ON texts BEGIN
INSERT INTO ftstexts(ftstexts, rowid, text) VALUES('delete', old.text_id, old.text);
INSERT INTO ftstexts(rowid, text) VALUES (new.text_id, new.text);
END;

SQLite Insert into a table with one column that is auto incrament

This should be an easy one. I need the SQL to insert into a table that has only one column and it is and autoincrement field.
Similar to this post but SQLite (I am new to SQLite).
Inserting rows into a table with one IDENTITY column only
create table ConnectorIDs
(
ID integer primary key AUTOINCREMENT
);
--none of the following work
INSERT INTO ConnectorIDs VALUES(DEFAULT);
INSERT ConnectorIDs DEFAULT VALUES;
Yes this is strange and if you care here is the reason, if you want to tell me a better way. I have several different item tables that all can have many-to-many links between them but sparse. Instead of having n! bridge tables, or one bridge table with a "Type" that I can't guarantee truly maps to the correct table. I will have one ConnectorID table and each item with have a connectorID key. Then I can have one bridge table.
Insert a null value:
INSERT INTO ConnectorIDs VALUES(NULL);
From the docs:
If no ROWID is specified on the insert, or if the specified ROWID has a value of NULL, then an appropriate ROWID is created automatically.

SQLite regular table and table for fts

I have table news (id, news_id, news_title) and I creat FTS table:
CREATE VIRTUAL TABLE news_search USING fts4 (news_title, tokenize=porter);
I use trigger to keep table NEWS and news_search in sync:
CREATE TRIGGER IF NOT EXISTS insert_news_trigger
AFTER INSERT ON news
BEGIN
INSERT OR IGNORE INTO news_search (news_title) VALUES (NEW.news_title);
END;
Question: how to use search? When I do MATCH in news_search table it returns me only records from this table, but I need *news_id* from news table. May be I should add *news_id* column to news_search table?
What is the proper way to use fts in sqlite?
Read the documentation; FTS tables also have a rowid column (also called docid) that you can set explicitly to the same value as the corresponding key of the original table.
Assuming that news.id is the rowid (i.e., INTEGER PRIMARY KEY), you should change your trigger to also copy that ID value into the news_search table.
You can the use that to look up the original record:
SELECT *
FROM news
WHERE id IN (SELECT docid
FROM news_search
WHERE news_title MATCH '😸')

Can I alter a column in an sqlite table to AUTOINCREMENT after creation?

Can I make a field AUTOINCREMENT after made a table? For example, if you create a table like this:
create table person(id integer primary key, name text);
Then later on realise it needs to auto increment. How do I fix it, ie in MySQL you can do:
alter table person modify column id integer auto_increment
Is table creation the only opportunity to make a column AUTOINCREMENT?
You can dump the content to a new table:
CREATE TABLE failed_banks_id (id integer primary key autoincrement, name text, city text, state text, zip integer, acquired_by text, close_date date, updated_date date);
INSERT INTO failed_banks_id(name, city, state, zip, acquired_by,close_date, updated_date)
SELECT name, city, state, zip, acquired_by,close_date, updated_date
FROM failed_banks;
And rename the table:
DROP TABLE failed_banks;
ALTER TABLE failed_banks_id RENAME TO failed_banks;
Background:
The new key will be unique over all
keys currently in the table, but it
might overlap with keys that have been
previously deleted from the table. To
create keys that are unique over the
lifetime of the table, add the
AUTOINCREMENT keyword to the INTEGER
PRIMARY KEY declaration.
http://www.sqlite.org/faq.html#q1
SQLite limitations:
SQLite supports a limited subset of
ALTER TABLE. The ALTER TABLE command
in SQLite allows the user to rename a
table or to add a new column to an
existing table. It is not possible to
rename a column, remove a column, or
add or remove constraints from a
table.
http://www.sqlite.org/lang_altertable.html
Hack seems to exist:
It appears that you can set
PRAGMA writable_schema=ON;
Then do a manual UPDATE of the
sqlite_master table to insert an "id
INTEGER PRIMARY KEY" into the SQL for
the table definition. I tried it and
it seems to work. But it is
dangerous. If you mess up, you
corrupt the database file.
http://www.mail-archive.com/sqlite-users#sqlite.org/msg26987.html
From the SQLite Faq
Short answer: A column declared INTEGER PRIMARY KEY will autoincrement
So when you create the table, declare the column as INTEGER PRIMARY KEY and the column will autoincrement with each new insert.
Or you use the SQL statment ALTER to change the column type to an INTEGER PRIMARY KEY after the fact, but if your creating the tables yourself, it's best to do it in the initial creation statement.
Simplest way — Just export and re-import
It is possible, and relatively easy. Export the database as an sql file. Alter the SQL file and re-import:
sqlite3 mydata.db .dump > /tmp/backup.sql
vi /tmp/backup.sql
mv mydata.db mydata.db.old
sqlite3 mydata.db
sqlite>.read /tmp/backup.sql
You can do it with SQLite Expert Personal 4:
1) Select the table and then go to "Design" tab > "Columns" tab.
2) Click "Add" and select the new column name, and type INTEGER and Not Null > Ok.
3) Go to "Primary Key" tab in "Desgin tab". Click "Add" and select the column you just created. Check the "Autoincrement" box.
4) Click "Apply" on the right bottom part of the window.
If you go back to the "Data" tab, you will see your new column with the autogenerated numbers in it.
While the Sqlite site gives you an example how to do it with a table with only a three fields, it gets nasty with one of 30 fields. Given you have a table called OldTable with many fields, the first of which is "ID" filled with integers.
Make a copy of your database for backup.
Using the command program dot commands,
.output Oldtable.txt
.dump Oldtable
Drop Table Oldtable;
Open Oldtable.txt in Microsoft Word or a grep like text editor. Find and Replace your Integer field elements with NULL.(You may need to adjust this to fit your fields). Edit the Create Table line so the field that was defined as Integer is now INTEGER PRIMARY KEY AUTOINCREMENT.
Save as NewTable.txt
Back in the command program dot
.read NewTable.txt
Done.
ID is now autoincrement.
Yes
Do you have phpmyadmin installed? I believe if you go to the 'structure' tab and look along the right columnn (where the field types are listed) - I think you can change a setting there to make it autoincrement. There is also a SQL query that will do the same thing.
You cannot alter columns on a SQLite table after it has been created. You also cannot alter a table to add an integer primary key to it.
You have to add the integer primary key when you create the table.
Yes, you can make a column which is autoincrement. Modify the table and add a column. Keep in mind that it is of type INTEGER Primary Key.
you can alter the table, altering the column definition
Simple Answer is as below,
CREATE TABLE [TEST] (
[ID] INTEGER PRIMARY KEY AUTOINCREMENT,
[NAME] VARCHAR(100));
and you are done.

Resources