SQLite FTS5 Match is returning nothing - sqlite

I have a SQLite3 table:
CREATE TABLE "test" (
"title" TEXT,
"shortdesc" TEXT,
"longdesc" TEXT,
"id" TEXT,
PRIMARY KEY("id")
);
I insert anything in it:
INSERT INTO test (id, title, shortdesc, longdesc) VALUES ("abc", "hello world", "this is a hello world", "a nice hello world article about hello worlds")
Then I create a FTS5 virtual table:
CREATE VIRTUAL TABLE IF NOT EXISTS test_fts USING fts5 (
id,
title,
shortdesc,
longdesc,
content=test
);
So I check data in virtual table:
Everything seems fine... Now I try to use MATCH to find the article:
SELECT * FROM test_fts WHERE test_fts MATCH 'hello'
...and I get nothing in result. Obviously this database I showed is just an example, same thing happens with actual database. I tried on different computers (and different clients), I also checked if FTS5 is enabled and compiled in it with PRAGMA compile_options and ENABLE_FTS5 is there, meaning it's enabled. Same thing happens with FTS3 and 4.
So what am I missing? SQLite version is 3.36.0.

The FTS5 tables still need to be populated. This includes external content cases. From the documentation:
It is still the responsibility of the user to ensure that the contents of an external content FTS5 table are kept up to date with the content table. One way to do this is with triggers. For example:
-- Create a table. And an external content fts5 table to index it.
CREATE TABLE tbl(a INTEGER PRIMARY KEY, b, c);
CREATE VIRTUAL TABLE fts_idx USING fts5(b, c, content='tbl', content_rowid='a');
-- Triggers to keep the FTS index up to date.
CREATE TRIGGER tbl_ai AFTER INSERT ON tbl BEGIN
INSERT INTO fts_idx(rowid, b, c) VALUES (new.a, new.b, new.c);
END;
CREATE TRIGGER tbl_ad AFTER DELETE ON tbl BEGIN
INSERT INTO fts_idx(fts_idx, rowid, b, c) VALUES('delete', old.a, old.b, old.c);
END;
CREATE TRIGGER tbl_au AFTER UPDATE ON tbl BEGIN
INSERT INTO fts_idx(fts_idx, rowid, b, c) VALUES('delete', old.a, old.b, old.c);
INSERT INTO fts_idx(rowid, b, c) VALUES (new.a, new.b, new.c);
END;
So, you could either create the table and triggers before populating the main table, or in your test case, after you create the FTS table you can run a query like this to populate the FTS table for the first time:
INSERT INTO test_fts SELECT * FROM test;
Then your query will work as expected.

Related

Use two columns for FTS4 from sqlite3

I have a table with two columns (ldap, name). I want to be able to full text search any of those columns with the library FTS4. Here I have a couple of statements I'm using to create the virtual table but when I create a statement using Match the result is empty although it should return data.
CREATE VIRTUAL TABLE IF NOT EXISTS sales_rep USING FTS4(ldap,name, content="__sales_rep");
CREATE TRIGGER IF NOT EXISTS __sales_rep___after_insert AFTER INSERT ON __sales_rep BEGIN INSERT INTO sales_rep (ldap, name) VALUES (new.ldap, new.name);END;
I am inserting a row (ldap, name) VALUES ('test', 'Bryan');
But using
SELECT * FROM sales_rep where name MATCH 'Bry';
The result is empty
Inserting data in an external content FTS table requires to provide explicitly a value for the docid, which should be the rowid of the content table.
In your case you need to change the trigger :
CREATE TRIGGER __sales_rep___after_insert
AFTER INSERT ON __sales_rep
BEGIN
INSERT INTO sales_rep (docid, ldap, name)
VALUES (new.rowid, new.ldap, new.name);
END;

Sqlite update and autoincrement value

I want to update the schema of a db. I have copied the auto-generated script, but the last line after each table's script is this:
UPDATE "main"."sqlite_sequence" SET seq = 8 WHERE name = 'table';
The sec value is indeed correct for my installed DB, but it could vary on other installations. So, would it be safe to set it to 0, or should I select it from each installation's table? Or could I just skip this line and run the script without it?
If by "auto-generated" script you mean the full .dump of your database, then it will include the create table statements, and the insert statements, so you probably want the update to be executed along.
If you modify that auto-generated script, then you can obviously change the seq value as necessary.
Here is what the documentation has to say:
SQLite keeps track of the largest ROWID that a table has ever held
using the special SQLITE_SEQUENCE table. The SQLITE_SEQUENCE table is
created and initialized automatically whenever a normal table that
contains an AUTOINCREMENT column is created. The content of the
SQLITE_SEQUENCE table can be modified using ordinary UPDATE, INSERT,
and DELETE statements. But making modifications to this table will
likely perturb the AUTOINCREMENT key generation algorithm. Make sure
you know what you are doing before you undertake such changes.
In the end, you need to make sure that the seq value matches the highest value. This demonstrates:
sqlite> create table foo (a INTEGER PRIMARY KEY AUTOINCREMENT, b text);
sqlite> insert into foo values (NULL, 'blabla');
sqlite> select * from foo;
1|blabla
sqlite> .dump
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE foo (a INTEGER PRIMARY KEY AUTOINCREMENT, b text);
INSERT INTO "foo" VALUES(1,'blabla');
DELETE FROM sqlite_sequence;
INSERT INTO "sqlite_sequence" VALUES('foo',1);
COMMIT;

SQLite full text search indexes

I currently have a diagnosis table. I want to make the code and description fields searchable using FTS. As I understand it though, FTS tables don't support indexes and I need to be able to lookup Diagnosis by diagnosisID very quickly. Am I going to have to create a second virtual table with all of the data duplicated just for full text searching or am I missing a solution where I dont have to duplicate all of my diagnosis codes and descriptions?
CREATE TABLE Diagnosis (
diagnosisID INTEGER PRIMARY KEY NOT NULL,
code TEXT,
collect INTEGER NOT NULL,
description TEXT
);
Turns out an FTS table has a hidden rowid field, which you can populate when you are entering data:
sqlite> create virtual table test1 using fts3;
sqlite> insert into test1 values ("This is a document!");
sqlite> insert into test1(docid,content) values (5,"this is another document");
sqlite> select rowid,* from test1;
1|This is a document!
5|this is another document
You could create an integer field in your standard table that refers to the FTS table by rowid, and move the columns you wish to make text-searchable into the FTS table.
All the info you need here :)

How to merge N SQLite database files into one if db has the primary field?

I have a bunch of SQLite db files, and I need to merge them into one big db files.
How can I do that?
Added
Based on this, I guess those three commands should merge two db into one.
attach './abc2.db' as toMerge;
insert into test select * from toMerge.test
detach database toMerge
The problem is the db has PRIMARY KEY field, and I got this message - "Error: PRIMARY KEY must be unique".
This is the test table for the db.
CREATE TABLE test (id integer PRIMARY KEY AUTOINCREMENT,value text,goody text)
I'm just thinking off my head here... (and probably after everybody else has moved on, too).
Mapping the primary key to "NULL" should yield the wanted result (no good if you use it as foreign key somewhere else, since the key probably exists, but has different contents)
attach './abc2.db' as toMerge;
insert into test select NULL, value, goody from toMerge.test;
detach database toMerge;
actual test:
sqlite> insert into test select * from toMerge.test;
Error: PRIMARY KEY must be unique
sqlite> insert into test select NULL, value, goody from toMerge.test;
sqlite> detach database toMerge;
I'm not 100% sure, but it seems that I should read all the elements and insert the element (except the PRIMARY KEY) one by one into the new data base.

Can I alter a column in an sqlite table to AUTOINCREMENT after creation?

Can I make a field AUTOINCREMENT after made a table? For example, if you create a table like this:
create table person(id integer primary key, name text);
Then later on realise it needs to auto increment. How do I fix it, ie in MySQL you can do:
alter table person modify column id integer auto_increment
Is table creation the only opportunity to make a column AUTOINCREMENT?
You can dump the content to a new table:
CREATE TABLE failed_banks_id (id integer primary key autoincrement, name text, city text, state text, zip integer, acquired_by text, close_date date, updated_date date);
INSERT INTO failed_banks_id(name, city, state, zip, acquired_by,close_date, updated_date)
SELECT name, city, state, zip, acquired_by,close_date, updated_date
FROM failed_banks;
And rename the table:
DROP TABLE failed_banks;
ALTER TABLE failed_banks_id RENAME TO failed_banks;
Background:
The new key will be unique over all
keys currently in the table, but it
might overlap with keys that have been
previously deleted from the table. To
create keys that are unique over the
lifetime of the table, add the
AUTOINCREMENT keyword to the INTEGER
PRIMARY KEY declaration.
http://www.sqlite.org/faq.html#q1
SQLite limitations:
SQLite supports a limited subset of
ALTER TABLE. The ALTER TABLE command
in SQLite allows the user to rename a
table or to add a new column to an
existing table. It is not possible to
rename a column, remove a column, or
add or remove constraints from a
table.
http://www.sqlite.org/lang_altertable.html
Hack seems to exist:
It appears that you can set
PRAGMA writable_schema=ON;
Then do a manual UPDATE of the
sqlite_master table to insert an "id
INTEGER PRIMARY KEY" into the SQL for
the table definition. I tried it and
it seems to work. But it is
dangerous. If you mess up, you
corrupt the database file.
http://www.mail-archive.com/sqlite-users#sqlite.org/msg26987.html
From the SQLite Faq
Short answer: A column declared INTEGER PRIMARY KEY will autoincrement
So when you create the table, declare the column as INTEGER PRIMARY KEY and the column will autoincrement with each new insert.
Or you use the SQL statment ALTER to change the column type to an INTEGER PRIMARY KEY after the fact, but if your creating the tables yourself, it's best to do it in the initial creation statement.
Simplest way — Just export and re-import
It is possible, and relatively easy. Export the database as an sql file. Alter the SQL file and re-import:
sqlite3 mydata.db .dump > /tmp/backup.sql
vi /tmp/backup.sql
mv mydata.db mydata.db.old
sqlite3 mydata.db
sqlite>.read /tmp/backup.sql
You can do it with SQLite Expert Personal 4:
1) Select the table and then go to "Design" tab > "Columns" tab.
2) Click "Add" and select the new column name, and type INTEGER and Not Null > Ok.
3) Go to "Primary Key" tab in "Desgin tab". Click "Add" and select the column you just created. Check the "Autoincrement" box.
4) Click "Apply" on the right bottom part of the window.
If you go back to the "Data" tab, you will see your new column with the autogenerated numbers in it.
While the Sqlite site gives you an example how to do it with a table with only a three fields, it gets nasty with one of 30 fields. Given you have a table called OldTable with many fields, the first of which is "ID" filled with integers.
Make a copy of your database for backup.
Using the command program dot commands,
.output Oldtable.txt
.dump Oldtable
Drop Table Oldtable;
Open Oldtable.txt in Microsoft Word or a grep like text editor. Find and Replace your Integer field elements with NULL.(You may need to adjust this to fit your fields). Edit the Create Table line so the field that was defined as Integer is now INTEGER PRIMARY KEY AUTOINCREMENT.
Save as NewTable.txt
Back in the command program dot
.read NewTable.txt
Done.
ID is now autoincrement.
Yes
Do you have phpmyadmin installed? I believe if you go to the 'structure' tab and look along the right columnn (where the field types are listed) - I think you can change a setting there to make it autoincrement. There is also a SQL query that will do the same thing.
You cannot alter columns on a SQLite table after it has been created. You also cannot alter a table to add an integer primary key to it.
You have to add the integer primary key when you create the table.
Yes, you can make a column which is autoincrement. Modify the table and add a column. Keep in mind that it is of type INTEGER Primary Key.
you can alter the table, altering the column definition
Simple Answer is as below,
CREATE TABLE [TEST] (
[ID] INTEGER PRIMARY KEY AUTOINCREMENT,
[NAME] VARCHAR(100));
and you are done.

Resources