I create a SQLite function which takes a string and returns another string, then I can use the return value as match strings. Code is here
It works very well except for the single quotes. In this case, it can't match any rows, but if I directly use the returned string, it can match. Anyone know what's the problem here?
sqlite> select simple_query('''');
"''"
sqlite> select ' ', simple_highlight(t1, 0, '[', ']') from t1 where x match simple_query('''');
sqlite> select ' ', simple_highlight(t1, 0, '[', ']') from t1 where x match '"''"';
|#English &special _characters."[']bacon-&and[']-eggs%
Full example here
This question finally answered in sqlite-forum, and I'd like to post the reason in here.
The reason is SQLite will try to escape the string for us, we can verify that after turn on quote mode, as you can see, our return value will be escaped from "'" to "''" by SQLite. Which means we don't need to escape single quote in our function.
sqlite> select simple_query('''');
"'"
sqlite> select simple_query('"');
""""
sqlite> .mode quote
sqlite> select simple_query('"');
'""""'
sqlite> select simple_query('''');
'"''"'
When in DELPHI XE3 with Firedac we create a table table0 and insert text in UTF8 encoding all is OK.
CREATE TABLE table0 (mytext Ntext,publishdate date);
INSERT INTO table0 (mytext,publishdate) VALUES ('привет','1998-12-07');
But, when we create a FTS table and insert text in UTF8.
CREATE VIRTUAL TABLE table1 USING FTS4(mytext,publishdate);
INSERT INTO table1 (mytext,publishdate) VALUES ('привет','1998-12-07');
and read the data with
SELECT mytext FROM table1
we get "??????".
The same commands in SQLITE Expert Personal return "привет". It means that in the table after Insert command we have 'привет' and select returns the data not in UTF8 encoding.
What may be done to get from the table correct value with Firedac
ADQuery.Open('SELECT mytext FROM table1')?
I think it is FireDac bug.
I've changed the following lines in ADSQLiteTypeName2ADDataType procedure of uADPhysSQLite.pas unit
SetLen(AOptions.FormatOptions.MaxStringSize, False);
AType := dtAnsiString;
to
SetLen(AOptions.FormatOptions.MaxStringSize, True);
AType := dtWideString;
for the case ABaseTypeName = 'VARCHAR'
and
SELECT mytext FROM table0
returns the correct value at runtime. But at design time still we have '??????'.
But I don't think it's a good solution.
I believe you should directly set parameter type as AsWideString:
Query.SQL.Text:='INSERT INTO table1 (mytext, publishdate) VALUES (:mytext,: publishdate);';
Query.Params[0].AsWideString := mytext;
Query.Params[1].AsDate := publishdate;
Query.ExecSQL;
Reference: http://docwiki.embarcadero.com/RADStudio/Rio/en/Unicode_Support_(FireDAC)#Parameter_Values
while inserting i can insert danish character in proper format in sqlite Db but while retrieving my query returns no result
String searchQuery= "SELECT * FROM article,product where article.ItemNo=product.ItemNo ";
if(searchText.length()>0)
{
searchQuery += " AND (article.itemNo like '"+ searchText +"%' OR product.Description like '"+ searchText +"%')";
}
in debug mode query is
`SELECT * FROM article,product where article.ItemNo=product.ItemNo AND (article.itemNo like '%ø%' OR product.Description like '%ø%')..`
No result returns
Proper query will be
SELECT * FROM article,product where article.ItemNo=product.ItemNo AND (article.itemNo like '%Ø%' OR product.Description like '%Ø%');
the desired description field value in Db is MØNTPUNG.
I am wondering is there any issue of case sensitivty?I am using UTF8 encoding for my raw file that will insert data to DB.
The documentation says:
SQLite only understands upper/lower case for ASCII characters by default. The LIKE operator is case sensitive by default for unicode characters that are beyond the ASCII range. For example, the expression 'a' LIKE 'A' is TRUE but 'æ' LIKE 'Æ' is FALSE.
To handle non-ASCII characters correctly, store an uppercase version of your string(s) in a separate column, and search in that with an uppercase search pattern.
I wish to have a table name with
hello [world]
So, I thought using the following syntax would work
create table [hello [world]]] (key INTEGER PRIMARY KEY);
I get
Error: unrecognized token: "]"
I was wondering, is having square bracket supported in SQLite?
There are multiple methods for quoting identifiers.
When using square brackets, it is not possible to have these characters in the identifier.
When using double quotes, you can escape them in the name by doubling them:
CREATE TABLE "hello ""world"""(key INTEGER PRIMARY KEY);
Try quoting the table name:
$ sqlite3 :memory:
sqlite> create table "Foo [bar]" (a int);
sqlite> .schema
CREATE TABLE "Foo [bar]" (a int);
sqlite> insert into "Foo [bar]" values(1);
sqlite> select * from "Foo [bar]";
1
sqlite> _
Ok, so I have a basic table called "ledger", it contains fields of various types, integers, varchar, etc.
In my program, I used to use a query with no "from" predicate to collect all of the rows, which of course works fine. But... I changed my code to allow selecting one row at a time using "where acctno = x" (where X is the account number I want to select at the time).
I thought this must be a bug in the client library for my programming language, so I tested it in the SQLite command-line client - and it still doesn't work!
I am relatively new to SQLite, but I have been using Oracle, MS SQL Server, etc. for years and never seen this type of issue before.
Other things I can tell you:
* Queries using other integer fields also don't work
* Queries on char fields work
* Querying it as a string (with the account number on quotes) still doesn't work. (I thought maybe the numbers were stored as a string inadvertently).
* Accessing rows by rowid works fine - which is why I can edit the database with GUI tools with no noticeable problem.
Examples:
Query with no WHERE (works fine):
1|0|0|JPY|8|Paid-In Capital|C|X|0|X|0|0||||0|0|0|
0|0|0|JPY|11|Root Account|P|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
3|0|0|JPY|13|Mitsubishi Bank Futsuu|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
4|0|0|JPY|14|Japan Post Bank|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
...
Query with WHERE clause: (no results)
sqlite> select * from ledger where acctno=1;
sqlite>
putting quotes around the 1 above changes nothing.
Interestingly enough, "select * from ledger where acctno > 1" returns results! However since it returns ALL results, it's not terrible useful.
I'm sure someone will ask about the table structure, so here goes:
sqlite> .schema ledger
CREATE TABLE "LEDGER" (
"ACCTNO" integer(10,0) NOT NULL,
"drbal" integer(20,0) NOT NULL,
"crbal" integer(20,0) NOT NULL,
"CURRKEY" char(3,0) NOT NULL,
"TEXTKEY" integer(10,0),
"TEXT" VARCHAR(64,0),
"ACCTYPECD" CHAR(1,0) NOT NULL,
"ACCSTCD" CHAR(1,0),
"PACCTNO" number(10,0) NOT NULL,
"CATCD" number(10,0),
"TRANSNO" number(10,0) NOT NULL,
"extrefno" number(10,0),
"UPDATEUSER" VARCHAR(32,0),
"UPDATEDATE" text(8,0),
"UPDATETIME" TEXT(6,0),
"PAYEECD" number(10,0) NOT NULL,
"drbal2" number(10,0) NOT NULL,
"crbal2" number(10,0) NOT NULL,
"delind" boolean,
PRIMARY KEY("ACCTNO"),
CONSTRAINT "fk_curr" FOREIGN KEY ("CURRKEY") REFERENCES "CURRENCY" ("CUR
RKEY") ON DELETE RESTRICT ON UPDATE CASCADE
);
The strangest thing is that I have other similar tables where this works fine!
sqlite> select * from journalhdr where transno=13;
13|Test transaction ATM Withdrawel 20130213|20130223||20130223||
TransNo in that table is also integer (10,0) NOT NULL - this is what makes me thing it is something to do with the values.
Another clue is that the sort order seems to be based on ascii, not numeric:
sqlite> select * from ledger order by acctno;
0|0|0|JPY|11|Root Account|P|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
1|0|0|JPY|8|Paid-In Capital|C|X|0|X|0|0||||0|0|0|
10|0|0|USD|20|Sallie Mae|L|X|0|X|0|0|SYSTEM|20121209|153900|0|0|0|
21|0|0|USD|21|Skrill|A|X|0|X|0|0|SYSTEM|20121209|154000|0|0|0|
22|0|0|USD|22|AES|L|X|0|X|0|0|SYSTEM|20121209|154200|0|0|0|
23|0|0|JPY|23|Marui|L|X|0|X|0|0|SYSTEM|20121209|154400|0|0|0|
24|0|0|JPY|24|Amex JP|L|X|0|X|0|0|SYSTEM|20121209|154500|0|0|0|
3|0|0|JPY|13|Mitsubishi Bank Futsuu|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
Of course the sort order on journalhdr (where the select works properly) is numeric.
Solved! (sort-of)
The data can be fixed like this:
sqlite> update ledger set acctno = 23 where rowid = 13;
sqlite> select * from ledger where acctno = 25;
25|0|0|JPY|0|Test|L|X|0|X|0|0|SYSTEM|20130224|132500|0|0|0|
Still, if it was stored as strings, then that leave a few questions:
1. Why couldn't I select it as a string using the quotes?
2. How did it get stored as a string since it is a valid integer?
3. How would you go about detecting this problem normally besides noticing bizzarre symptoms?
Although the data would normally be entered by my program, some of it was created by hand using Navicat, so I assume the problem must lie there.
You are victim of SQLite dynamic typing.
Even though SQLite defines system of type affinity, which sets some rules on how input strings or numbers will be converted to actual internal values, but it does NOT prevent software that is using prepared statements to explicitly set any type (and data value) for the column (and this can be different per row!).
This can be shown by this simple example:
CREATE TABLE ledger (acctno INTEGER, name VARCHAR(16));
INSERT INTO ledger VALUES(1, 'John'); -- INTEGER '1'
INSERT INTO ledger VALUES(2 || X'00', 'Zack'); -- BLOB '2\0'
I have inserted second row not as INTEGER, but as binary string containing embedded zero byte. This reproduces your issue exactly, see this SQLFiddle, step by step. You can also execute these commands in sqlite3, you will get the same result.
Below is Perl script that also reproduces this issue
This script creates just 2 rows with acctno having values of integer 1 for first, and "2\0" for second row. "2\0" means string consisting of 2 bytes: first is digit 2, and second is 0 (zero) byte.
Of course, it is very difficult to visually tell "2\0" from just "2", but this is what script below demonstrates:
#!/usr/bin/perl -w
use strict;
use warnings;
use DBI qw(:sql_types);
my $dbh = DBI->connect("dbi:SQLite:test.db") or die DBI::errstr();
$dbh->do("DROP TABLE IF EXISTS ledger");
$dbh->do("CREATE TABLE ledger (acctno INTEGER, name VARCHAR(16))");
my $sth = $dbh->prepare(
"INSERT INTO ledger (acctno, name) VALUES (?, ?)");
$sth->bind_param(1, "1", SQL_INTEGER);
$sth->bind_param(2, "John");
$sth->execute();
$sth->bind_param(1, "2\0", SQL_BLOB);
$sth->bind_param(2, "Zack");
$sth->execute();
$sth = $dbh->prepare(
"SELECT count(*) FROM ledger WHERE acctno = ?");
$sth->bind_param(1, "1");
$sth->execute();
my ($num1) = $sth->fetchrow_array();
print "Number of rows matching id '1' is $num1\n";
$sth->bind_param(1, "2");
$sth->execute();
my ($num2) = $sth->fetchrow_array();
print "Number of rows matching id '2' is $num2\n";
$sth->bind_param(1, "2\0", SQL_BLOB);
$sth->execute();
my ($num3) = $sth->fetchrow_array();
print "Number of rows matching id '2<0>' is $num3\n";
Output of this script is:
Number of rows matching id '1' is 1
Number of rows matching id '2' is 0
Number of rows matching id '2<0>' is 1
If you were to look at resultant table using any SQLite tool (including sqlite3), it will print 2 for second row - they all get confused by trailing 0 inside a BLOB when it gets coerced to string or number.
Note that I had to use custom param binding to coerce type to BLOB and permit null bytes stored:
$sth->bind_param(1, "2\0", SQL_BLOB);
Long story short, it is either some of your client programs, or some of client tools like Navicat which screwed it up.