When in DELPHI XE3 with Firedac we create a table table0 and insert text in UTF8 encoding all is OK.
CREATE TABLE table0 (mytext Ntext,publishdate date);
INSERT INTO table0 (mytext,publishdate) VALUES ('привет','1998-12-07');
But, when we create a FTS table and insert text in UTF8.
CREATE VIRTUAL TABLE table1 USING FTS4(mytext,publishdate);
INSERT INTO table1 (mytext,publishdate) VALUES ('привет','1998-12-07');
and read the data with
SELECT mytext FROM table1
we get "??????".
The same commands in SQLITE Expert Personal return "привет". It means that in the table after Insert command we have 'привет' and select returns the data not in UTF8 encoding.
What may be done to get from the table correct value with Firedac
ADQuery.Open('SELECT mytext FROM table1')?
I think it is FireDac bug.
I've changed the following lines in ADSQLiteTypeName2ADDataType procedure of uADPhysSQLite.pas unit
SetLen(AOptions.FormatOptions.MaxStringSize, False);
AType := dtAnsiString;
to
SetLen(AOptions.FormatOptions.MaxStringSize, True);
AType := dtWideString;
for the case ABaseTypeName = 'VARCHAR'
and
SELECT mytext FROM table0
returns the correct value at runtime. But at design time still we have '??????'.
But I don't think it's a good solution.
I believe you should directly set parameter type as AsWideString:
Query.SQL.Text:='INSERT INTO table1 (mytext, publishdate) VALUES (:mytext,: publishdate);';
Query.Params[0].AsWideString := mytext;
Query.Params[1].AsDate := publishdate;
Query.ExecSQL;
Reference: http://docwiki.embarcadero.com/RADStudio/Rio/en/Unicode_Support_(FireDAC)#Parameter_Values
Related
Very similar to How to display blob value using x'abc' binary string literal syntax?:
How can I have the sqlite3 shell display always display blob columns using the hex notation, as per e.g. quote(blob_column_name), without explicitly using quote, and in select * queries (and other contexts where blob_column_name isn't mentioned explicitly)?
(I suspect the answer is "you can't", but I'm hoping to be pleasantly surprised.)
There are two output modes that use SQL syntax (not only for blobs but for all values):
sqlite> .mode quote
sqlite> SELECT 1, x'123ABC', 'hello', null;
1,X'123abc','hello',NULL
sqlite> .mode insert
sqlite> SELECT 1, x'123ABC', 'hello', null;
INSERT INTO "table" VALUES(1,X'123abc','hello',NULL);
Ok, so I have a basic table called "ledger", it contains fields of various types, integers, varchar, etc.
In my program, I used to use a query with no "from" predicate to collect all of the rows, which of course works fine. But... I changed my code to allow selecting one row at a time using "where acctno = x" (where X is the account number I want to select at the time).
I thought this must be a bug in the client library for my programming language, so I tested it in the SQLite command-line client - and it still doesn't work!
I am relatively new to SQLite, but I have been using Oracle, MS SQL Server, etc. for years and never seen this type of issue before.
Other things I can tell you:
* Queries using other integer fields also don't work
* Queries on char fields work
* Querying it as a string (with the account number on quotes) still doesn't work. (I thought maybe the numbers were stored as a string inadvertently).
* Accessing rows by rowid works fine - which is why I can edit the database with GUI tools with no noticeable problem.
Examples:
Query with no WHERE (works fine):
1|0|0|JPY|8|Paid-In Capital|C|X|0|X|0|0||||0|0|0|
0|0|0|JPY|11|Root Account|P|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
3|0|0|JPY|13|Mitsubishi Bank Futsuu|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
4|0|0|JPY|14|Japan Post Bank|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
...
Query with WHERE clause: (no results)
sqlite> select * from ledger where acctno=1;
sqlite>
putting quotes around the 1 above changes nothing.
Interestingly enough, "select * from ledger where acctno > 1" returns results! However since it returns ALL results, it's not terrible useful.
I'm sure someone will ask about the table structure, so here goes:
sqlite> .schema ledger
CREATE TABLE "LEDGER" (
"ACCTNO" integer(10,0) NOT NULL,
"drbal" integer(20,0) NOT NULL,
"crbal" integer(20,0) NOT NULL,
"CURRKEY" char(3,0) NOT NULL,
"TEXTKEY" integer(10,0),
"TEXT" VARCHAR(64,0),
"ACCTYPECD" CHAR(1,0) NOT NULL,
"ACCSTCD" CHAR(1,0),
"PACCTNO" number(10,0) NOT NULL,
"CATCD" number(10,0),
"TRANSNO" number(10,0) NOT NULL,
"extrefno" number(10,0),
"UPDATEUSER" VARCHAR(32,0),
"UPDATEDATE" text(8,0),
"UPDATETIME" TEXT(6,0),
"PAYEECD" number(10,0) NOT NULL,
"drbal2" number(10,0) NOT NULL,
"crbal2" number(10,0) NOT NULL,
"delind" boolean,
PRIMARY KEY("ACCTNO"),
CONSTRAINT "fk_curr" FOREIGN KEY ("CURRKEY") REFERENCES "CURRENCY" ("CUR
RKEY") ON DELETE RESTRICT ON UPDATE CASCADE
);
The strangest thing is that I have other similar tables where this works fine!
sqlite> select * from journalhdr where transno=13;
13|Test transaction ATM Withdrawel 20130213|20130223||20130223||
TransNo in that table is also integer (10,0) NOT NULL - this is what makes me thing it is something to do with the values.
Another clue is that the sort order seems to be based on ascii, not numeric:
sqlite> select * from ledger order by acctno;
0|0|0|JPY|11|Root Account|P|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
1|0|0|JPY|8|Paid-In Capital|C|X|0|X|0|0||||0|0|0|
10|0|0|USD|20|Sallie Mae|L|X|0|X|0|0|SYSTEM|20121209|153900|0|0|0|
21|0|0|USD|21|Skrill|A|X|0|X|0|0|SYSTEM|20121209|154000|0|0|0|
22|0|0|USD|22|AES|L|X|0|X|0|0|SYSTEM|20121209|154200|0|0|0|
23|0|0|JPY|23|Marui|L|X|0|X|0|0|SYSTEM|20121209|154400|0|0|0|
24|0|0|JPY|24|Amex JP|L|X|0|X|0|0|SYSTEM|20121209|154500|0|0|0|
3|0|0|JPY|13|Mitsubishi Bank Futsuu|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
Of course the sort order on journalhdr (where the select works properly) is numeric.
Solved! (sort-of)
The data can be fixed like this:
sqlite> update ledger set acctno = 23 where rowid = 13;
sqlite> select * from ledger where acctno = 25;
25|0|0|JPY|0|Test|L|X|0|X|0|0|SYSTEM|20130224|132500|0|0|0|
Still, if it was stored as strings, then that leave a few questions:
1. Why couldn't I select it as a string using the quotes?
2. How did it get stored as a string since it is a valid integer?
3. How would you go about detecting this problem normally besides noticing bizzarre symptoms?
Although the data would normally be entered by my program, some of it was created by hand using Navicat, so I assume the problem must lie there.
You are victim of SQLite dynamic typing.
Even though SQLite defines system of type affinity, which sets some rules on how input strings or numbers will be converted to actual internal values, but it does NOT prevent software that is using prepared statements to explicitly set any type (and data value) for the column (and this can be different per row!).
This can be shown by this simple example:
CREATE TABLE ledger (acctno INTEGER, name VARCHAR(16));
INSERT INTO ledger VALUES(1, 'John'); -- INTEGER '1'
INSERT INTO ledger VALUES(2 || X'00', 'Zack'); -- BLOB '2\0'
I have inserted second row not as INTEGER, but as binary string containing embedded zero byte. This reproduces your issue exactly, see this SQLFiddle, step by step. You can also execute these commands in sqlite3, you will get the same result.
Below is Perl script that also reproduces this issue
This script creates just 2 rows with acctno having values of integer 1 for first, and "2\0" for second row. "2\0" means string consisting of 2 bytes: first is digit 2, and second is 0 (zero) byte.
Of course, it is very difficult to visually tell "2\0" from just "2", but this is what script below demonstrates:
#!/usr/bin/perl -w
use strict;
use warnings;
use DBI qw(:sql_types);
my $dbh = DBI->connect("dbi:SQLite:test.db") or die DBI::errstr();
$dbh->do("DROP TABLE IF EXISTS ledger");
$dbh->do("CREATE TABLE ledger (acctno INTEGER, name VARCHAR(16))");
my $sth = $dbh->prepare(
"INSERT INTO ledger (acctno, name) VALUES (?, ?)");
$sth->bind_param(1, "1", SQL_INTEGER);
$sth->bind_param(2, "John");
$sth->execute();
$sth->bind_param(1, "2\0", SQL_BLOB);
$sth->bind_param(2, "Zack");
$sth->execute();
$sth = $dbh->prepare(
"SELECT count(*) FROM ledger WHERE acctno = ?");
$sth->bind_param(1, "1");
$sth->execute();
my ($num1) = $sth->fetchrow_array();
print "Number of rows matching id '1' is $num1\n";
$sth->bind_param(1, "2");
$sth->execute();
my ($num2) = $sth->fetchrow_array();
print "Number of rows matching id '2' is $num2\n";
$sth->bind_param(1, "2\0", SQL_BLOB);
$sth->execute();
my ($num3) = $sth->fetchrow_array();
print "Number of rows matching id '2<0>' is $num3\n";
Output of this script is:
Number of rows matching id '1' is 1
Number of rows matching id '2' is 0
Number of rows matching id '2<0>' is 1
If you were to look at resultant table using any SQLite tool (including sqlite3), it will print 2 for second row - they all get confused by trailing 0 inside a BLOB when it gets coerced to string or number.
Note that I had to use custom param binding to coerce type to BLOB and permit null bytes stored:
$sth->bind_param(1, "2\0", SQL_BLOB);
Long story short, it is either some of your client programs, or some of client tools like Navicat which screwed it up.
I need to get column names and their tables in a SQLite database. What I need is a resultset with 2 columns: table_name | column_name.
In MySQL, I'm able to get this information with a SQL query on database INFORMATION_SCHEMA. However the SQLite offers table sqlite_master:
sqlite> create table students (id INTEGER, name TEXT);
sqlite> select * from sqlite_master;
table|students|students|2|CREATE TABLE students (id INTEGER, name TEXT)
which results a DDL construction query (CREATE TABLE) which is not helpful for me and I need to parse this to get relevant information.
I need to get list of tables and join them with columns or just get columns along with table name column. So PRAGMA table_info(TABLENAME) is not working for me since I don't have table name. I want to get all column metadata in the database.
Is there a better way to get that information as a result set by querying database?
You've basically named the solution in your question.
To get a list of tables (and views), query sqlite_master as in
SELECT name, sql FROM sqlite_master
WHERE type='table'
ORDER BY name;
(see the SQLite FAQ)
To get information about the columns in a specific table, use PRAGMA table_info(table-name); as explained in the SQLite PRAGMA documentation.
I don't know of any way to get tablename|columnname returned as the result of a single query. I don't believe SQLite supports this. Your best bet is probably to use the two methods together to return the information you're looking for - first get the list of tables using sqlite_master, then loop through them to get their columns using PRAGMA table_info().
Recent versions of SQLite allow you to select against PRAGMA results now, which makes this easy:
SELECT
m.name as table_name,
p.name as column_name
FROM
sqlite_master AS m
JOIN
pragma_table_info(m.name) AS p
ORDER BY
m.name,
p.cid
where p.cid holds the column order of the CREATE TABLE statement, zero-indexed.
David Garoutte answered this here, but this SQL should execute faster, and columns are ordered by the schema, not alphabetically.
Note that table_info also contains
type (the datatype, like integer or text),
notnull (1 if the column has a NOT NULL constraint)
dflt_value (NULL if no default value)
pk (1 if the column is the table's primary key, else 0)
RTFM: https://www.sqlite.org/pragma.html#pragma_table_info
There are ".tables" and ".schema [table_name]" commands which give kind of a separated version to the result you get from "select * from sqlite_master;"
There is also "pragma table_info([table_name]);" command to get a better result for parsing instead of a construction query:
sqlite> .tables
students
sqlite> .schema students
create table students(id INTEGER, name TEXT);
sqlite> pragma table_info(students);
0|id|INTEGER|0||0
1|name|TEXT|0||0
Hope, it helps to some extent...
Another useful trick is to first get all the table names from sqlite_master.
Then for each one, fire off a query "select * from t where 1 = 0". If you analyze the structure of the resulting query - depends on what language/api you're calling it from - you get a rich structure describing the columns.
In python
c = ...db.cursor()
c.execute("select * from t where 1=0");
c.fetchall();
print c.description;
Juraj
PS. I'm in the habit of using 'where 1=0' because the record limiting syntax seems to vary from db to db. Furthermore, a good database will optimize out this always-false clause.
The same effect, in SQLite, is achieved with 'limit 0'.
FYI, if you're using .Net you can use the DbConnection.GetSchema method to retrieve information that usually is in INFORMATION_SCHEMA. If you have an abstraction layer you can have the same code for all types of databases (NOTE that MySQL seems to swich the 1st 2 arguments of the restrictions array).
Try this sqlite table schema parser, I implemented the sqlite table parser for parsing the table definitions in PHP.
It returns the full definitions (unique, primary key, type, precision, not null, references, table constraints... etc)
https://github.com/maghead/sqlite-parser
The syntax follows sqlite create table statement syntax: http://www.sqlite.org/lang_createtable.html
This is an old question but because of the number of times it has been viewed we are adding to the question for the simple reason most of the answers tell you how to find the TABLE names in the SQLite Database
WHAT DO YOU DO WHEN THE TABLE NAME IS NOT IN THE DATABASE ?
This is happening to our app because we are creating TABLES programmatically
So the code below will deal with the issue when the TABLE is NOT in or created by the Database Enjoy
public void toPageTwo(View view){
if(etQuizTable.getText().toString().equals("")){
Toast.makeText(getApplicationContext(), "Enter Table Name\n\n"
+" OR"+"\n\nMake Table First", Toast.LENGTH_LONG
).show();
etQuizTable.requestFocus();
return;
}
NEW_TABLE = etQuizTable.getText().toString().trim();
db = dbHelper.getWritableDatabase();
ArrayList<String> arrTblNames = new ArrayList<>();
Cursor c = db.rawQuery("SELECT name FROM sqlite_master WHERE
type='table'", null);
if (c.moveToFirst()) {
while ( !c.isAfterLast() ) {
arrTblNames.add( c.getString( c.getColumnIndex("name")) );
c.moveToNext();
}
}
c.close();
db.close();
boolean matchFound = false;
for(int i=0;i<arrTblNames.size();i++) {
if(arrTblNames.get(i).equals(NEW_TABLE)) {
Intent intent = new Intent(ManageTables.this, TableCreate.class
);
startActivity( intent );
matchFound = true;
}
}
if (!matchFound) {
Toast.makeText(getApplicationContext(), "No Such Table\n\n"
+" OR"+"\n\nMake Table First", Toast.LENGTH_LONG
).show();
etQuizTable.requestFocus();
}
}
I want to create a table in a SQLite database only if doesn't exist already. Is there any way to do this? I don't want to drop the table if it exists, only create it if it doesn't.
From http://www.sqlite.org/lang_createtable.html:
CREATE TABLE IF NOT EXISTS some_table (id INTEGER PRIMARY KEY AUTOINCREMENT, ...);
Am going to try and add value to this very good question and to build on #BrittonKerin's question in one of the comments under #David Wolever's fantastic answer. Wanted to share here because I had the same challenge as #BrittonKerin and I got something working (i.e. just want to run a piece of code only IF the table doesn't exist).
# for completeness lets do the routine thing of connections and cursors
conn = sqlite3.connect(db_file, timeout=1000)
cursor = conn.cursor()
# get the count of tables with the name
tablename = 'KABOOM'
cursor.execute("SELECT count(name) FROM sqlite_master WHERE type='table' AND name=? ", (tablename, ))
print(cursor.fetchone()) # this SHOULD BE in a tuple containing count(name) integer.
# check if the db has existing table named KABOOM
# if the count is 1, then table exists
if cursor.fetchone()[0] ==1 :
print('Table exists. I can do my custom stuff here now.... ')
pass
else:
# then table doesn't exist.
custRET = myCustFunc(foo,bar) # replace this with your custom logic
I am looking to retrieve a list of columns in a table. The database is the latest release of SQLite (3.6, I believe). I am looking for code that does this with a SQL query. Extra bonus points for metadata related to the columns (e.g. length, data type, etc...)
What you're looking for is called the data dictionary. In sqlite a list of all tables can be found by querying sqlite_master table (or view?)
sqlite> create table people (first_name varchar, last_name varchar, email_address varchar);
sqlite> select * from sqlite_master;
table|people|people|2|CREATE TABLE people (first_name varchar, last_name varchar, email_address varchar)
To get column information you can use the pragma table_info(table_name) statement:
sqlite> pragma table_info(people);
0|first_name|varchar|0||0
1|last_name|varchar|0||0
2|email_address|varchar|0||0
For more information on the pragma statements, see the documentation.
Here's the simple way:
.schema <table>
The question is old but the following hasn't been mentioned yet.
Another convenient way in many cases is to turn headers on by:
sqlite> .headers on
Then,
sqlite> SELECT ... FROM table
will display a headline showing all selected fields (all if you SELECT *) at the top of the output.
Here's a SELECT statement that lists all tables and columns in the current database:
SELECT m.name as tableName,
p.name as columnName
FROM sqlite_master m
left outer join pragma_table_info((m.name)) p
on m.name <> p.name
order by tableName, columnName
;
just go into your sqlite shell:
$ sqlite3 path/to/db.sqlite3
and then just hit
sqlite> .schema
and you will get everything.
This is a query that lists all tables with their columns, and all the metadata I could get about each column as OP requested (as bonus points).
SELECT
m.name AS table_name,
p.cid AS col_id,
p.name AS col_name,
p.type AS col_type,
p.pk AS col_is_pk,
p.dflt_value AS col_default_val,
p.[notnull] AS col_is_not_null
FROM sqlite_master m
LEFT OUTER JOIN pragma_table_info((m.name)) p
ON m.name <> p.name
WHERE m.type = 'table'
ORDER BY table_name, col_id
Thanks to #David Garoutte for showing me how to get pragma_table_info to work in a query.
Run this query to see all the table metadata:
SELECT * FROM sqlite_master WHERE type = 'table'
Building on the above, you can do it all at once:
sqlite3 yourdb.db ".schema"
That will give you the SQL to create the table, which is effectively a list of the columns.
In case if you want to get all column names into one single comma separated string, you can use below.
SELECT GROUP_CONCAT(NAME,',') FROM PRAGMA_TABLE_INFO('table_name')
Here the pragma table_info is used as pragma_table_info for the select statement and GROUP_CONCAT is to combine all the field names into one string. for the second parameter of GROUP_CONCAT you can pass the separator.
I know, it’s been a long time but it’s never too late…
I had a similar question with TCL as interpreter and after several search, found nothing good for me. So I propose something based on PRAGMA, knowing that your DB is “main”
db eval { PRAGMA main.table_info(<your table name>) } TBL { puts $TBL(name) }
And array use to obtain a list
set col_list {}
db eval { PRAGMA main.table_info(<your table name>) } TBL { lappend col_list $TBL(name) }
puts $col_list