tcl var substitution in sqlite3 eval - sqlite

I have a file filled with sqlite statements I'd like to parse. I've done so previously using the C-interface but now I have trouble with the Tcl interface.
sqlite3 cspdb ":memory:"
set s [read [set f [open csp_sql.txt]]]
set ms [string map {"\n" "\0"} $s]
puts $ms
cspdb eval {$ms}
The error I get is:
"near "$ms": syntax error while executing cspdb eval {$ms}"
It works fine when I paste the output from puts $ms directly into the eval brackets. I also tried "{$ms}" and just plain $ms but the result is the same; syntax error.
I might chose another solution for the problem all together but it really bugs my why it does not work...
Here is what the file looks like:
BEGIN TRANSACTION;
CREATE TABLE Symbol (
Label TEXT PRIMARY KEY,
Type TEXT DEFAULT('no_type')
);
CREATE TABLE Process (
Name INTEGER PRIMARY KEY,
Type TEXT DEFAULT('no_type')
);
CREATE TABLE Named_Process (
Label TEXT UNIQUE,
Definition INTEGER UNIQUE,
FOREIGN KEY(Label) REFERENCES Symbol (Label),
FOREIGN KEY(Definition) REFERENCES Definition(Name)
);
CREATE TABLE Definition (
Name INTEGER UNIQUE,
Definition INTEGER,
FOREIGN KEY(Name) REFERENCES Process(Name),
FOREIGN KEY(Definition) REFERENCES Process(Name)
);
CREATE TABLE Reference (
Name INTEGER UNIQUE,
Reference TEXT,
FOREIGN KEY(Name) REFERENCES Process(Name),
FOREIGN KEY(Reference) REFERENCES Definition(Name)
);
CREATE TABLE Event (
Label TEXT PRIMARY KEY,
Type TEXT DEFAULT('no_type')
);
CREATE TABLE Environment (
Label TEXT UNIQUE,
FOREIGN KEY (Label) REFERENCES Event (Label)
);
CREATE TABLE Prefix (
Name INTEGER UNIQUE,
P INTEGER,
Event TEXT,
FOREIGN KEY (Name) REFERENCES Process (Name),
FOREIGN KEY (P) REFERENCES Process (Name),
FOREIGN KEY (Event) REFERENCES Event (Label)
);
CREATE TABLE Choice (
Name INTEGER UNIQUE,
P INTEGER,
Q INTEGER,
FOREIGN KEY(Name) REFERENCES Process(Name),
FOREIGN KEY(P) REFERENCES Process(Name),
FOREIGN KEY(Q) REFERENCES Process(Name)
);
/* Language defined processes */
/* SKIP */
INSERT INTO Symbol(Label,Type) VALUES('SKIP','named_process');
INSERT INTO Named_Process(Label) VALUES('SKIP');
INSERT INTO Process(type) VALUES('definition');
UPDATE Named_Process SET Definition=last_insert_rowid() WHERE Label='SKIP';
INSERT INTO Definition(Name,Definition) VALUES(last_insert_rowid(),last_insert_rowid());
END TRANSACTION;
(Not so sure removing newlines is necessary...)

If the string that you've read from the file is SQL, you should be able to do this:
sqlite3 cspdb ":memory:"
set f [open "csp_sql.txt"]
set sql [read $f]
close $f
cspdb eval $sql
By comparison, the literal string $ms is not a valid SQL statement or query, nor is it syntactically legal to surround a SQL statement with braces (which is what "{$ms}" ended up doing; the outer "…" makes the inside just a bunch of characters).
I don't know why you are translating newlines into NULs, but that's really unlikely to be a good idea. Bulk import of data (possibly with NULs in it) should be done in a different way.

The eval subcommand created by Sqlite does limited variable substitution, but only in places where a string value is expected.
Try
cspdb eval $ms
to let the Tcl interpreter substitute the SQL statements before sending them to eval.

turns out the newline-to-null was the culprit making substitution fail. final working code:
sqlite3 cspdb ":memory:"
cspdb eval [read [set f [open csp_sql.txt]]]
the reason newline-to-null turned up at all was when trying to debug making the string match my original working C-code which used multiline string literals in which newlines diappeared. Except, "\0" is not a "disappearing" character...

Related

MariaDB - Foreign key constraint incorrect?

I tried to re-engineer a database, that I use at work. The one at work is MS Access. At home, it's MariaDB. For convenience, I use MySQL Workbench.
When sending the complete SQL dump to the server, I get an error concerning some foreign key not being correctly formed. I guess, it is a minor mistake, but still I cannot find it.
My InnoDB status tells me this:
LATEST FOREIGN KEY ERROR
2018-10-03 00:18:29 409c7450 Error in foreign key constraint of table `mydb`.`IF`:
FOREIGN KEY (`belegid`)
REFERENCES `mydb`.`tblBelegPositionen` (`belegfID`)
ON DELETE NO ACTION
ON UPDATE NO ACTION,
CONSTRAINT `fk_tblBelege_tblECKassenschnittPositionen10`
FOREIGN KEY (`belegid`)
REFERENCES `mydb`.`tblECKassenschnittPositionen` (`belegfID`)
ON DELETE NO ACTION
ON UPDATE NO ACTION)
ENGINE = InnoDB:
Cannot find an index in the referenced table where the
referenced columns appear as the first columns, or column types
in the table and the referenced table do not match for constraint.
Note that the internal storage type of ENUM and SET changed in
tables created with >= InnoDB-4.1.12, and such columns in old tables
cannot be referenced by such columns in new tables.
See http://dev.mysql.com/doc/refman/5.6/en/innodb-foreign-key-constraints.html
for correct foreign key definition.
Create table '`mydb`.`IF`' with foreign key constraint failed. There is no index in the referenced table where the referenced columns appear as the first columns near '
FOREIGN KEY (`belegid`)
REFERENCES `mydb`.`tblBelegPositionen` (`belegfID`)
ON DELETE NO ACTION
ON UPDATE NO ACTION,
CONSTRAINT `fk_tblBelege_tblECKassenschnittPositionen10`
FOREIGN KEY (`belegid`)
REFERENCES `mydb`.`tblECKassenschnittPositionen` (`belegfID`)
ON DELETE NO ACTION
ON UPDATE NO ACTION)
ENGINE = InnoDB'.
The really weird thing is that I do not have any table named "IF"...
Can anyone make heads or tails of this for me? That would be very much appreciated.
-- Table `mydb`.`tblBelegPositionen`
-- -----------------------------------------------------
DROP TABLE IF EXISTS `mydb`.`tblBelegPositionen` ;
SHOW WARNINGS;
CREATE TABLE IF NOT EXISTS `mydb`.`tblBelegPositionen` (
`belegposid` INT NOT NULL AUTO_INCREMENT,
`belegposBetrag` DOUBLE NOT NULL,
`zahlartfID` INT NOT NULL,
`belegfID` INT NOT NULL,
PRIMARY KEY (`belegposid`))
ENGINE = InnoDB;
SHOW WARNINGS;
-- Table `mydb`.`tblECKassenschnittPositionen`
-- -----------------------------------------------------
DROP TABLE IF EXISTS `mydb`.`tblECKassenschnittPositionen` ;
SHOW WARNINGS;
CREATE TABLE IF NOT EXISTS `mydb`.`tblECKassenschnittPositionen` (
`ecposid` INT NOT NULL AUTO_INCREMENT,
`belegfID` INT NOT NULL,
`ecposBetrag` DOUBLE NOT NULL,
`kassenschnittfID` INT NOT NULL,
PRIMARY KEY (`ecposid`))
ENGINE = InnoDB;
SHOW WARNINGS;
-- Table `mydb`.`tblBelege`
-- -----------------------------------------------------
DROP TABLE IF EXISTS `mydb`.`tblBelege` ;
SHOW WARNINGS;
CREATE TABLE IF NOT EXISTS `mydb`.`tblBelege` (
`belegid` INT UNSIGNED NOT NULL AUTO_INCREMENT,
`belegKassierer` INT NOT NULL,
`belegDatum` DATETIME NOT NULL,
`kassefID` INT NOT NULL,
`belegSchicht` INT NULL,
`gvfID` INT NOT NULL,
`belegJahr` YEAR NULL,
`belegDruckErfolgt` TINYINT(1) NULL,
`belegDruckDatum` DATETIME NULL,
`belegPeriodenfremdeBuchung` TINYINT(1) NULL,
PRIMARY KEY (`belegid`, `gvfID`, `kassefID`),
CONSTRAINT `fk_tblBelege_tblBelegPositionen10`
FOREIGN KEY (`belegid`)
REFERENCES `mydb`.`tblBelegPositionen` (`belegfID`)
ON DELETE NO ACTION
ON UPDATE NO ACTION,
CONSTRAINT `fk_tblBelege_tblECKassenschnittPositionen10`
FOREIGN KEY (`belegid`)
REFERENCES `mydb`.`tblECKassenschnittPositionen` (`belegfID`)
ON DELETE NO ACTION
ON UPDATE NO ACTION)
ENGINE = InnoDB;
SHOW WARNINGS;
Ok, so there are a couple of things to know here, as there were a couple of errors. You must check all the items that the error mentions, for correctness, to avoid trouble.
The cannot find an index in the referenced table portion of the error message means that the column/field specified in the REFERENCES clause must be indexed in the other table.
Then, the column type definition of the column specified in the FOREIGN KEY clause must match the column type of the column specified in the REFERENCES clause too, so even though the first item corrects part of the problem, there will still be an error related to another portion of the message: ...or column types in the table and the referenced table do not match.
So, to fix item 1, run these 2 queries:
ALTER TABLE `tblbelegpositionen` ADD INDEX(`belegfID`);
ALTER TABLE `tbleckassenschnittpositionen` ADD INDEX(`belegfID`);
Then to fix item 2, I had to change the first column of table tblBelege
from this:
`belegid` INT UNSIGNED NOT NULL AUTO_INCREMENT,
to this:
`belegid` INT NOT NULL,
...so that they matched the same type as the belegfID column as defined in the other tables. After those two changes, I was able to successfully run your CREATE TABLE `tblBelege` statement.
So, to recap:
Run your create table statements for tblbelegpositionen and
tbleckassenschnittpositionen.
Then run the 2 ALTER statements shown above for item 1.
Modify the first column of table tblBelege to match the column types as defined to belegfID in the other tables for item 2.
Then run your modified CREATE TABLE statement (with the item 2 change applied) to create the tblBelege table.
I'm a little confused about the same FOREIGN KEY referencing 2 different tables, but if it works for you, then ok. (not saying that it cannot be done, I've never used a foreign key that way) Perhaps you meant the opposite, to have a foreign key in the other 2 tables (1 in each table) that refer to tblBelege instead? If so, then you could add unsigned to the type definition for belegfID and it would work, and would not need the change that I mentioned with item 2.
Oh, and after you run the ALTER statements, you can view the table structure by running:
SHOW CREATE TABLE `tblbelegpositionen`;
SHOW CREATE TABLE `tbleckassenschnittpositionen`;
...to get the KEY definition that was added to include with your CREATE TABLE statements. Since they both create the same key, you really only need to run one of those statements and then add the KEY definition to both table statements.

SQLite syntax error on insert default value during trigger after

The problem: SQLite yields "near DEFAULT: syntax error" when running this through sqlite3_exec. The insertion works fine outside the trigger, and other statements works inside the trigger, but somehow the DEFAULT VALUES won´t work inside the trigger. Why is this happening?
SQLite code:
CREATE TABLE Symbol (
Label VARCHAR(127) PRIMARY KEY
);
CREATE TABLE Process (
Name INTEGER PRIMARY KEY
);
CREATE TABLE Named_Process_Definition (
Label VARCHAR(127),
Name INTEGER,
FOREIGN KEY (Label) REFERENCES Symbol (Label),
FOREIGN KEY (Name) REFERENCES Process_Definition (Name)
);
CREATE TRIGGER pre_new_named_process BEFORE INSERT ON Named_Process_Definition
BEGIN
INSERT INTO Symbol (Label) VALUES (NEW.Label);
END;
CREATE TRIGGER post_new_named_process AFTER INSERT ON Named_Process_Definition
BEGIN
INSERT INTO Process DEFAULT VALUES;
UPDATE Named_Process_Definition SET Name=last_insert_rowid() WHERE rowid=NEW.rowid;
END;
The triggers are meant to simplify inserting Named_Process_Definitions by automatically generating internal "unnamed" resources such as Process.
sqlite docs state:
The "INSERT INTO table DEFAULT VALUES" form of the INSERT statement is not supported.
You can work around this by inserting a null, e.g.:
CREATE TRIGGER post_new_named_process AFTER INSERT ON Named_Process_Definition
BEGIN
INSERT INTO Process(rowid) VALUES(NULL);
END;

T-SQL to SQLite conversion

I have the following T-SQL statement for creating a table which I have modified to work with SQLite
CREATE TABLE prerequisite_lesson(
next_lesson_id INT NOT NULL,
prereq_lesson_id INT NOT NULL,
CHECK(prereq_lesson_id) NOT (lesson_id),
CONSTRAINT pk_prereq PRIMARY KEY (lesson_id, prereq_lesson_id),
FOREIGN KEY(next_lesson_id) REFERENCES lessons(lesson_id),
FOREIGN KEY(prereq_lesson_id) REFERENCES lessons(lesson_id));
I'm getting 4 errors when I try to create the table as follows:
Unrecognized data type. (near "(" at position 111)
A comma or a closing bracket was expected. (near "prereq_lesson_id" at position 112)
Unexpected beginning of statement. (near "prereq_lesson_id" at position 112)
Unrecognized statement type. (near "NOT" at position 130)
Can anyone offer help with the errors? Thanks in advance.
Had to create lessons first.
create table lessons(
lesson_id int primary key not null
);
Then there is something with constraint check syntax. Please see an example here. Also for primary key pair probably need next_lesson_id instead of lesson_id.
CREATE TABLE prerequisite_lesson(
next_lesson_id INT NOT NULL,
prereq_lesson_id INT NOT NULL CHECK(NOT(next_lesson_id)),
CONSTRAINT pk_prereq PRIMARY KEY (next_lesson_id, prereq_lesson_id),
FOREIGN KEY(next_lesson_id) REFERENCES lessons(lesson_id),
FOREIGN KEY(prereq_lesson_id) REFERENCES lessons(lesson_id));
Full schema creation fiddle here.

Strange SQLite behavior: Not returning results on simple queries

Ok, so I have a basic table called "ledger", it contains fields of various types, integers, varchar, etc.
In my program, I used to use a query with no "from" predicate to collect all of the rows, which of course works fine. But... I changed my code to allow selecting one row at a time using "where acctno = x" (where X is the account number I want to select at the time).
I thought this must be a bug in the client library for my programming language, so I tested it in the SQLite command-line client - and it still doesn't work!
I am relatively new to SQLite, but I have been using Oracle, MS SQL Server, etc. for years and never seen this type of issue before.
Other things I can tell you:
* Queries using other integer fields also don't work
* Queries on char fields work
* Querying it as a string (with the account number on quotes) still doesn't work. (I thought maybe the numbers were stored as a string inadvertently).
* Accessing rows by rowid works fine - which is why I can edit the database with GUI tools with no noticeable problem.
Examples:
Query with no WHERE (works fine):
1|0|0|JPY|8|Paid-In Capital|C|X|0|X|0|0||||0|0|0|
0|0|0|JPY|11|Root Account|P|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
3|0|0|JPY|13|Mitsubishi Bank Futsuu|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
4|0|0|JPY|14|Japan Post Bank|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
...
Query with WHERE clause: (no results)
sqlite> select * from ledger where acctno=1;
sqlite>
putting quotes around the 1 above changes nothing.
Interestingly enough, "select * from ledger where acctno > 1" returns results! However since it returns ALL results, it's not terrible useful.
I'm sure someone will ask about the table structure, so here goes:
sqlite> .schema ledger
CREATE TABLE "LEDGER" (
"ACCTNO" integer(10,0) NOT NULL,
"drbal" integer(20,0) NOT NULL,
"crbal" integer(20,0) NOT NULL,
"CURRKEY" char(3,0) NOT NULL,
"TEXTKEY" integer(10,0),
"TEXT" VARCHAR(64,0),
"ACCTYPECD" CHAR(1,0) NOT NULL,
"ACCSTCD" CHAR(1,0),
"PACCTNO" number(10,0) NOT NULL,
"CATCD" number(10,0),
"TRANSNO" number(10,0) NOT NULL,
"extrefno" number(10,0),
"UPDATEUSER" VARCHAR(32,0),
"UPDATEDATE" text(8,0),
"UPDATETIME" TEXT(6,0),
"PAYEECD" number(10,0) NOT NULL,
"drbal2" number(10,0) NOT NULL,
"crbal2" number(10,0) NOT NULL,
"delind" boolean,
PRIMARY KEY("ACCTNO"),
CONSTRAINT "fk_curr" FOREIGN KEY ("CURRKEY") REFERENCES "CURRENCY" ("CUR
RKEY") ON DELETE RESTRICT ON UPDATE CASCADE
);
The strangest thing is that I have other similar tables where this works fine!
sqlite> select * from journalhdr where transno=13;
13|Test transaction ATM Withdrawel 20130213|20130223||20130223||
TransNo in that table is also integer (10,0) NOT NULL - this is what makes me thing it is something to do with the values.
Another clue is that the sort order seems to be based on ascii, not numeric:
sqlite> select * from ledger order by acctno;
0|0|0|JPY|11|Root Account|P|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
1|0|0|JPY|8|Paid-In Capital|C|X|0|X|0|0||||0|0|0|
10|0|0|USD|20|Sallie Mae|L|X|0|X|0|0|SYSTEM|20121209|153900|0|0|0|
21|0|0|USD|21|Skrill|A|X|0|X|0|0|SYSTEM|20121209|154000|0|0|0|
22|0|0|USD|22|AES|L|X|0|X|0|0|SYSTEM|20121209|154200|0|0|0|
23|0|0|JPY|23|Marui|L|X|0|X|0|0|SYSTEM|20121209|154400|0|0|0|
24|0|0|JPY|24|Amex JP|L|X|0|X|0|0|SYSTEM|20121209|154500|0|0|0|
3|0|0|JPY|13|Mitsubishi Bank Futsuu|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
Of course the sort order on journalhdr (where the select works properly) is numeric.
Solved! (sort-of)
The data can be fixed like this:
sqlite> update ledger set acctno = 23 where rowid = 13;
sqlite> select * from ledger where acctno = 25;
25|0|0|JPY|0|Test|L|X|0|X|0|0|SYSTEM|20130224|132500|0|0|0|
Still, if it was stored as strings, then that leave a few questions:
1. Why couldn't I select it as a string using the quotes?
2. How did it get stored as a string since it is a valid integer?
3. How would you go about detecting this problem normally besides noticing bizzarre symptoms?
Although the data would normally be entered by my program, some of it was created by hand using Navicat, so I assume the problem must lie there.
You are victim of SQLite dynamic typing.
Even though SQLite defines system of type affinity, which sets some rules on how input strings or numbers will be converted to actual internal values, but it does NOT prevent software that is using prepared statements to explicitly set any type (and data value) for the column (and this can be different per row!).
This can be shown by this simple example:
CREATE TABLE ledger (acctno INTEGER, name VARCHAR(16));
INSERT INTO ledger VALUES(1, 'John'); -- INTEGER '1'
INSERT INTO ledger VALUES(2 || X'00', 'Zack'); -- BLOB '2\0'
I have inserted second row not as INTEGER, but as binary string containing embedded zero byte. This reproduces your issue exactly, see this SQLFiddle, step by step. You can also execute these commands in sqlite3, you will get the same result.
Below is Perl script that also reproduces this issue
This script creates just 2 rows with acctno having values of integer 1 for first, and "2\0" for second row. "2\0" means string consisting of 2 bytes: first is digit 2, and second is 0 (zero) byte.
Of course, it is very difficult to visually tell "2\0" from just "2", but this is what script below demonstrates:
#!/usr/bin/perl -w
use strict;
use warnings;
use DBI qw(:sql_types);
my $dbh = DBI->connect("dbi:SQLite:test.db") or die DBI::errstr();
$dbh->do("DROP TABLE IF EXISTS ledger");
$dbh->do("CREATE TABLE ledger (acctno INTEGER, name VARCHAR(16))");
my $sth = $dbh->prepare(
"INSERT INTO ledger (acctno, name) VALUES (?, ?)");
$sth->bind_param(1, "1", SQL_INTEGER);
$sth->bind_param(2, "John");
$sth->execute();
$sth->bind_param(1, "2\0", SQL_BLOB);
$sth->bind_param(2, "Zack");
$sth->execute();
$sth = $dbh->prepare(
"SELECT count(*) FROM ledger WHERE acctno = ?");
$sth->bind_param(1, "1");
$sth->execute();
my ($num1) = $sth->fetchrow_array();
print "Number of rows matching id '1' is $num1\n";
$sth->bind_param(1, "2");
$sth->execute();
my ($num2) = $sth->fetchrow_array();
print "Number of rows matching id '2' is $num2\n";
$sth->bind_param(1, "2\0", SQL_BLOB);
$sth->execute();
my ($num3) = $sth->fetchrow_array();
print "Number of rows matching id '2<0>' is $num3\n";
Output of this script is:
Number of rows matching id '1' is 1
Number of rows matching id '2' is 0
Number of rows matching id '2<0>' is 1
If you were to look at resultant table using any SQLite tool (including sqlite3), it will print 2 for second row - they all get confused by trailing 0 inside a BLOB when it gets coerced to string or number.
Note that I had to use custom param binding to coerce type to BLOB and permit null bytes stored:
$sth->bind_param(1, "2\0", SQL_BLOB);
Long story short, it is either some of your client programs, or some of client tools like Navicat which screwed it up.

How to search my database by using an array of words?

I'm attempting to setup a search function from a string a user types. (ex: "John Doe" or "Doe, John")
I was thinking I would use Replace(SearchString, ",", "") to get rid of the commas the user might enter, and then use Split(SearchString, " ") to get all the words into an array. Once they're in the array I would execute a Stored Procedure on each of the terms and build a DataTable with the results.
Below is what I'm wanting to use for executing my stored procedure.
oCommand = DataAccess.GetSQLCommand("MyStoredProcedure", CommandType.StoredProcedure, SourceServer.ConnectionLocal)
oCommand.Parameters.AddWithValue("#MySearchString", SearchString)
oAdapter = New SqlDataAdapter(oCommand)
oAdapter.Fill(MyDataTable)
Now I'm thinking the "SearchString" I will assign while looping through my array of words... but this doesn't seem like the right way to do this. Maybe it is but I don't know how to append my next result to the previous DataTable either.
There are some great ideas for using arrays and Lists in SQL Server on this page - http://www.sommarskog.se/arrays-in-sql-2005.html
I personally find the XML method the most useful;
http://www.sommarskog.se/arrays-in-sql-2005.html#XML
An example of how I've used this in the past is;
DECLARE #indata nvarchar(max)
DECLARE #hDoc int
SET #indata = '
<ROOT>
<SearchTerm code="Test search term"></SearchTerm>
<SearchTerm code="Other search term"></SearchTerm>
<SearchTerm code="Next search term"></SearchTerm>
</ROOT>'
CREATE TABLE #searchTerm (
code varchar(40)
)
EXEC sp_xml_preparedocument #hDoc OUTPUT, #indata
INSERT Into #searchTerm
SELECT code
FROM OPENXML(#hDoc, '/ROOT/SearchTerm',1)
WITH (code varchar(50))
EXEC sp_xml_removedocument #hDoc
-- Use the data in #searchTerm as needed in your query
SELECT * FROM MyTable WHERE searchValue IN (SELECT code FROM #searchTerm)
DROP TABLE #searchTerm
Try passing in the comma separated values in as a single string of nvarchar. Then use the SELECT FROM WHERE IN structure within your stored procedure. Create the sql command wwithin your stored procedure by concatenation and then call EXEC #sql
Declare #sql =
'SELECT * FROM tbl
WHERE person IN(' + #Application + ')'
exec(#sql)
Beware, if your list of search criteria is large this may not be the best solution for you.
Note that the commas should be between the full names not the first name and last name.

Resources