It seems .import will only fail if the last record fails. Below I use .bail on and the script continues after the first two imports despite them both causing errors.
.bail on
PRAGMA foreign_keys = ON;
create table x (a primary key);
create table y (c references x(a));
.import --csv "|echo 2" x
.import --csv "|seq 2" y
.print 'Did not bail despite failing to insert 1'
create table z (a, b NOT NULL);
.import --csv "|printf 'a\nb,c\n'" z
.print "Did not bail despite failing to insert `a`"
.import --csv "|printf '1,2\n3\n'" z
.print 'This will not print because in this case, the bad record was at the
end'
This outputs:
<pipe>:1: INSERT failed: FOREIGN KEY constraint failed
Did not bail despite failing to insert 1
<pipe>:1: expected 2 columns but found 1 - filling the rest with NULL
<pipe>:1: INSERT failed: NOT NULL constraint failed: z.b
Did not bail despite failing to insert `a`
<pipe>:2: expected 2 columns but found 1 - filling the rest with NULL
<pipe>:2: INSERT failed: NOT NULL constraint failed: z.b
Related
I've just started learning Databases, and the instructor is teaching SQLite with the command line shell. I downloaded sqlite-tools-win32-x86-3350400.zip from sqlite.org/download
When I enter false commands, like the first two you can see, I get the correct error messages. But when I enter a syntax error: ("Steve, 87654) for example, it's stuck on ...> waiting for more input.
After searching on the internet, users suggested the solution "CTRL+D", but it doesn't work for me. CTRL+C however just exits sqlite3.
My question: How do I get out of ...> to continue with my commands? Why CTRL+D doesn't work for me, to get out of the ...> because of the syntax error or other possible causes? How do I fix my problem?
C:\Users\myUser>sqlite3 test.db
SQLite version 3.35.4 2021-04-02 15:20:15
Enter ".help" for usage hints.
sqlite> SELECT * FROM contacts;
Tim|654321|tim#email.com
Brian|1234|brian#mygmail.com
sqlite> INSERT INTO contacts VALUES("Steve", 87654);
Error: table contacts has 3 columns but 2 values were supplied
sqlite> INSERT INTO contacts VALUES(Steve, 87654);
Error: no such column: Steve
sqlite> INSERT INTO contacts VALUES("Steve, 87654); // Intentional syntax error //
...> ^D // Entered CTRL+D //
...> ;
...> // Entered CTRL+C //
Error: unrecognized token: ""Steve, 87654)"
C:\Users\myUser>
You need to
a) close the quotes using " and
b) finish the command line using ;
can do both together (as in the first example)
e.g. :-
sqlite> INSERT INTO contacts VALUES("Steve,87654);
...> anything
...> ^D
...> ;
...> '
...> ";
Error: near ";": syntax error
sqlite> INSERT INTO contacts VALUES("Steve,87654);
...> "
...> ;
Error: near ";": syntax error
Of course you could even do something like :-
sqlite> INSERT INTO contacts VALUES("Steve,87654);
sqlite> INSERT INTO contacts VALUES("Steve,87654);
...> ",0987,'steve#email.com');
sqlite> SELECT * FROM contacts;
Tim|654321|tim#email.com
Brain|1234|brian#mygmail.com
Steve,87654);
|987|steve#email.com
This question already has an answer here:
UPSERT for "INSERT INTO tab SELECT * FROM another_tab"
(1 answer)
Closed 2 years ago.
I am trying to load a daily CSV load file into a main table, so that all new records are inserted and all existing records are updated. I am using UPSERT feature of Sqlite for this purpose.
Here is my shell script.
LOAD_FILE="/tmp/main.csv"
LOAD_TABLE="tbl_tmp_main"
MAIN_TABLE="tbl_main"
COLUMNS="t_id, t_col1, t_col2"
CREATE_TABLE_QUERY="DROP TABLE IF EXISTS $LOAD_TABLE; CREATE TABLE $LOAD_TABLE AS SELECT $COLUMNS FROM $MAIN_TABLE WHERE false"
LOAD_TABLE_QUERY=".separator ','\n.import '$LOAD_FILE' $LOAD_TABLE"
UPSERT_TABLE_QUERY="
INSERT INTO $MAIN_TABLE($COLUMNS) SELECT $COLUMNS FROM $LOAD_TABLE
ON CONFLICT(t_id) DO UPDATE
SET
t_col1 = excluded.t_col1,
t_col2 = excluded.t_col2
;
"
echo ""
echo "$CREATE_TABLE_QUERY" | sqlite3 mydatabase.sqlite3 # <-- This works
echo "[INFO] Temporary Table Created."
echo "$LOAD_TABLE_QUERY" | sqlite3 mydatabase.sqlite3 # <-- This works
echo "[INFO] Data Loaded into Temporary Table."
echo "$UPSERT_TABLE_QUERY" | sqlite3 mydatabase.sqlite3 # <-- This errors out
echo "[INFO] Records Inserted/Updated."
echo ""
Here is the error.
Error: near line 2: near "DO": syntax error
I tried to check the documentation, but I am not sure what is wrong in the query above. Can somebody point out why it isnt working.
Here is my sqlite version
-> sqlite3 --version
3.34.1 2021-01-20 14:10:07 10e20c0b43500cfb9bbc0eaa061c57514f715d87238f4d835880cd846b9ebd1f
The error is due to Parsing Ambiguity as mentioned in the documentation. You need to add WHERE true as part of the SELECT clause and that should fix it.
Here is your updated UPSERT query.
INSERT INTO $MAIN_TABLE($COLUMNS) SELECT $COLUMNS FROM $LOAD_TABLE WHERE true
-- ^^^^^^^^^^
ON CONFLICT(t_id) DO UPDATE
SET
t_col1 = excluded.t_col1,
t_col2 = excluded.t_col2
;
Here is the section in Docs.
Parsing Ambiguity
When the INSERT statement to which the UPSERT is attached takes its values from a SELECT statement, there is a potential parsing ambiguity. The parser might not be able to tell if the "ON" keyword is introducing the UPSERT or if it is the ON clause of a join. To work around this, the SELECT statement should always include a WHERE clause, even if that WHERE clause is just "WHERE true".
Ambiguous use of ON:
INSERT INTO t1 SELECT * FROM t2
ON CONFLICT(x) DO UPDATE SET y=excluded.y;
Ambiguity resolved using a WHERE clause:
INSERT INTO t1 SELECT * FROM t2 WHERE true
ON CONFLICT(x) DO UPDATE SET y=excluded.y;
these both statements seem to make a difference, but I do not yet get why:
ALTER TABLE FOO ADD COLUMN DELETED BOOLEAN NOT NULL DEFAULT FALSE
seems to behave differently than:
ALTER TABLE FOO ADD COLUMN DELETED BOOLEAN NOT NULL DEFAULT 0
can anyone shed some light on this - I thought FALSE is 0 and TRUE is 1 - but this boolean seems to have >= 4 states:
➜ ~ sqlite3
SQLite version 3.8.7.4 2014-12-09 01:34:36
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> CREATE TABLE FOO ( id INTEGER );sqlite> INSERT INTO FOO ( id ) VALUES (1);
sqlite> select * from FOO;
1
sqlite> ALTER TABLE FOO ADD COLUMN DELETED BOOLEAN NOT NULL DEFAULT FALSE;sqlite> select * from FOO;
1|FALSE
sqlite> ALTER TABLE FOO ADD COLUMN DELETED2 BOOLEAN NOT NULL DEFAULT 0;
sqlite> select * from FOO;
1|FALSE|0
sqlite> ALTER TABLE FOO ADD COLUMN DELETED3 BOOLEAN NOT NULL DEFAULT TRUE;
sqlite> select * from FOO;
1|FALSE|0|TRUE
sqlite> ALTER TABLE FOO ADD COLUMN DELETED4 BOOLEAN NOT NULL DEFAULT 1;
sqlite> select * from FOO;
1|FALSE|0|TRUE|1
sqlite> select * from FOO WHERE DELETED;
sqlite> select * from FOO WHERE DELETED2;
sqlite> select * from FOO WHERE DELETED3;
sqlite> select * from FOO WHERE DELETED4;
1|FALSE|0|TRUE|1
Why exactly SQLite allows these ALTER TABLE statements to run as shown I don't know but the missing piece of the puzzle here is that those FALSE and TRUE values will be stored as strings.
Try this in the SQLite command line tool:
CREATE TABLE test (id integer, xx boolean not null default TRUE);
INSERT INTO test (id) VALUES (1)
.dump
You'll get this output:
sqlite> .dump
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE test (id integer, xx boolean not null default TRUE);
INSERT INTO "test" VALUES(1,'TRUE');
COMMIT;
As you can see, the TRUE value there is stored as a string.
Now, let's try to query for the TRUE value:
SELECT * FROM test WHERE xx = TRUE
this gives this error message:
Error: no such column: TRUE
So in short, TRUE or FALSE are not magical constants in SQLite SQL syntax for the boolean values. Why the ALTER TABLE statements allow for them to be specified without quotes I don't know, there's probably a good reason.
Here's the example from my comment:
ALTER TABLE test ADD COLUMN xy INTEGER NOT NULL DEFAULT MONKEYDOODLE;
Output from .dump:
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE test (id integer, xx boolean not null default TRUE, xy INTEGER NOT
NULL DEFAULT MONKEYDOODLE);
INSERT INTO "test" VALUES(1,'TRUE','MONKEYDOODLE');
COMMIT;
One final piece of the puzzle is also that SQLite doesn't really prevent you from storing values of different data types in the same column.
From the documentation:
Any column in an SQLite version 3 database, except an INTEGER PRIMARY KEY column, may be used to store a value of any storage class.
You can see this in effect here:
CREATE TABLE test (id integer);
INSERT INTO test VALUES (1);
INSERT INTO test VALUES ('test');
INSERT INTO test VALUES (DATE());
INSERT INTO test VALUES (10.5);
This is the output from .dump:
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE test (id integer);
INSERT INTO "test" VALUES(1);
INSERT INTO "test" VALUES('test');
INSERT INTO "test" VALUES('2015-07-20');
INSERT INTO "test" VALUES(10.5);
COMMIT;
When I run sqlite3 foo.db from cmd (Windows) and enter these commands (assuming TABLE 'test' does not exist):
sqlite> CREATE TABLE test (id integer PRIMARY KEY, name char(1));
sqlite> INSERT INTO test (name) VALUES ('aaaaaa');
there are no errors. To verify,
//Input
sqlite> SELECT * FROM test;
//Output
1|aaaaaa
Again, to verify,
sqlite> .schema test
CREATE TABLE test (id integer PRIMARY KEY, name char(1)); //output
and the schema isn't changed.
Is there something wrong, especially with the name char(1) part? For the record, I compiled SQLite3 using MinGW64 with
--host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --prefix=/mingw // where /mingw is included in my PATH
Thanks in advance.
As there is no types in SQLite, the question is meaningless. All TEXT values are the same and are stored as "unlimited" (up to SQLITE_MAX_LENGTH) length strings.
What you see is a WAD: "working as designed".
SQLite doesn't enforce the length of a char(n) or varchar(n) column as it's declared in a CREATE TABLE statement. But you can enforce length restrictions using a CHECK constraint.
sqlite> create table test (
...> id integer primary key,
...> name char(1),
...> check (length(name)<=1)
...> );
sqlite> INSERT INTO test (name) VALUES ('aaaaaa');
Error: CHECK constraint failed: test
I am trying to create 3 tables but I am getting this error:
CREATE TABLE dj_abonent
(
dj_klientID INT NOT NULL PRIMARY KEY,
emer_klienti varchar2(10),
mbiemer_klienti VARCHAR2(10),
sasia_cel INT
);
CREATE TABLE dj_phones
(
phone_number varchar2(12),
activated number(1) default 0,
activation_date date default null,
CONSTRAINT dj_phone_number_check
CHECK (substr(phone_number,1,5) in( '35566','35567','35568','35569') ),
CONSTRAINT dj_activated_check
CHECK (activated in(1,0) )
dj_KlientID int FOREIGN KEY REFERENCES dj_Abonenti(dj_KlientID)
);
CREATE TABLE dj_telef
(
start_time date,
end_time date,
abonent_1 varchar2(10),
abonent_2 varchar2(10)
);
Error at Command Line : 26 Column : 17
Error report -
SQL Error: ORA-00907: missing right parenthesis
00907. 00000 - "missing right parenthesis"
*Cause:
*Action:
The line number is from your SQL Developer script window, which isn't entirely helpful as it doesn't seem to align with the issue. There may be other things too but you're missing a comma after your check constraint (just like a previous question). But you should put the constraints at the end of the command:
CREATE TABLE dj_phones
(
phone_number varchar2(12),
activated number(1) default 0,
activation_date date default null,
dj_KlientID int FOREIGN KEY REFERENCES dj_Abonenti(dj_KlientID)
CONSTRAINT dj_phone_number_check
CHECK (substr(phone_number,1,5) in( '35566','35567','35568','35569') ),
CONSTRAINT dj_activated_check
CHECK (activated in(1,0) )
);
You might find it easier to debug these issues if you ran one statement at a time, either with the run statement command (control-enter), or by highlighting the text one one command and using run script (F5).