I have a table called deliverysimp into which I am trying to insert some data. I am aware that the data types for the columns are just affinities and not restrictions, however I need to store the parcelid column below as TEXT.
CREATE TABLE IF NOT EXISTS deliverysimp (parcelid TEXT, expected integer, primary key (parcelid))
I am using the following javascript to insert the data to the database:
context.executeSql("INSERT INTO deliverysimp(parcelid, expected) values(?,?)",
[
'' + delivery.parcelid,
delivery.expected
], function () { }, me.ErrorHandler);
You can see I have tried to add a blank '' + before the parcelid to try and force the affinity, but the behaviour is the same without; namely:
if I try to store the parcelid 33333333333322222222222222222222223 this is stored into the database as 3.3333333333322223e+34 and I need this to be a text/string representation.
Any ideas how I can get SQLite to honour this as TEXT?
I suspect that you already have a string, just not the string you expected. Since the number you have cannot be represented by an 8-byte integer, it gets converted into a real number and that gets converted into a string, i.e., '3.3333333333322223e+34'. So, if you want the value to be '33333333333322222222222222222222223', then that's what you have to insert into the table.
To check, do a SELECT parcelid, TYPEOF(parcelid) FROM deliverysimp; using the sqlite3 command-line tool and see what you get.
Related
After creating an sqlite table with a generated column in it, the type only shows up, if it was specified, and there can be cases when junk gets into the type description as well.
Example:
create table test(
id integer primary key not null,
gen generated always as (id * 2) stored
);
Using pragma table_xinfo(test); afterwards nets the following output:
0|id|INTEGER|1||1|0
1|gen||0||0|3
The type is simply missing from the correct column.
If the column were to be created with:
gen integer generated always as (id * 2) stored
instead, then the type would correctly show up as INTEGER.
Are there any methods that would get the type of a column in a table without having to resort to parsing the table creation code?
Nevermind, as usual, I find the answer right after asking it. According to sqlite documentation:
The datatype and collating sequence of the generated column are determined only by the datatype and COLLATE clause on the column definition. The datatype and collating sequence of the GENERATED ALWAYS AS expression have no affect on the datatype and collating sequence of the column itself.
Which I assume means, that just as in other places, if the datatype is not specified, it is thought of as a blob.
I have a table with like this:
id
values
user_id
1
["8","7","6"]
5
Now I'm running a query with WHERE condition on values column:
SELECT * from table_name WHERE values = ["8","7","6"]
But MySQL returns this error:
Error Code : 1064
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '["8","7","6"]'
If you want to compare for strict equality, you want to do the comparison as JSON objects. You can do this by using JSON_EXTRACT to parse the data as JSON instead of text:
SELECT * from table_name WHERE
JSON_EXTRACT(`values`, '$') = JSON_EXTRACT('["8","7","6"]', '$');
You should be able to use this for any type of JSON as long as you want strict equality. If you want to return all rows that match the given JSON object, use JSON_CONTAINS.
For example to find all values with the string "8" in it, you'd use the following:
SELECT * from table_name WHERE JSON_CONTAINS(`values`, '"8"');
Note that this matching is not as simple as you'd expect and matches any value in the document. If your data consists of JSON arrays, this should still be adequate.
The information about your column datatype, especially values are crucial. Since the column stores a mix of numbers and non-numbers characters, we can assume that it might be stored in VARCHAR() or TEXT datatype. But since the data format looks like a JSON array, it's also a possibility that the column datatype is JSON. Now both of these datatypes have a very different query structure.
First, let's address some issues:
Whenever the cell values include other characters than numerical, it will be considered as string. Hence, using plain .. WHERE values = ["8","7","6"] without wrapping it in quotes ('), you'll get that Error Code : 1064.
VALUES is a reserved word in MySQL so if you want to stick to it as your table column names, you always need to wrap it in backticks. If not, this will also return Error Code : 1064:
.. WHERE `values` = ..
Now let's try this:
If the column datatype for values is VARCHAR() or TEXT, you just have to simply wrap the search value in single quote like:
SELECT * from table_name WHERE `values` = '["8","7","6"]';
Refer this fiddle
updated for MariaDB
If the column datatype for values is JSON, it's something like this:
SELECT * from table_name where JSON_UNQUOTE(`values`)= '["8","7","6"]'
Refer this fiddle for JSON
The JSON method I've referred to this MariaDB documentation.
P/S: According to this documentation JSON is an alias for LONGTEXT introduced for compatibility reasons with MySQL's JSON data type. In other words, when creating a table with JSON datatype in MariaDB, it will be shown as LONGTEXT but with extra definition than just plain LONGTEXT datatype. See this fiddle for more detail.
In a React Native App I'm attempting to insert data into a local sqlite db
let submissionID = "1-2-3";
this.dbQuery("INSERT INTO Submissions (ID, Data) VALUES("+submissionID+",'Test')");
(dbQuery is the name of a function I made to simplify my queries but the statement inside it should be the same)
If I viewed the Submissions table after this insert statement I would expect to see a row with [ID:"1-2-3",Data:"Test"] but instead I see [ID:"-4",Data:"Test"]
I created the table like so
CREATE TABLE IF NOT EXISTS Submissions(ID BLOB PRIMARY KEY NOT NULL, Data BLOB NOT NULL)
I used Blob because I read "The value is a blob of data, stored exactly as it was input." but I've also tried Text. I've also casted submissionID as a string like so
this.dbQuery("INSERT INTO Submissions (ID, Data) VALUES("+String(submissionID)+",'Test')");
But none of that worked. I do see here how sqlite takes advantage of arithmetic operators
https://www.w3resource.com/sqlite/arithmetic-operators.php
but I'm not sure how to stop it from doing so.
How would I get sqlite to treat my hyphens as hyphens instead of subtraction signs?
What you're doing is the equivalent of:
this.dbQuery("INSERT INTO Submissions (ID, Data) VALUES(1-2-3,'Test')");
passing the numeric expression 1-2-3 to the INSERT statement. The simplest fix is to quote the string literal.
let submissionID = "1-2-3";
this.dbQuery("INSERT INTO Submissions (ID, Data) VALUES('"+submissionID+"','Test')");
However, to guard against SQL injection attacks, you really ought to be using prepared statements instead of using string concatenation to build SQL statements.
Enclose the string in single quotes i.e.
this.dbQuery("INSERT INTO Submissions (ID, Data) VALUES('"+String(submissionID)+"','Test')");
Thus the value is treated as a literal by SQLite, without enclosing the value it will either be treated as a numeric value or as an identifier (column, table, trigger, view depending upon where it is coded and thus what the parser expects).
The data type (column affinity) has little bearing other than if you specified ID INTEGER PRIMARY KEY, then you could not store anything other than an integer. As ID INTEGER PRIMARY key has a special interpretation that is the column is an alias of the rowid.
I used Blob because I read "The value is a blob of data, stored
exactly as it was input." but I've also tried Text. I've also casted
submissionID as a string like so
That is only if the value to be inserted is a BLOB byte[] or in the case of raw SQL x'FF01FE02', otherwise SQLite will store the value according to how it interprets the type should be stored.
I'm trying to append zeroes to a blob field in a sqlite3 db.
What I tried is this:
UPDATE Logs
SET msg_buffer = msg_buffer || zeroblob(1)
WHERE msg_id = 'F0'
As you can see, the field name is "msg_buffer", and what I want is to append byte zero. It seems that the concat operator || doesn't work.
How could I achieve this?
Reading the doc link posted by Googie (3.2. Affinity Of Expressions), I managed to find the way:
UPDATE Logs
SET msg_buffer = CAST(msg_buffer || zeroblob(1) AS blob)
WHERE msg_id = 'F0'
The CAST operator can take the expression with the concatenate operator and force the blob affinity.
SQLite3 does support datatypes. See https://www.sqlite.org/datatype3.html
They are not strictly linked with declared type of a column, but rather individual per each cell value. The type is determined by how it was created/modified. For example if you insert 5, it will be INTEGER. If you insert 5.5, it will be REAL. If you insert 'test' it will be TEXT, if you insert zeroblob(1), it will be BLOB and if you insert null, it will be NULL.
Now, what you are doing is that you're trying to concatenate current value with a BLOB type. If current value is TEXT (or basically if you use || operator, as you do, you are converting any type into a TEXT), it will be concatenated with byte \x00, which actually determines the end of a string. In other words, you are adding yet another string terminator, to an already existing one, that the TEXT type has.
There will be no change on output of this operation. TEXT always ends with byte zero and it is always excluded from the result, as it's a meta character, not the value itself.
Additional information from http://sqlite.1065341.n5.nabble.com/Append-data-to-a-BLOB-field-td46003.html - appending binary data to BLOB field is not possible. You can modify prealocated blob:
Append is not possible. But if you preallocate space using
zeroblob() or similar, you can write to it using the incremental
blob API:
http://www.sqlite.org/c3ref/blob_open.html
Finally, please see accepted answer, as author of the question found an interesting solution.
I have a SQlite3 table that has typeless columns like in this example:
CREATE TABLE foo(
Timestamp INT NOT NULL,
SensorID,
Value,
PRIMARY KEY(Timestamp, SensorID)
);
I have specific reasons not to declare the type of the columns SensorID and Value.
When inserting rows with numeric SensorID and Value columns I notice that they are being written as plain text into the .db file.
When I change the CREATE TABLE statement to...
CREATE TABLE foo(
Timestamp INT NOT NULL,
SensorID INT,
Value REAL,
PRIMARY KEY(Timestamp, SensorID)
);
...then the values seem to be written in some binary format to the .db file.
Since I need to write several millions of rows to the database, I have concerns about the file size this data produces and so would like to avoid value storage in plain text form.
Can I force SQLite to use binary representation in it's database file without using explicitly typed columns?
Note: Rows are currently written with PHP::PDO using prepared statements.
The example in section 3.4 in the sqlite docs about types demonstrates the insertion of a number as int in a column without an explicit declaration of type. I guess the trick is leaving out the quotes around the number, which would convert it to a string (which, in the case of typed columns, would be coerced back into a number).
Section 2 in the page linked above also provides a lot of info about the type conversions taking place.