This should be an easy one for someone with more SQLite experience than myself.
I need a select statement to get the name out of the following example string:
{"email":"12345678#facebook.com","user_key":"FACEBOOK:12345678","name":"John Smith"}
The output I need is John Smith.
The number of characters before the name is not always the same so a simple substr command won't work. It needs to be dynamic so it can locate where the name starts and then spit it out. I think ltrim or rtrim may help, but even after researching those commands, I don't understand them very well. Also, SQLite doesn't offer instr or position, which might have been helpful, too!
Edit: the schema for this table is as follows:
CREATE TABLE messages (msg_id TEXT PRIMARY KEY, thread_id TEXT, action_id INTEGER, subject TEXT, text TEXT, sender TEXT, timestamp_ms INTEGER, timestamp_sent_ms INTEGER, attachments TEXT, shares TEXT, msg_type INTEGER, affected_users TEXT, coordinates TEXT, offline_threading_id TEXT, source TEXT, channel_source TEXT, is_non_authoritative INTEGER, pending_send_media_attachment STRING, handled_internally_time INTEGER, pending_shares STRING, pending_attachment_fbid STRING, client_tags TEXT, send_error STRING, send_error_message STRING, send_error_timestamp_ms INTEGER, publicity TEXT, tracking TEXT );
CREATE INDEX messages_offline_threading_id_index ON messages ( offline_threading_id );
CREATE INDEX messages_timestamp_index ON messages ( thread_id, timestamp_ms DESC );
CREATE INDEX messages_type_index ON messages ( thread_id, msg_type, timestamp_ms );
The string I have above that I'm working with is from the sender column.
I don't know the language you're using but most of them support user defined functions (http://www.sqlite.org/c3ref/create_function.html) and you could do
select jsonGetName(columnWithJsonText) from messages where ...
It's like a callback to your programming language where you defined which C/Java/PHP function gets called when using jsonGetName() in sqlite.
In that function (in your language of choice) you decode the json string and return the name property.
Related
Whenever I try to create a table the text stays as a string, im I doing something wrong?
Remove the last comma, just after social integer
c.execute("""CREATE TABLE personalinfo (
name text,
lastname text,
age integer,
social integer
)""")
I have a file filled with sqlite statements I'd like to parse. I've done so previously using the C-interface but now I have trouble with the Tcl interface.
sqlite3 cspdb ":memory:"
set s [read [set f [open csp_sql.txt]]]
set ms [string map {"\n" "\0"} $s]
puts $ms
cspdb eval {$ms}
The error I get is:
"near "$ms": syntax error while executing cspdb eval {$ms}"
It works fine when I paste the output from puts $ms directly into the eval brackets. I also tried "{$ms}" and just plain $ms but the result is the same; syntax error.
I might chose another solution for the problem all together but it really bugs my why it does not work...
Here is what the file looks like:
BEGIN TRANSACTION;
CREATE TABLE Symbol (
Label TEXT PRIMARY KEY,
Type TEXT DEFAULT('no_type')
);
CREATE TABLE Process (
Name INTEGER PRIMARY KEY,
Type TEXT DEFAULT('no_type')
);
CREATE TABLE Named_Process (
Label TEXT UNIQUE,
Definition INTEGER UNIQUE,
FOREIGN KEY(Label) REFERENCES Symbol (Label),
FOREIGN KEY(Definition) REFERENCES Definition(Name)
);
CREATE TABLE Definition (
Name INTEGER UNIQUE,
Definition INTEGER,
FOREIGN KEY(Name) REFERENCES Process(Name),
FOREIGN KEY(Definition) REFERENCES Process(Name)
);
CREATE TABLE Reference (
Name INTEGER UNIQUE,
Reference TEXT,
FOREIGN KEY(Name) REFERENCES Process(Name),
FOREIGN KEY(Reference) REFERENCES Definition(Name)
);
CREATE TABLE Event (
Label TEXT PRIMARY KEY,
Type TEXT DEFAULT('no_type')
);
CREATE TABLE Environment (
Label TEXT UNIQUE,
FOREIGN KEY (Label) REFERENCES Event (Label)
);
CREATE TABLE Prefix (
Name INTEGER UNIQUE,
P INTEGER,
Event TEXT,
FOREIGN KEY (Name) REFERENCES Process (Name),
FOREIGN KEY (P) REFERENCES Process (Name),
FOREIGN KEY (Event) REFERENCES Event (Label)
);
CREATE TABLE Choice (
Name INTEGER UNIQUE,
P INTEGER,
Q INTEGER,
FOREIGN KEY(Name) REFERENCES Process(Name),
FOREIGN KEY(P) REFERENCES Process(Name),
FOREIGN KEY(Q) REFERENCES Process(Name)
);
/* Language defined processes */
/* SKIP */
INSERT INTO Symbol(Label,Type) VALUES('SKIP','named_process');
INSERT INTO Named_Process(Label) VALUES('SKIP');
INSERT INTO Process(type) VALUES('definition');
UPDATE Named_Process SET Definition=last_insert_rowid() WHERE Label='SKIP';
INSERT INTO Definition(Name,Definition) VALUES(last_insert_rowid(),last_insert_rowid());
END TRANSACTION;
(Not so sure removing newlines is necessary...)
If the string that you've read from the file is SQL, you should be able to do this:
sqlite3 cspdb ":memory:"
set f [open "csp_sql.txt"]
set sql [read $f]
close $f
cspdb eval $sql
By comparison, the literal string $ms is not a valid SQL statement or query, nor is it syntactically legal to surround a SQL statement with braces (which is what "{$ms}" ended up doing; the outer "…" makes the inside just a bunch of characters).
I don't know why you are translating newlines into NULs, but that's really unlikely to be a good idea. Bulk import of data (possibly with NULs in it) should be done in a different way.
The eval subcommand created by Sqlite does limited variable substitution, but only in places where a string value is expected.
Try
cspdb eval $ms
to let the Tcl interpreter substitute the SQL statements before sending them to eval.
turns out the newline-to-null was the culprit making substitution fail. final working code:
sqlite3 cspdb ":memory:"
cspdb eval [read [set f [open csp_sql.txt]]]
the reason newline-to-null turned up at all was when trying to debug making the string match my original working C-code which used multiline string literals in which newlines diappeared. Except, "\0" is not a "disappearing" character...
Having an issue where my queries used to work fine with = as in WHERE some_int_field = some_other_int_field. When I do that now I get 0 results. However if I do a WHERE some_int_field LIKE some_other_int_field I get my results. I have checked the length of the fields for hidden characters/spaces and the length of the fields are correct. They are both integer fields. Thoughts? Two tables structure below:
CREATE TABLE "languages"(
"language_id" Integer,
"name" Text,
"english" Text,
"spanish" Text,
"portuguese" Text,
"french" Text );
-- Create index languagesIdx
CREATE INDEX "languagesIdx" ON "languages"( "name" );
BEGIN;
-------------
CREATE TABLE "drop_downs"(
"mode_data" Integer,
"text_index" Integer,
"language_id" Integer );
-- Create index drop_downsIdx
CREATE INDEX "drop_downsIdx" ON "drop_downs"( "mode_data", "language_id" );
BEGIN;
SQLite uses dynamic typing and does not care about the declared type of the fields.
You have strings in your fields.
To check which rows have strings, use something like this:
SELECT * FROM drop_downs WHERE typeof(mode_data) = 'text'
To convert all values in a column into numbers, use something like this:
UPDATE drop_downs SET mode_data = CAST(mode_data AS integer)
I'm getting a confusing select...where result, the table definition is:
CREATE TABLE modes (
key INTEGER,
mode INTEGER,
channel INTEGER,
name TEXT,
short_name TEXT,
def INTEGER,
highlight INTEGER,
catagory TEXT,
subcatagory TEXT);
It's populated with:
sqlite> select * from modes;
3|6|5|Green|G|0|255|a|b
3|6|6|Blue|B|0|255|a|b
3|9|1|Mode|Mode|0|255|a|b
3|9|2|Auto Mode Speed|Speed|0|255|a|b
3|9|3|Strobe|Strobe|0|255|a|b
3|9|4|Red|R|0|255|a|b
3|9|5|Green|G|0|255|a|b
3|9|6|Blue|B|0|255|a|b
3|9|7|Red2|R2|0|255|a|b
3|9|8|Green2|G2|0|255|a|b
3|9|9|Blue2|B2|0|255|a|b
3|6|4|Red|R|0|255|a|b
3|6|1|6|6|0|255|a|b
3|6|2|Auto mode speed|speed|0|255|a|b
3|6|3|Strobe|Strobe|0|255|strobe|b
Note the row 3rd from the bottom:
3|6|1|6|6|0|255|a|b
If I do a select:
SELECT * FROM modes where mode=6 and name="Mode" order by channel;
It returns:
3|6|1|6|6|0|255|a|b
Columns 4 and 5 (name and short_name) should not match, they are 6 and the match term is "Mode". If I change the match string "Mode" to any other string it works as expected. Is "Mode" a reserved word? or Did I somehow set a variable "Mode" to 6?. I don't understand this behavior.
If you use ["] it means columns, so when you do
name="Mode"
it means you search in column name that have same value as column Mode. So you select where mode=6 and name=6 actually on your code.
If you want to use string, use ['] not ["]
I try to use that code on my database..
select * from products
where name="Mode"
And I got error
ERROR: column "Mode" does not exist
LINE 2: where name="Mode"
I finally found it in the docs if anyone else is looking.
it's in the the sqlite3 FAQ
My familiarity is not with sqlite3, but I would say that it looks to me like you are doing name="Mode" which is taking the modes.mode object and checking it.
If you do SELECT * FROM modes where name="Mode" order by channel. You may find that it just gives you 3|6|1|6|6|0|255|a|b as well.
Ok, so I have a basic table called "ledger", it contains fields of various types, integers, varchar, etc.
In my program, I used to use a query with no "from" predicate to collect all of the rows, which of course works fine. But... I changed my code to allow selecting one row at a time using "where acctno = x" (where X is the account number I want to select at the time).
I thought this must be a bug in the client library for my programming language, so I tested it in the SQLite command-line client - and it still doesn't work!
I am relatively new to SQLite, but I have been using Oracle, MS SQL Server, etc. for years and never seen this type of issue before.
Other things I can tell you:
* Queries using other integer fields also don't work
* Queries on char fields work
* Querying it as a string (with the account number on quotes) still doesn't work. (I thought maybe the numbers were stored as a string inadvertently).
* Accessing rows by rowid works fine - which is why I can edit the database with GUI tools with no noticeable problem.
Examples:
Query with no WHERE (works fine):
1|0|0|JPY|8|Paid-In Capital|C|X|0|X|0|0||||0|0|0|
0|0|0|JPY|11|Root Account|P|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
3|0|0|JPY|13|Mitsubishi Bank Futsuu|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
4|0|0|JPY|14|Japan Post Bank|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
...
Query with WHERE clause: (no results)
sqlite> select * from ledger where acctno=1;
sqlite>
putting quotes around the 1 above changes nothing.
Interestingly enough, "select * from ledger where acctno > 1" returns results! However since it returns ALL results, it's not terrible useful.
I'm sure someone will ask about the table structure, so here goes:
sqlite> .schema ledger
CREATE TABLE "LEDGER" (
"ACCTNO" integer(10,0) NOT NULL,
"drbal" integer(20,0) NOT NULL,
"crbal" integer(20,0) NOT NULL,
"CURRKEY" char(3,0) NOT NULL,
"TEXTKEY" integer(10,0),
"TEXT" VARCHAR(64,0),
"ACCTYPECD" CHAR(1,0) NOT NULL,
"ACCSTCD" CHAR(1,0),
"PACCTNO" number(10,0) NOT NULL,
"CATCD" number(10,0),
"TRANSNO" number(10,0) NOT NULL,
"extrefno" number(10,0),
"UPDATEUSER" VARCHAR(32,0),
"UPDATEDATE" text(8,0),
"UPDATETIME" TEXT(6,0),
"PAYEECD" number(10,0) NOT NULL,
"drbal2" number(10,0) NOT NULL,
"crbal2" number(10,0) NOT NULL,
"delind" boolean,
PRIMARY KEY("ACCTNO"),
CONSTRAINT "fk_curr" FOREIGN KEY ("CURRKEY") REFERENCES "CURRENCY" ("CUR
RKEY") ON DELETE RESTRICT ON UPDATE CASCADE
);
The strangest thing is that I have other similar tables where this works fine!
sqlite> select * from journalhdr where transno=13;
13|Test transaction ATM Withdrawel 20130213|20130223||20130223||
TransNo in that table is also integer (10,0) NOT NULL - this is what makes me thing it is something to do with the values.
Another clue is that the sort order seems to be based on ascii, not numeric:
sqlite> select * from ledger order by acctno;
0|0|0|JPY|11|Root Account|P|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
1|0|0|JPY|8|Paid-In Capital|C|X|0|X|0|0||||0|0|0|
10|0|0|USD|20|Sallie Mae|L|X|0|X|0|0|SYSTEM|20121209|153900|0|0|0|
21|0|0|USD|21|Skrill|A|X|0|X|0|0|SYSTEM|20121209|154000|0|0|0|
22|0|0|USD|22|AES|L|X|0|X|0|0|SYSTEM|20121209|154200|0|0|0|
23|0|0|JPY|23|Marui|L|X|0|X|0|0|SYSTEM|20121209|154400|0|0|0|
24|0|0|JPY|24|Amex JP|L|X|0|X|0|0|SYSTEM|20121209|154500|0|0|0|
3|0|0|JPY|13|Mitsubishi Bank Futsuu|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
Of course the sort order on journalhdr (where the select works properly) is numeric.
Solved! (sort-of)
The data can be fixed like this:
sqlite> update ledger set acctno = 23 where rowid = 13;
sqlite> select * from ledger where acctno = 25;
25|0|0|JPY|0|Test|L|X|0|X|0|0|SYSTEM|20130224|132500|0|0|0|
Still, if it was stored as strings, then that leave a few questions:
1. Why couldn't I select it as a string using the quotes?
2. How did it get stored as a string since it is a valid integer?
3. How would you go about detecting this problem normally besides noticing bizzarre symptoms?
Although the data would normally be entered by my program, some of it was created by hand using Navicat, so I assume the problem must lie there.
You are victim of SQLite dynamic typing.
Even though SQLite defines system of type affinity, which sets some rules on how input strings or numbers will be converted to actual internal values, but it does NOT prevent software that is using prepared statements to explicitly set any type (and data value) for the column (and this can be different per row!).
This can be shown by this simple example:
CREATE TABLE ledger (acctno INTEGER, name VARCHAR(16));
INSERT INTO ledger VALUES(1, 'John'); -- INTEGER '1'
INSERT INTO ledger VALUES(2 || X'00', 'Zack'); -- BLOB '2\0'
I have inserted second row not as INTEGER, but as binary string containing embedded zero byte. This reproduces your issue exactly, see this SQLFiddle, step by step. You can also execute these commands in sqlite3, you will get the same result.
Below is Perl script that also reproduces this issue
This script creates just 2 rows with acctno having values of integer 1 for first, and "2\0" for second row. "2\0" means string consisting of 2 bytes: first is digit 2, and second is 0 (zero) byte.
Of course, it is very difficult to visually tell "2\0" from just "2", but this is what script below demonstrates:
#!/usr/bin/perl -w
use strict;
use warnings;
use DBI qw(:sql_types);
my $dbh = DBI->connect("dbi:SQLite:test.db") or die DBI::errstr();
$dbh->do("DROP TABLE IF EXISTS ledger");
$dbh->do("CREATE TABLE ledger (acctno INTEGER, name VARCHAR(16))");
my $sth = $dbh->prepare(
"INSERT INTO ledger (acctno, name) VALUES (?, ?)");
$sth->bind_param(1, "1", SQL_INTEGER);
$sth->bind_param(2, "John");
$sth->execute();
$sth->bind_param(1, "2\0", SQL_BLOB);
$sth->bind_param(2, "Zack");
$sth->execute();
$sth = $dbh->prepare(
"SELECT count(*) FROM ledger WHERE acctno = ?");
$sth->bind_param(1, "1");
$sth->execute();
my ($num1) = $sth->fetchrow_array();
print "Number of rows matching id '1' is $num1\n";
$sth->bind_param(1, "2");
$sth->execute();
my ($num2) = $sth->fetchrow_array();
print "Number of rows matching id '2' is $num2\n";
$sth->bind_param(1, "2\0", SQL_BLOB);
$sth->execute();
my ($num3) = $sth->fetchrow_array();
print "Number of rows matching id '2<0>' is $num3\n";
Output of this script is:
Number of rows matching id '1' is 1
Number of rows matching id '2' is 0
Number of rows matching id '2<0>' is 1
If you were to look at resultant table using any SQLite tool (including sqlite3), it will print 2 for second row - they all get confused by trailing 0 inside a BLOB when it gets coerced to string or number.
Note that I had to use custom param binding to coerce type to BLOB and permit null bytes stored:
$sth->bind_param(1, "2\0", SQL_BLOB);
Long story short, it is either some of your client programs, or some of client tools like Navicat which screwed it up.