I have the following query working in pure SQL on SQLite but do not know how to convert it to pyDAL:
SELECT * FROM buy WHERE date('now','-2 days') < timestamp;
The buy table schema is:
CREATE TABLE "buy"(
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"order_id" CHAR(512),
"market" CHAR(512),
"purchase_price" DOUBLE,
"selling_price" DOUBLE,
"amount" DOUBLE
, "timestamp" TIMESTAMP, "config_file" CHAR(512));
Instead of using the SQLite date function, you can create the comparison date in Python:
import datetime
cutoff_time = datetime.datetime.now() - datetime.timedelta(2)
rows = db(db.buy.timestamp > cutoff_time).select()
Alternatively, you can also pass a raw SQL string as the query:
rows = db('buy.timestamp > date("now", "-2 days")').select(db.buy.ALL)
Note, in this case, because the query within db() is simply a string, the DAL will not know which table is being selected, so it is necessary to explicitly specify the fields in the .select() (alternatively, you could add a dummy query that selects all records, such as (db.buy.id != None)).
Related
I constantly retrieve JSON data from some API and put that data into a MariaDB table.
The JSON ships with a timestamp which I'd like to place an index on, because this attribute is used for querying the table.
The JSON looks something like this (stripped):
{
"time": "2021-12-26T14:00:00.007294Z",
"some_measure": "0.10031"
}
I create a table:
CREATE TABLE some_table (
my_json JSON NOT NULL,
time TIMESTAMP AS (JSON_VALUE(my_json , '$.time')),
some_measure DOUBLE AS (JSON_VALUE(my_json , '$.some_measure'))
)
ENGINE=InnoDB
DEFAULT CHARSET=utf8mb4
COLLATE=utf8mb4_general_ci;
my_json holds the entire JSON snippet, time and some_measure are virtual columns properly extracting the corresponding JSON values on the fly.
Now, trying to add an index on the TIMESTAMP attribute:
CREATE INDEX some_index ON some_table (time);
This fails:
SQL Error [1292] [22007]: (conn=454) Incorrect datetime value:
'2021-12-26T14:00:00.007294Z' for column `some_db`.`some_table`.`time` at row 1
How can I add an index on that timestamp?
The issue here is that converting a string (the JSON timestamp) to a TIMESTAMP is non-deterministic because it involves server side settings (sql_mode) and timezone settings.
Indexing virtual columns which are non-deterministic is not supported.
You would want to use a VARCHAR data type instead and index that:
CREATE TABLE some_table (
my_json JSON NOT NULL,
time VARCHAR(100) AS (JSON_VALUE(my_json , '$.time')),
some_measure DOUBLE AS (JSON_VALUE(my_json , '$.some_measure'))
)
ENGINE=InnoDB
DEFAULT CHARSET=utf8mb4
COLLATE=utf8mb4_general_ci;
You should be able to create your index:
CREATE INDEX some_index ON some_table (`time`);
You can still query time because MariaDB automatically converts DATETIMEs if used against a VARCHAR:
SELECT
*
FROM some_table
WHERE time > '2008-12-31 23:59:59' + INTERVAL 1 SECOND;
The query will use the index:
I finally came up with a solution that works for me.
Changes are:
use STR_TO_DATE() to create a valid DATETIME from the JSON timestamp
make the generated (virtual) column PERSISTENT
use data type DATETIME instead of TIMESTAMP
So the new code looks like this:
CREATE TABLE some_table (
my_json JSON NOT NULL,
time DATETIME AS (STR_TO_DATE((JSON_VALUE(my_json , '$.time')), '%Y-%m-%d%#%T%.%#%#')) PERSISTENT,
some_measure DOUBLE AS (JSON_VALUE(my_json , '$.some_measure'))
)
ENGINE=InnoDB
DEFAULT CHARSET=utf8mb4
COLLATE=utf8mb4_general_ci;
CREATE INDEX some_index ON some_table (`time`);
I'm trying to find all the results between two dates in SQLite.
select log_ID
from Message
where Timestamp between DATE('2014-04-17 03:27:08','YYYY-MM-DD HH:MI:SS')
and DATE('2014-04-18 03:27:08','YYYY-MM-DD HH:MI:SS');
This query is executing successfully but it gives no results.
I've tried using DATE and TO_DATE functions as well.
Schema:
CREATE TABLE Message(
Log_ID INTEGER PRIMARY KEY,
Session_ID TEXT NOT NULL,
Timestamp DATETIME NOT NULL,
Admill_Msg TEXT, Logcat_Msg TEXT);
Sample tuples:
155|admil.out.txt|2014-04-17 03:26:48.730000||PID:926 TID:926 TAG:I/Zygote LOG:Preloading resources...
156|admil.out.txt|2014-04-17 03:26:48.730000||PID:926 TID:926 TAG:W/Resources LOG:Preloaded drawable resource #0x1080096 (android:drawable/toast_frame) that varies with configuration!!
157|admil.out.txt|2014-04-17 03:26:48.740000||PID:926 TID:926 TAG:W/Resources LOG:Preloaded drawable resource #0x1080105 (android:drawable/btn_check_on_pressed_holo_light) that varies with configuration!!
You are using the date function wrong; there is not format parameter.
strftime could do such formatting, but that is not necessary because the timestamps are simply strings.
Just compare the strings directly:
select log_ID
from Message
where Timestamp between '2014-04-17 03:27:08'
and '2014-04-18 03:27:08';
When defining a relation, I want to update an attribute to the timestamp at insert. For example, a working table that I have right now
CREATE TABLE t1(
id INTEGER PRIMARY KEY AUTOINCREMENT,
time TIMESTAMP
DEFAULT CURRENT_TIMESTAMP,
txt TEXT);
This is updating a timestamp on insert, for example, insert into t1 (txt) values ('hello') adds the row 1|2012-07-19 08:07:20|hello|. However, I want to have this date formatted in unixepoch format.
I read the docs but this wasn't clear. For example, I modified the table relation to time TIMESTAMP DEFAULT DATETIME('now','unixepoch') but I get an error. Here, as in the docs, now was my time string and unixepoch was the modifier but it didn't work. Could someone help me how to format it as a unixepoch timestamp?
Use strftime:
sqlite> select strftime('%s', 'now');
1342685993
Use it in CREATE TABLE like this:
sqlite> create table t1 (
...> id integer primary key,
...> time timestamp default (strftime('%s', 'now')),
...> txt text);
sqlite> insert into t1 (txt) values ('foo');
sqlite> insert into t1 (txt) values ('bar');
sqlite> insert into t1 (txt) values ('baz');
sqlite> select * from t1;
1|1342686319|foo
2|1342686321|bar
3|1342686323|baz
See https://www.sqlite.org/lang_createtable.html#tablecoldef
If the default value of a column is an expression in parentheses, then the expression is evaluated once for each row inserted and the results used in the new row.
Note 'timestamp' is not a data type known to SQLite (see list here). The default value generated by strftime() would actually be stored as Text.
If it is important to store the value as a number instead of as a string, declare the field as an Integer and add a CAST() into the mix, like so:
create table t1(
...
ts_field integer(4) default (cast(strftime('%s','now') as int)),
...
);
Indeed strftime, which can also be used like so:
SELECT strftime('%s', timestamp) as timestamp FROM ... ;
Gives you:
1454521888
'timestamp' table column can be a text field even, using the current_timestamp as DEFAULT.
Without strftime:
SELECT timestamp FROM ... ;
Gives you:
2016-02-03 17:51:28
I am using asp.net2008 and MY SQL.
I want to auto-generate the value for the field username with the format as
"SISI001", "SISI002",
etc. in SQL whenever the new record is going to inserted.
How can i do it?
What can be the SQL query ?
Thanks.
Add a column with auto increment integer data type
Then get the maximum value of that column in the table using "Max()" function and assign the value to a integer variable (let the variable be 'x').
After that
string userid = "SISI";
x=x+1;
string count = new string('0',6-x.ToString().length);
userid=userid+count+x.ToString();
Use userid as your username
Hope It Helps. Good Luck.
PLAN A>
You need to keep a table (keys) that contains the last numeric ID generated for various entities. This case the entity is "user". So the table will contain two cols viz. entity varchar(100) and lastid int.
You can then have a function written that will receive the entity name and return the incremented ID. Use this ID concatenated with the string component "SISI" to be passed to MySQL for insertion to the database.
Following is the MySQL Table tblkeys:
CREATE TABLE `tblkeys` (
`entity` varchar(100) NOT NULL,
`lastid` int(11) NOT NULL,
PRIMARY KEY (`entity`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
The MySQL Function:
DELIMITER $$
CREATE FUNCTION `getkey`( ps_entity VARCHAR(100)) RETURNS INT(11)
BEGIN
DECLARE ll_lastid INT;
UPDATE tblkeys SET lastid = lastid+1 WHERE tblkeys.entity = ps_entity;
SELECT tblkeys.lastid INTO ll_lastid FROM tblkeys WHERE tblkeys.entity = ps_entity;
RETURN ll_lastid;
END$$
DELIMITER ;
The sample function call:
SELECT getkey('user')
Sample Insert command:
insert into users(username, password) values ('SISI'+getkey('user'), '$password')
Plan B>
This way the ID will be a bit larger but will not require any extra table. Use the following SQL to get a new unique ID:
SELECT ROUND(NOW() + 0)
You can pass it as part of the insert command and concatenate it with the string component of "SISI".
I am not an asp.net developer but i can help you
You can do something like this...
create a sequence in your mysql database as-
CREATE SEQUENCE "Database_name"."SEQUENCE1" MINVALUE 1 MAXVALUE 9999999999999999999999999999 INCREMENT BY 001 START WITH 21 CACHE 20 NOORDER NOCYCLE ;
and then while inserting use this query-----
insert into testing (userName) values(concat('SISI', sequence1.nextval))
may it help you in your doubt...
Try this:
CREATE TABLE Users (
IDs int NOT NULL IDENTITY (1, 1),
USERNAME AS 'SISI' + RIGHT('000000000' + CAST(IDs as varchar(10)), 4), --//getting uniqueness of IDs field
Address varchar(150)
)
(not tested)
Is there a way to create a table in sqlite3 that has a datetime column that defaults to 'now'?
The following statement returns a syntax error:
create table tbl1(id int primary key, dt datetime default datetime('now'));
Update: Here's the correct ddl courtesy of Sky Sanders:
create table tbl1(id int primary key, dt datetime default current_timestamp);
Try this:
create table tbl1(id int primary key, dt datetime default current_timestamp);
Background:
The DEFAULT constraint specifies a
default value to use when doing an
INSERT. The value may be NULL, a
string constant, a number, or a
constant expression enclosed in
parentheses. The default value may
also be one of the special
case-independant keywords
CURRENT_TIME, CURRENT_DATE or
CURRENT_TIMESTAMP. If the value is
NULL, a string constant or number, it
is inserted into the column whenever
an INSERT statement that does not
specify a value for the column is
executed. If the value is
CURRENT_TIME, CURRENT_DATE or
CURRENT_TIMESTAMP, then the current
UTC date and/or time is inserted into
the columns. For CURRENT_TIME, the
format is HH:MM:SS. For CURRENT_DATE,
YYYY-MM-DD. The format for
CURRENT_TIMESTAMP is "YYYY-MM-DD
HH:MM:SS".
From http://www.sqlite.org/lang_createtable.html
... default (datetime(current_timestamp))
The expression following default must be in parentheses. This form is useful if you want to perform date arithmetic using SQLite date and time functions or modifiers.
CURRENT_TIMESTAMP is a literal-value just like 'mystring'
column-constraint:
literal-value:
you can use the following query for using current date value in your table
create table tablename (date_field_name Created_on default CURRENT_DATE);