Formatted local time in a virtual generated column of SQLite - sqlite

I want to store date in milliseconds, but also I want to see the formatted representation of it.
In order to not waste the drive space it makes sense to use a virtual generated column for this.
I wrote it:
DROP TABLE IF EXISTS example;
CREATE TABLE IF NOT EXISTS example (
time INTEGER,
formatted_time GENERATED ALWAYS AS (strftime('%Y.%m.%d %H:%m', time/1000, 'unixepoch')) VIRTUAL
);
INSERT INTO example (time) VALUES (1605960000000);
INSERT INTO example (time) VALUES (1615413202000);
It works, but I can't set the second modifier of strftime(format,timestring,modifier,modifier...) to get the local time. (It returns UTC time by default.)
When I use:
formatted_time GENERATED ALWAYS AS (strftime('%Y.%m.%d %H:%m', time/1000, 'unixepoch', 'localtime')) VIRTUAL
It throws Result: non-deterministic use of strftime() in a generated column when I insert a data.
While it works as expected:
select strftime('%Y.%m.%d %H:%m', 1615413202000/1000, 'unixepoch', 'localtime');
How to create a virtual column with formatted local time?

It's not possible.
Based on this answers:
Creating generate column based on today's date in SQLite
Computed column 'Month' in table cannot be persisted because the column is non-deterministic
SQLite requires to the value of a generated column would be the same on any machine in any time zone (be deterministic). Even for a virtual generated column.
In my case for time value 1615413202000 the formatted_time would be different for the same table opened in different time zones, so the the table would be "non-deterministic".
As a workaround it possible to create a view:
CREATE VIEW example_view AS
SELECT time, strftime('%Y.%m.%d %H:%m', time/1000, 'unixepoch', 'localtime') as formatted_time_local
FROM example;
(based on Shawn's answer)

Related

Sqlite get the date part of a DateTime and return as a DateTime, not a string

In Sqlite I want to extract the date and time portions of a DateTime field separately in a view and return them also as a datetime, not strings. I've tried Cast, Date(), datetime(), but they all return strings.
I've read the SQLite documentation and understand how there is not an actual Date data type. Yet a Table field defined as DateTime is able to be parsed as a Date by an Excel query, but calculations on that field are not. I'm trying to do all data prep in the database view.
My data has the following field taken directly from the table definition:
LastModifiedDate datetime
I want the date (without time) to have the same DateTime data type as LastModifiedDate, not Text, because I use this view in many spreadsheets. I can apply Excel Date functions and formatting to LastModifiedDate field directly as returned from the ODBC query to Excel, and want to do the same to the Date-only part. I don't want to have to put a string-to-date conversion in every spreadsheet when I know it can get the date natively from Sqlite in LastModifiedDate.
SELECT LastModifiedDate,
date(LastModifiedDate) as Datepart,
cast(LastModifiedDate as numeric) as Date2
FROM Transactions
LastModifiedDate Datepart Date2
2019-07-28 18:22:38.9165394 2019-07-28 2019
LastModifiedDate in the above query is interpreted in Excel as a date to which date formats and date functions can be applied with no further processing required. Datepart above is returned as Text to Excel, and I can't apply date functions and formats without further pre-processing in Excel. I would like Datepart to be interpreted a date in Excel just as LastModifiedDate is.
I'm looking at the ch-werner.de sqliteodbc-0.9998. It will return an ODBC TIMESTAMP type only if the column decltype starts with timestamp or datetime. It returns ODBC TIME only for decltypes starting with time and ODBC DATE only for decltypes starting with date.
sqlite3 provides this decltype only for result table columns that are direct database column references. So if your SELECT statement has some expression that is more than a plain column reference, the decltype is lost. sqlite3 works like this at least up to version 3.39.0. It is documented.
The CAST expression converts the value of given expression to a storage classes by the determined affinity of the given declared type, but does not assign decltype to the result.
If you want to see the decltypes for query columns, you can use the sqlite3 cli and give it command .stats 2. Then it'll output the column declared types for each statement it executes.
If the decltype is found, the sqliteodbc-0.9998 will always parse string values into ODBC types. If DSN Option JDConv is enabled, it'll also parse floating point julianday values (whether provided as float or a string of a float) into ODBC types and when writing it'll write floating point into database.
If you can afford to change the schema, you can add a generated virtual column. This is cheap in storage, because data is not affected, but it costs when you query the column. This column can calculate other column into the values and decltypes you need for ODBC.
ALTER TABLE data ADD COLUMN
Datepart date AS (date(LastModifiedDate))
Then to get the Datepart, you simply query the column.
SELECT Datepart FROM data

Format date from lookup table

My report has a data lookup table with dates in dd/mm/yyyy formart. When I bring it into Crystal it changes the data type to date-time instead of just date.
I tried converting to just a date within Crystal, but when I run a lookup based on parameters (10-1-2016 - 10-31-2016) I get a running cycle of dates. As soon as it hits 10-31-2016 it starts over from 10-1-2016.
I tried setting it to not provider duplicate values to no avail. How I could be doing this better?
If you just need to display the date without the time value, you can format the field when it displays in your report as "System Default Short Format" in the Format Editor. (Date and Time tab)
Otherwise leave the date as-is and create a seperate Formula Date({table.Field}) to use as "just the date". This way you keep the original datetime value as-is, but can use your new Formula when the time value needs to be removed.

SQLite: select all rows made in a specific month

i want to get all entries from a SQLite table, which have the timestamp from the same month.
For example, the user can type in "July" and then i want to get all entries made in the 7. month.
The current "time"-column is a simple string and in the Format (DD.MM.YYYY HH:MM:SS)
Is there a way to do this with SQLite or will i need to use code in my program?
Assuming that your time strings have a fixed length, you could use a query like this:
SELECT * FROM MyTable WHERE time LIKE '__.07%';
However, you should always stored dates in one of the supported date/time formats so that you are able to use the built-int date/time functions.

How can I store the current timestamp in SQLite as ticks?

I have a SQLite database where I store the dates as ticks. I am not using the default ISO8601 format. Let's say I have a table defined as follows:
CREATE TABLE TestDate (LastModifiedTime DATETIME)
Using SQL, I wish to insert the current date and time. If I execute any of the below statements, I end up getting the date and time stored as a string and not in ticks.
INSERT INTO TestDate (LastModifiedTime) VALUES(CURRENT_TIMESTAMP)
INSERT INTO TestDate (LastModifiedTime) VALUES(DateTime('now'))
I have looked at the SQLite documenation, but I do not seem to find any option to obtain the current timestamp in ticks.
I can of course define a parameter in C# and store the value as a System.DateTime. This does result in the datetime getting stored to the database in ticks.
What I would like to do is be able to insert and update the current timestamp directly from within the SQL statement. How would I do this?
Edit:
The reason I want the data stored as ticks in the database, is that the dates are stored in the same format as stored by the ADO.Net data provider, and so that when the data is also queried using the ADO.Net provider it is correctly retrieved as a System.DataTime .Net type.
This particular oddity of SQLite caused me much anguish.
Easy way - store and retrieve as regular timestamp
create table TestDate (
LastModifiedTime datetime
);
insert into TestDate (LastModifiedTime) values (datetime('now'));
select datetime(LastModifiedTime), strftime('%s.%f', LastModifiedTime) from TestDate;
Output: 2011-05-10 21:34:46|1305063286.46.000
Painful way - store and retrieve as a UNIX timestamp
You can use strftime to retrieve the value in ticks. Additionally, to store a UNIX timestamp (roughly equivalent to ticks), you can can surround the number of seconds in single-quotes.
insert into TestDate (LastModifiedTime) values ('1305061354');
SQLite will store this internally as some other value that is not a UNIX timestamp. On retrieval, you need to explicitly tell SQLite to retrieve it as a UNIX timestamp.
select datetime(LastModifiedTime, 'unixepoch') FROM TestDate;
To store the current date and time, use strftime('%s', 'now').
insert into TestDate (LastModifiedTime) VALUES (strftime('%s', 'now'));
Full example:
create table TestDate (
LastModifiedTime datetime
);
insert into TestDate (LastModifiedTime) values (strftime('%s', 'now'));
select datetime(LastModifiedTime, 'unixepoch') from TestDate;
When executed by sqlite3, this script with print:
2011-05-10 21:02:34 (or your current time)
After further study of the SQLite documentation and other information found on date number conversions, I have come up with the following formula, which appears to produce correct results:
INSERT INTO TestDate(LastModifiedTime)
VALUES(CAST((((JulianDay('now', 'localtime') - 2440587.5)*86400.0) + 62135596800) * 10000000 AS BIGINT))
Seems like a painful way to produce something that I would expect to be available as a built-in datetime format, especially that the database supports the storing of datetime values in ticks. Hopefully, this becomes useful for others too.
Update:
The above formula is not perfect when it comes to daylight savings. See section Caveats And Bugs in SQLite docs regarding local time calculation.
The following will return the number of milliseconds since the UNIX Epoch:
SELECT (strftime('%s', 'now') - strftime('%S', 'now') + strftime('%f', 'now')) * 1000 AS ticks
It works by grabbing the number of seconds since the Unix Epoch (%s), subtracting the number of seconds in the current time (%S), adding the number of seconds with decimal places (%f), and multiplying the result by 1000 to convert from seconds to milliseconds.
The subtraction and addition are to add precision to the value without skewing the result. As stated in the SQLite Documentation, all uses of 'now' within the same step will return the same value.

How do I pull values between two datetimes at specific interval in MySQL?

I have an application that writes temperature values to a MySQL table every second, It consists of the temperature and a datetime field.
I need to pull these values out of the table at specific intervals, every second, minute, hour etc.
So for example I will need to pull out values between 2 datetime fields and show the temperature at the hour for each of them.
One option I've considered is to create a temporary table that holds a list of the time intervals generated using MySQL INTERVAL and then joining that against the main table.
I'm just wondering if there's some time and date functions in MySQL that I'm overlooking that would make this easier?
Thanks.
You could use between for your date, and then a conditional WHERE clause using time() that looks at the structure of the timestamp. If it has 00:00 (for instance, 16:00:00) within it, take it, if not, leave it.
Example (untested):
SELECT temp, date
FROM temperatures
WHERE (date BETWEEN '2009/01/03 12:00:00' AND '2009/01/04 12:00:00')
AND (time(date) LIKE '%:00:00')
ORDER BY date ASC
LIMIT 10

Resources