Reformat dates in column - sqlite

I have some data in an SQLite DB of the form:
id column1 date
111 280 1/1/2014
114 275 1/2/2014
The date field is of type TEXT. I've been made aware (https://www.sqlite.org/lang_datefunc.html) that I should have the dates formatted like YYYY-MM-DD to take advantage of SQLite's datetime functionality. Is there a query I could run to change the format from
mm/dd/yyyy
to
YYYY-MM-DD
in place?

Your current date format has four possible forms:
m/d/yyyy
m/dd/yyyy
mm/d/yyyy
mm/dd/yyyy
To rearrange the fields, extract them with substr() and then combine them again.
It might be possible to determine the positions of the slashes with instr(), but for a one-off conversion, just using four queries is simpler:
UPDATE MyTable
SET date = substr(date, 6, 4) || '-' ||
substr(date, 1, 2) || '-' || '0' ||
substr(date, 4, 1)
WHERE date LIKE '__/_/____';
-- this is mm/d/yyyy; similarly for the other forms, modify positions and zeros

Without any frills such as exception handling!
This approach is slightly simpler because strptime doesn't mind about presence or absence of leading zeroes in days and months.
>>> from datetime import datetime
>>> import sqlite3
>>> con = sqlite3.connect(':memory:')
>>> cur = con.cursor()
>>> cur.execute('CREATE TABLE EXAMPLE (date_column text)')
<sqlite3.Cursor object at 0x00000000038D07A0>
>>> cur.execute('INSERT INTO EXAMPLE VALUES ("1/1/2014")')
<sqlite3.Cursor object at 0x00000000038D07A0>
>>> def transformDate(aDate):
... tempDate = datetime.strptime(aDate, '%d/%m/%Y')
... return tempDate.strftime('%Y-%m-%d')
...
>>> transformDate('1/1/2014')
'2014-01-01'
>>> con.create_function('transformDate', 1, transformDate)
>>> cur.execute('UPDATE EXAMPLE SET date_column = transformDate(date_column)')
<sqlite3.Cursor object at 0x00000000038D07A0>

Related

How to create data table using dynamic dates functions like now() or ago()?

Is there a way to create a data table using dynamic dates function like now() or ago()? I am not sure it's possible or just an issue of finding the right syntax.
Works
let Logs = datatable(timestamp: datetime) [
datetime(2022-12-02 2:00:00.00),
datetime(2022-12-02 6:00:00.00),
];
Does not work
let Logs = datatable(timestamp: datetime) [
now(),
datetime(now()- ago(3h)),
];
Thoughts?
datatable accept only literals (no expressions, no functions).
You can use the union operator with multiple print statements:
union
(print 1, now())
,(print 2, ago(3h))
| project-rename id = print_0, timestamp = print_1
id
timestamp
2
2023-02-15T05:29:23.2203689Z
1
2023-02-15T08:29:23.2203689Z
Fiddle
P.S.
datetime(now()- ago(3h)) is wrong.
datetime accept only literal.
now() - ago(3h) is equal to 3h.
Use ago(3h) or now() - 3h

SQlite REPLACE issue

I have a date in the format 'MM/DD/YYYY'. I need the month, day, and year to be separated by : or - for functions like strftime to work in sqlite. I'm attempting to use REPLACE but it won't work on '/'.
Any help is appreciated.
UPDATE table SET date = REPLACE(date,'/','-')
Even if you replace / with -, the date functions of SQLite will not work with your dates.
The only valid date format for SQLite is YYYY-MM-DD.
Update the column date to that format like this:
UPDATE tablename
SET date = SUBSTR(date, -4) || '-' || SUBSTR(date, 1, 2) || '-' || SUBSTR(date, 4, 2);
See the demo.

Hive: Convert string datetime with missing seconds in "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"

I'm using the following code to convert a string datetime variable to datetime, but the converted string is missing SSS part.
Code used:
cast(FROM_UNIXTIME(UNIX_TIMESTAMP(oldtime, "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"),"yyyy-MM-dd HH:mm:ss.SSS") as timestamp) as newtime
The outcome:
2019-03-08T18:28:36.901Z is converted to 08MAR2019:18:28:36.000000
Some other oldtimes in string:
2020-03-09T16:05:06:827Z
2020-03-09T16:03:19:354Z
2020-03-11T16:03:57:280Z
2020-03-10T16:02:57:642Z
2020-03-10T16:04:07:455Z
2020-03-10T16:04:09:737Z
2020-03-10T16:03:57:280Z
2020-03-10T16:02:46:816Z
The SSS part '901' is missing in the converted time. Would like help on keeping the SSS part since I need to sort the records by their exact time.
Thank you!
from_unixtime is always until minutes(yyyy-MM-dd HH:mm:ss) to get millisecs we need to do some workarounds.
we will extract the millisecs from the old_time using regexp_extract then concat that to from_unixtime result and finally cast to timestamp.
Example:
select old_time,
timestamp(concat_ws(".", --concat_ws with . and cast
FROM_UNIXTIME(UNIX_TIMESTAMP(old_time, "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"),"yyyy-MM-dd HH:mm:ss"), -- from_unixtime and unix_timestamp to convert without millisecs
regexp_extract(string(old_time),".+\\.(.*)(?i)z",1))) as newtime from --regexp_extract to extract last 3 digits before z then concat
(select string("2020-03-11T21:14:41.335Z")old_time)e
+------------------------+-----------------------+
|old_time |newtime |
+------------------------+-----------------------+
|2020-03-11T21:14:41.335Z|2020-03-11 21:14:41.335|
+------------------------+-----------------------+
UPDATE:
Your sample data have : before milliseconds, Try with below query:
select old_time,
timestamp(concat_ws(".", --concat_ws with . and cast
FROM_UNIXTIME(UNIX_TIMESTAMP(old_time, "yyyy-MM-dd'T'HH:mm:ss:SSS'Z'"),"yyyy-MM-dd HH:mm:ss"), -- from_unixtime and unix_timestamp to convert without millisecs
regexp_extract(string(old_time),".+\\:(.*)(?i)z",1))) as newtime from --regexp_extract to extract last 3 digits before z then concat
(select string("2020-03-11T21:14:41:335Z")old_time)e
Simply replace 'T' with space ' ' remove 'Z' and replace last ':' with dot, like this :
select regexp_replace('2020-03-09T16:05:06:827Z','(.*?)T(.*?):([^:]*?)Z$','$1 $2\\.$3');
Result:
2020-03-09 16:05:06.827
Read also this answer if you need to convert to different format, preserving milliseconds: https://stackoverflow.com/a/59645846/2700344

Retrive date wise data from SQLITE database kotlin

I have one SQlite database where i store some data with date.
Now i want to get data date wise like:
Month wise - it means i pass the value like Current date is EndDate to this month 1st date.
Year wise - it means i want to get data 1st-april-20..previews to 31-march-20..current
Start and End date wise - hear is which i pass date.
For that i got one solution is HERE for java.
But i have no idea this HOW TO WORK. Anyone please explain me how to work this and How to i get data as i mention above. FOR KOTLIN
TABLE
db.createTable(customerDetail, true,
credit_id to INTEGER + PRIMARY_KEY + AUTOINCREMENT,
credit_type to TEXT,
c_id to TEXT,
credit_amount to TEXT,
credit_date to TEXT,
addFrom to TEXT
)
UPDATE
For Month wise data i'll try below query like:
"SELECT * FROM $customerDetail WHERE $credit_date BETWEEN strftime('%d-%m-%Y', '$startDate') AND strftime('%d-%m-%Y', '$currentDate')"
/*SELECT * FROM CustomerDetail WHERE credit_date BETWEEN strftime('%d-%m-%Y', '01/02/2019') AND strftime('%d-%m-%Y', '23/02/2019')*/
But it's give me arralistSize = 0.
After that i can write new query like:
"SELECT * FROM $customerDetail WHERE $credit_date BETWEEN '$startDate' AND '$currentDate'"
/*SELECT * FROM CustomerDetail WHERE credit_date BETWEEN '01/02/2019' AND '23/02/2019'*/
In this query data will return. But it's return all data without any filtering.
If anyone knows why this work like this please help me.
MY DATA LIST
Thanks in advance.
Solution :
Solution for MONTH wise
I just change date format "dd/mm/yyyy" TO "yyyy/mm/dd" and re insert all data.
AND Fire below QUERY :
"SELECT * FROM $customerDetail WHERE $credit_date BETWEEN '$startDate' AND '$currentDate'"
SQLite does not have a Date data type like other RDBMS's. It treats dates as TEXT.
So when it compares the credit_date column it actually does compare strings.
This would be fine if you stored your dates in the format YYYY-MM-DD and compared against the same format.
But if you store your dates in the format DD/MM/YYYY then you can't compare.
So the best solution would be to change the format of the column credit_date to YYYY-MM-DD.
If this is not possible then you have to transform the string value of the column like this:
substr(credit_date, 7, 4) || '-' || substr(credit_date, 4, 2) || '-' || substr(credit_date, 1, 2)
Now you can use this in a query like:
SELECT * FROM $customerDetail
WHERE substr($credit_date, 7, 4) || '-' || substr($credit_date, 4, 2) || '-' || substr($credit_date, 1, 2)
BETWEEN '$startDate' AND '$currentDate'
But you have to make sure that $startDate and $currentDate are also in the format YYYY-MM-DD

How to change date format in sqlite database

How do I change the format of a date in %d/%m/%Y or any other from this string 10/09/2016 in sqlite?
I have tried this but it is not working:
select strftime('%d/%m/%Y', '09-10-2016'),t_date from party_trans
try this query
select strftime('%d/%m/%Y',datetime(substr(t_date, 7, 4) || '-' || substr(t_date, 4, 2) || '-' || substr(t_date, 1, 2))) from party_trans;
I would suggest to store date as time stamp instead of string. Timestamp gives you more flexibility and accuracy than date. Convert your date into millisecond (timestamp) format and store it into your database as INTEGER type.
Or try this:
select strftime('%d/%m/%Y', '2016-10-09');
Store date into INTEGER type in database.
select strftime('%d/%m/%Y', date) from table

Resources