How to proper cast in sqllite - sqlite

When i cast datetime in SQLLite, it truncates the string.
for example
select cast("2017-04-23 9:12:08 PM" as datetime) as dt
returns
2017

SQLite's CAST can only cast to the defined storage classes and can therefore only be used to cast to
NONE (blob), TEXT, REAL, INTEGER or NUMERIC.
However the normal rules for determing column-affinity are applied to the type so by coding CAST(value AS datetime) you are effectively using CAST(value AS NONE) (i.e. a BLOB).
CAST expressions
Therefore you can't effectively use CAST. However you simply use the DateTime functions against an appropriate value (accepted formats) as per Date And Time Functions e.g. :-
SELECT datetime("2017-04-23 09:12:08") as dt;
results in
2017-04-23 09:12:08
or to show date manipulation
select date(dt), dt FROM (
select datetime("2017-04-23 09:12:08") as dt
);
results in
2017-04-23
and
2017-04-23 09:12:08
However considering that your format isn't one of the accepted formats you could convert the value. This is more complex but it can be done. Here's an example that will perform the conversion (not substantially tested though) :-
SELECT
CASE WHEN (CAST(hour AS INTEGER) + CAST(adjustment AS INTEGER)) > 9 THEN
datepart||' '||CAST(CAST(hour AS INTEGER) + CAST(adjustment AS INTEGER) AS TEXT)||':'||mins_and_secs
ELSE
datepart||' 0'||CAST(CAST(hour AS INTEGER) + CAST(adjustment AS INTEGER) AS TEXT)||':'||mins_and_secs
END AS converted
FROM (
SELECT substr(ts,1,10) as datepart,
CASE WHEN instr(ts,"PM") THEN 12 ELSE 0 END AS adjustment,
CASE WHEN length(ts) = 21 THEN substr(ts,12,1) ELSE substr(ts,12,2) END AS hour,
CASE WHEN length(ts) = 21 THEN substr(ts,14,5) ELSE substr(ts,15,5) END AS mins_and_secs
FROM (
select("2017-04-23 9:12:08 PM") as ts
)
);
This would result in 2017-04-23 21:12:08.
Using select("2017-04-23 9:12:08 AM") results in 2017-04-23 09:12:08
Using select("2017-04-23 11:12:08 PM") results in 2017-04-23 23:12:08
Using select("2017-04-23 11:12:08 AM") results in 2017-04-23 11:12:08

The closest I could come up with is:
select date(datetime(strftime('%s','2017-04-23 09:12:08'), 'unixepoch'))
Result:
2017-04-23
The dateformat you have is not recognised by SQLite:
"2017-04-23 9:12:08 PM"
It does not conform to the Time string formats recognised:
A time string can be in any of the following formats:
YYYY-MM-DD
YYYY-MM-DD HH:MM
YYYY-MM-DD HH:MM:SS
YYYY-MM-DD HH:MM:SS.SSS
YYYY-MM-DDTHH:MM
YYYY-MM-DDTHH:MM:SS
YYYY-MM-DDTHH:MM:SS.SSS
HH:MM
HH:MM:SS
HH:MM:SS.SSS
now
DDDDDDDDDD
Date And Time Functions

Related

Hive: Convert string datetime with missing seconds in "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"

I'm using the following code to convert a string datetime variable to datetime, but the converted string is missing SSS part.
Code used:
cast(FROM_UNIXTIME(UNIX_TIMESTAMP(oldtime, "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"),"yyyy-MM-dd HH:mm:ss.SSS") as timestamp) as newtime
The outcome:
2019-03-08T18:28:36.901Z is converted to 08MAR2019:18:28:36.000000
Some other oldtimes in string:
2020-03-09T16:05:06:827Z
2020-03-09T16:03:19:354Z
2020-03-11T16:03:57:280Z
2020-03-10T16:02:57:642Z
2020-03-10T16:04:07:455Z
2020-03-10T16:04:09:737Z
2020-03-10T16:03:57:280Z
2020-03-10T16:02:46:816Z
The SSS part '901' is missing in the converted time. Would like help on keeping the SSS part since I need to sort the records by their exact time.
Thank you!
from_unixtime is always until minutes(yyyy-MM-dd HH:mm:ss) to get millisecs we need to do some workarounds.
we will extract the millisecs from the old_time using regexp_extract then concat that to from_unixtime result and finally cast to timestamp.
Example:
select old_time,
timestamp(concat_ws(".", --concat_ws with . and cast
FROM_UNIXTIME(UNIX_TIMESTAMP(old_time, "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"),"yyyy-MM-dd HH:mm:ss"), -- from_unixtime and unix_timestamp to convert without millisecs
regexp_extract(string(old_time),".+\\.(.*)(?i)z",1))) as newtime from --regexp_extract to extract last 3 digits before z then concat
(select string("2020-03-11T21:14:41.335Z")old_time)e
+------------------------+-----------------------+
|old_time |newtime |
+------------------------+-----------------------+
|2020-03-11T21:14:41.335Z|2020-03-11 21:14:41.335|
+------------------------+-----------------------+
UPDATE:
Your sample data have : before milliseconds, Try with below query:
select old_time,
timestamp(concat_ws(".", --concat_ws with . and cast
FROM_UNIXTIME(UNIX_TIMESTAMP(old_time, "yyyy-MM-dd'T'HH:mm:ss:SSS'Z'"),"yyyy-MM-dd HH:mm:ss"), -- from_unixtime and unix_timestamp to convert without millisecs
regexp_extract(string(old_time),".+\\:(.*)(?i)z",1))) as newtime from --regexp_extract to extract last 3 digits before z then concat
(select string("2020-03-11T21:14:41:335Z")old_time)e
Simply replace 'T' with space ' ' remove 'Z' and replace last ':' with dot, like this :
select regexp_replace('2020-03-09T16:05:06:827Z','(.*?)T(.*?):([^:]*?)Z$','$1 $2\\.$3');
Result:
2020-03-09 16:05:06.827
Read also this answer if you need to convert to different format, preserving milliseconds: https://stackoverflow.com/a/59645846/2700344

Convert string to date with date and time together in Hive

I need convert string to datetime (date and time together).
I try this:
cast(to_date(from_unixtime(unix_timestamp('20190303164305', 'yyyyMMddHHmmss'))) as date) as date_data_chamada
timezone: Brazil
But this way returns just date, like this: 2019-03-03, and I need: 2019-03-03 16:43:05
Thanks!
Full code:
INSERT INTO p_b.este PARTITION (dt_originacao_fcdr)
SELECT
tp_registro_fcdr,
seq_registro_fcdr,
tp_cdr_fcdr,
dt_atendimento_fcdr,
data_atendimento_completa_fcdr,
cast(from_unixtime(unix_timestamp(data_atendimento_completa_fcdr, 'yyyyMMddHHmmss'),"yyyy-MM-dd HH:mm:ss")as timestamp) as date_data_atendimento_fcdr,
hr_atendimento_fcdr,
duracao_atend_fcdr,
hr_originacao_fcdr,
duracao_total_fcdr,
duracao_chamada_tarifada_fcdr,
st_chamada_fcdr,
fim_sel_orig_fcdr
FROM p_b.norm;
Remove date casting and to_date functions as you are expecting timestamp!
Example:
hive> select from_unixtime(unix_timestamp('20190303164305', 'yyyyMMddHHmmss'),"yyyy-MM-dd HH:mm:ss") as date_data_chamada;
RESULT:
2019-03-03 16:43:05
If you use to_date or cast('string' as date) then hive results only date(yyyy-MM-dd)!
Ex:
hive> select to_date(from_unixtime(unix_timestamp('20190303164305', 'yyyyMMddHHmmss'),"yyyy-MM-dd HH:mm:ss")) as date_data_chamada;
--2019-03-03
Pass the second argument format string to from_unixtime. Note that the returned type is string.
from_unixtime(unix_timestamp('20190303164305','yyyyMMddHHmmss'),'yyyy-MM-dd HH:mm:ss')

Select data based on month and year

We storing date as String in column created_at by below format 2019-10-09T15:29:28.000+08:00 in Moor.
We would like to write a select query, to retrieve data where month are October and year is 2019.
Future<ABC> selectReadingBasedOnMonth(
int month, int year) {
return (select(abcs)
..where((t) {
final sqliteDate = FunctionCallExpression<DateTime, DateTimeType>(
'date', [t.createdAt]);
return sqliteDate.year.equals(year) &
sqliteDate.month.equals(month);
}))
.getSingle();
}
But we are not getting any data. This is the query displayed in log
I/flutter (12004): Moor: Sent SELECT * FROM abcs WHERE
(CAST(strftime("%Y", date(created_at), "unixepoch") AS INTEGER)) = ?
AND (CAST(strftime("%m", date(created_at), "unixepoch") AS INTEGER)) =
?; with args [2019, 10]
The unixepoch modifier can only be used with date/time strings that are solely digits.
The "unixepoch" modifier (11) only works if it immediately follows a
timestring in the DDDDDDDDDD format.
This modifier causes the
DDDDDDDDDD to be interpreted not as a Julian day number as it normally
would be, but as Unix Time - the number of seconds since 1970.
If the
"unixepoch" modifier does not follow a timestring of the form
DDDDDDDDDD which expresses the number of seconds since 1970 or if
other modifiers separate the "unixepoch" modifier from prior
DDDDDDDDDD then the behavior is undefined.
For SQLite versions before
3.16.0 (2017-01-02), the "unixepoch" modifier only works for dates between 0000-01-01 00:00:00 and 5352-11-01 10:52:47 (unix times of
-62167219200 through 106751991167).
Date And Time Functions
For example consider the following (based upon your query) :-
DROP TABLE IF EXISTS abcs;
CREATE TABLE IF NOT EXISTS abcs (created_at TEXT);
INSERT INTO abcs VALUES ('2019-10-09T15:29:28.000+08:00');
SELECT *,
CAST(strftime('%Y', date(created_at)/*, 'unixepoch'*/) AS INTEGER) AS year_nounixepoch,
CAST(strftime('%m', date(created_at)/*, 'unixepoch'*/) AS INTEGER) AS month_nounixepoch,
CAST(strftime('%Y', date(created_at), 'unixepoch') AS INTEGER) AS year_invalid,
CAST(strftime('%m', date(created_at), 'unixepoch') AS INTEGER) AS month_invalid,
CAST(strftime('%Y', strftime('%s',date(created_at)), 'unixepoch') AS INTEGER) AS year_unixepoch,
CAST(strftime('%m', strftime('%s',date(created_at)), 'unixepoch') AS INTEGER) AS month_unixepoch
FROM abcs
WHERE CAST(strftime('%Y', strftime('%s',date(created_at)), 'unixepoch') AS INTEGER) = 2019 AND CAST(strftime('%m', strftime('%s',date(created_at)), 'unixepoch') AS INTEGER) = 10;
DROP TABLE IF EXISTS abcs; /* cleanup test environment */
Note that single quotes have been used to replace double quotes, which would be the correct SQL although this discrepancy may be due to how the message is output.
This results in :-
i.e. where the unixepoch modifier has been used it results in null as the date/time is not solely digits.
the selection criteria, with unixepoch via strftime('%s',..... works as expected.
Thanks for the answer provided by simolus3
final asDate = FunctionCallExpression('date', [t.createdAt]);
final year = FunctionCallExpression<String, StringType>(
'strftime', [const Constant<String, StringType>('%Y'), asDate]);
final month = FunctionCallExpression<String, StringType>(
'strftime', [const Constant<String, StringType>('%m'), asDate]);
return year.equals('2019') & month.equals('07');

Join on time part is failing

I am facing an issue where i am not able to join two time columns:
The reported date is timestamp 0 with values like
4/1/2017 19:58:00
8/19/2017 19:58:00
Fault_Order_Submitted_Dt DATE FORMAT 'YYYY-MM-DD',
Fault_Order_Submitted_Tm INTEGER FORMAT '99:99:99',
sel a.completion_date, b.Fault_Order_Completed_Dt||b.fault_order_submitted_tm as Fault_time from
DG.LL_FMFTMX a
inner join DG.fault_order b
on a.fault_number=b.Fault_Order_Num
and cast(a.reported_date as date) =b.fault_order_submitted_dt
and cast(cast( a.reported_date as time(0) ) as integer format '99:99:99') = b.fault_order_submitted_tm
where fault_status='P'
The Join part on time is failing with invalid operation on date/time.
Please suggest.
To get the time portion as HHMISS integer you need to cast it to a string first:
Cast(To_Char(reported_date, 'hhmiss') AS INT)

Converting a datetime string (3/24/2017 10:00:00 PM) to (3-24-2017 22:00:00) hive i.e convert from 12 hour to 24 hour format

I have a datetime field in a hive table which is of data type string.
It looks as below:
datetime 3/24/2017 10:00:00 PM
Tried to convert it to the right format desired by hive and also tried removing the AM/PM to a 24 hour format but to no avail.
select from_unixtime(unix_timestamp(datetime,'mm-dd-yyyy HH:MM:SS')) from test_table
You can achieve this using below command:
select from_unixtime(unix_timestamp(datetime,'MM/dd/yyyy hh:mm:ss aa'),'MM-dd-yyyy HH:mm:ss') from test_table;
The format is MM-dd-yyyy HH:mm:ss aa
select from_unixtime(unix_timestamp(datetime,'MM-dd-yyyy HH:mm:ss aa')) from test_table;
With 'MM/dd/yyyy hh:mm:ss aa' does not work. You can archive this with:
FROM_UNIXTIME
(
(
CASE WHEN
datetime LIKE '%AM%'
THEN
UNIX_TIMESTAMP(datetime,'MM/dd/yyyy HH:mm:ss aa')
ELSE
(UNIX_TIMESTAMP(datetime,'MM/dd/yyyy HH:mm:ssaa') + (12 * 3600))
END
)
,'MM-dd-yyyy HH:mm:ss'
)

Resources