I have 3 tables I would like to work on using the date, however one of the tables includes the date in unix epoch format. Here is an example of the 3 fields:
Table1: 2017-02-01T07:58:40.756031Z
Table2: 2017-02-07T10:16:46Z
Table3: 1489236559
I would like to convert the date in table 3 to match the format of table 2 as closely as possible. This is what I have right now:
SELECT cast(from_unixtime(tstart) as timestamp) as ISODATE from scada_logs
This gives me something tantalizingly close, but not quite there. Here is what I get:
ISODATE
2017-03-20 14:31:06.000
2017-03-20 14:31:06.000
I've played around with this for a couple of hours but I'm not getting any closer. Does anyone have any ideas?
Thank you!
Option 1: date_format
presto> select date_format(from_unixtime(1489236559),'%Y-%m-%dT%H:%i:%sZ');
_col0
----------------------
2017-03-11T12:49:19Z
Option 2: to_iso8601
presto> select to_iso8601(from_unixtime(1489236559));
_col0
--------------------------
2017-03-11T12:49:19.000Z
Related
I'm trying to convert the snowflake timestamp that is compatible with Iterable date format
My attempt:
I have a table with the following timestamps:
|------------------------------|
| table_timestamp |
|------------------------------|
| "2021-07--19 02:45:91.000 Z" |
|------------------------------|
I tried using TIMESTAMP_TZ and I got the following resut
SELECT table_timestamp::TIMESTAMP_TZ
from my_table
gave me output 2021-07--19 02:45:91.000 +0000
How can I get the above timestamp in this format 2021-07--19 02:45:91 +00:00?
You can cast to a varchar and give, as the second parameter, the format that you want:
SELECT TO_VARCHAR('2021-07-19 02:45:31.000'::Timestamp_TZ, 'yyyy-mm-dd hh:mi:ss')
2021-07-19 02:45:31
(Note I changed the seconds to 31 as there isn't 91 seconds in a minute and also changed your double dash between month and day to a single. I'm assuming those were typos)
Give you are correct converting the date/timestamp, this is not a convertion problem, it is a presentation problem.
Thus the Date and Time output formatting help is what you need.
I wish to convert following value which is a chararray in pig
2016-05-11 23:59:57.628197
to
2016-05-11T23:59:57.628-05:00
How can I do it ?
Following is what I tried considering alias 'a2' contains list of datetime values in chararray in the column named 'input_date_value'
FOREACH a2 GENERATE input_date_value AS input_date:chararray,
ToDate(input_date_value,'YYYY-MM-DD HH:mm:ss.SSSSSS') AS modification_datetime:datetime;
For input -
2002-07-11 16:58:40.249764
Output is -
2002-01-11T16:58:40.249-05:00
The month values like '07' are not getting picked up,
The created timestamp has month set to 01' i.e. January everytime for all dates.
Can someone help. What am I doing wrong ?
https://pig.apache.org/docs/r0.11.1/func.html#to-date ToDate takes SimpleDateFormat only supports milliseconds http://docs.oracle.com/javase/6/docs/api/java/text/SimpleDateFormat.html
The -05:00 you see is the time zone ToDate is actually truncating to 3 digits as it supports only milliseconds
Use lowercase character d instead of uppercase D for parsing date values.
Now, I have managed to fix it myself on (In Pig 0.11)
Apparently Pig 0.11 does not support the date format components I used earlier for parsing the month and date.
I found below inference which hints on the incompatibility as mentioned https://www.w3.org/TR/NOTE-datetime
Use:
'YYYY-MM-dd HH:mm:ss.SSSSSS'
instead of 'YYYY-MM-DD HH:mm:ss.SSSSSS'
It now gives correct output.
Input:
2001-11-28 16:04:49.22388
Output:
2001-11-28T16:04:49.223-05:00
I've just started using SQLite, instead of SQL Server, and it doesn't seem to want to do ORDER BY, MAX() or MIN() on dates.
The Survey_Date column is a text field, so ordering it sorts it from January to December, instead of by the year. If I include date(Survey_Date) in my SELECT statement, it will sort the data by year -- but I can't override the DESC sort, and it doesn't actually display the date.
SELECT Survey_Date FROM Surveys WHERE Loc_ID = 32 ORDER BY Survey_Date;
yields results like:
01/10/2009
01/20/2013
02/05/2010
...
SELECT date(Survey_Date), Survey_Date FROM Surveys WHERE Loc_ID = 32 ORDER BY 1 ASC;
yields results like:
|01/20/2013
|02/05/2010
|01/10/2009
...
It's clearly sorting on the date correctly now, but it doesn't display the formatted date, and doesn't recognize the ASC command.
Can anyone explain what it's doing?
As already explained at least million times over this site, SQLite does not have date/timestamp type.
So if you want to sort by date, you must store them in format that sorts correctly either numerically, or lexicographically (asciibetically/unicodebetically).
The recommended format is ISO-8601, that is yyyy-mm-dd (yyyy-mm-ddTHH:MM:SS if you want a time too). This is what SQLite has some utility functions to work with, too.
Other possible formats are numeric, either number of seconds since some specific point, e.g. Unix time or number of days, e.g. (Modified) Julian day.
01/10/2009
01/20/2013
02/05/2010
They are strings, so this is correctly sorted.
SELECT date(Survey_Date), Survey_Date FROM Surveys WHERE Loc_ID = 32 ORDER BY 1 ASC;
The date function expects ISO-8601 format. It does not understand what you are giving it and returns NULL. Fully expected.
Sorting by all NULL values effectively does nothing. The rows came out sorted by accident.
I have a column of data type TEXT:
date
----
DD/MM/YYYY
but I want to convert all rows in the column to:
date
----
YYYY-MM-DD 00:00:00
(Yes, 00:00:00 for all rows)
Is there any way to do it in SQLite?
Use strftime.
strftime('%Y-%m-%d %H:%M:%S', date_str);
EDIT:
Yes, my first quess do not work. This one does, though:
SELECT
date,
substr(date,7,4)||'-'||substr(date,4,2)||'-'||substr(date,1,2)||' 00:00:00' as text_repr,
datetime(substr(date,7,4)||'-'||substr(date,4,2)||'-'||substr(date,1,2)||' 00:00:00') as datetime_repr
FROM
t
Simply put - You have to parse it on Your own, as stated here or here...
I'm trying to re-partition some table using week number counting from some day:
my_fact table contains a field called time_stamp of type TIMESTAMPTZ
Unfortunately, re-partition doesn't work, and I'm getting the error:
MyDB=> ALTER TABLE my_fact PARTITION BY MOD(TIMESTAMPDIFF('day', time_stamp::TIMESTAMP, TIMESTAMP '2013-09-23'), 156) REORGANIZE;
NOTICE 4954: The new partitioning scheme will produce 12 partitions
ROLLBACK 2552: Cannot use meta function or non-deterministic function in PARTITION BY expression
Should the cast of time_stamp to TIMESTAMP strip any time zone related info from this field thus making it deterministic?
Thanks!
Take a look at the date_part() function, you can use the TIMESTAMPTZ as its source column:
Example :
**The number of the week of the calendar year that the day is in.**
SELECT EXTRACT(WEEK FROM TIMESTAMP '2001-02-16 20:38:40');
Result: 7
SELECT EXTRACT(WEEK FROM DATE '2001-02-16');
Result: 7
Since I got no answer, I'm writing here what I've ended up with:
ALTER TABLE my_fact PARTITION BY
MOD(
TIMESTAMPDIFF(
'day',
'2013-09-23'::timestamptz AT TIME ZONE 'UTC',
time_stamp AT TIME ZONE 'UTC'),
156)
REORGANIZE;
This solution works.