I'm trying to re-partition some table using week number counting from some day:
my_fact table contains a field called time_stamp of type TIMESTAMPTZ
Unfortunately, re-partition doesn't work, and I'm getting the error:
MyDB=> ALTER TABLE my_fact PARTITION BY MOD(TIMESTAMPDIFF('day', time_stamp::TIMESTAMP, TIMESTAMP '2013-09-23'), 156) REORGANIZE;
NOTICE 4954: The new partitioning scheme will produce 12 partitions
ROLLBACK 2552: Cannot use meta function or non-deterministic function in PARTITION BY expression
Should the cast of time_stamp to TIMESTAMP strip any time zone related info from this field thus making it deterministic?
Thanks!
Take a look at the date_part() function, you can use the TIMESTAMPTZ as its source column:
Example :
**The number of the week of the calendar year that the day is in.**
SELECT EXTRACT(WEEK FROM TIMESTAMP '2001-02-16 20:38:40');
Result: 7
SELECT EXTRACT(WEEK FROM DATE '2001-02-16');
Result: 7
Since I got no answer, I'm writing here what I've ended up with:
ALTER TABLE my_fact PARTITION BY
MOD(
TIMESTAMPDIFF(
'day',
'2013-09-23'::timestamptz AT TIME ZONE 'UTC',
time_stamp AT TIME ZONE 'UTC'),
156)
REORGANIZE;
This solution works.
Related
I am working locally with an sqllite DB. I have imported some records from teradata where there was a date field in the format of 'YYYY-MM-DD'. When i imported the records the date switched from a date to a number. I know this is a feature of sqllite and that one can access it via date(sqllite_date) when selecting it in a where clause.
My problem is that the dates now appear to be a bit odd. For example the year appears to be negative.
Is there anyway to recover this to the correct format?
Below is an example of converting a number in the database into a date
SELECT date(18386)
# -4662-03-28
SELECT datetime('now')
# 2021-02-11 10:41:52
SELECT date(sqllite_date) FROM mydb
# Returns -4662-03-28
# Should return 2020-05-04
I am very new to this area so apologies if this is a basic question. Thank you very much for your time
In SQLite you can store dates as TEXT, REAL or INTEGER.
It seems that you stored the dates in a column with INTEGER or REAL affinity.
In this case, if you use the function date(), it considers a value like 18386 as a Julian day, meaning the number of days since noon in Greenwich on November 24, 4714 B.C.
This is why date(18386) returns 4662-03-28B.C.
But I suspect that the date values that you have are the number of days since '1970-01-01'.
In this case, 18386 days after '1970-01-01' is '2020-05-04'.
So you can get the dates in the format YYYY-MM-DD if you add the value of your column as days to '1970-01-01':
SELECT date('1970-01-01', datecolumn || ' day') FROM tablename
Or by transforming your date values to seconds and treat them as UNIX time (the number of seconds since '1970-01-01 00:00:00 UTC'):
SELECT date(datecolumn * 24 * 3600, 'unixepoch') FROM tablename
Replace datecolumn with the name of your column.
I have a table including date column and hour column which is an integer type column varying from 0 to 24. I need to combine these two fields and create an hourly composite datetime field.
However, I was able to create that kind of variable by using || and cast. But I am unable to transform this code to Hive editor syntax. Can you help me with this problem
SQL Code:
CAST(CAST(CAST(DATE_OF_TRANSACTION AS FORMAT 'yyyy-mm-dd') AS VARCHAR(11))||' '||CAST(CAST( BasketHour AS FORMAT '99:') AS VARCHAR(10))||'00:00' AS TIMESTAMP(0)) Date_Time
Thank you very much
For example like this:
cast(concat(DATE_OF_TRANSACTION, ' ', lpad(BasketHour ,2,0),':00:00.0' ) as timestamp)
I tried to group my daily data by week (given a reference date) to generate a smaller panel data set.
I used postgres before and there it was quite easy:
CREATE TABLE videos_weekly AS SELECT channel_id,
CEIL(DATE_PART('day', observation_date - '2016-02-10')/7) AS week
FROM videos GROUP BY channel_id, week;
But it seems like it is not possible to subtract a timestamp with a date string in Drill. I found the AGE function, which returns an interval between two dates, but how to convert this into an integer (number of days or weeks)?
DATE_SUB may help you here. Following is an example:
SELECT extract(day from date_sub('2016-11-13', cast('2015-01-01' as timestamp)))/7 FROM (VALUES(1));
This will return number of weeks between 2015-01-01 and 2016-11-13.
Click here for documentation
I have a table called messages that stores messages from a chat with the following columns: username, message, datetime, where the type of datetime is TEXT and it is stored in the following format: "yyyy/MM/dd hh:mm:ss". I want to retrieve the average count of rows within a specific time range, without bothering with the date. For instance:
SELECT avg(count(message))
FROM messages
WHERE datetime < "2016/mm/dd 13:00:00" AND
datetime > "2016/mm/dd 12:00:00"
Is there some operator that allows any character to take the place of "mm" and "dd". Essentially, I am trying to construct a query that retrieves the average amount of messages within a specific time range, not the amount of messages on a specific date.
If I read your question correctly, you want to use your WHERE clause to restrict to any calendar date in 2016 between 12 and 13 hours. In this case, you can use STRFTIME to extract the year and hour in string format from your datetime column.
SELECT COUNT(message)
FROM messages
WHERE STRFTIME('%Y', datetime) = '2016' AND
STRFTIME('%H', datetime) < '13' AND
STRFTIME('%H', datetime) > '12'
Note that the reason while the inequalities should work with strings is because numerical strings still sort based on their lexigraphical order.
Update:
Since your datetime column is in a non standard format, you may be able to workaround this by substringing off the various pieces you need to use in the WHERE clause:
SELECT COUNT(message)
FROM messages
WHERE SUBSTR(datetime, 1, 4) = '2016' AND
SUBSTR(datetime, 12, 2) < '13' AND
SUBSTR(datetime, 12, 2) > '12'
I've just started using SQLite, instead of SQL Server, and it doesn't seem to want to do ORDER BY, MAX() or MIN() on dates.
The Survey_Date column is a text field, so ordering it sorts it from January to December, instead of by the year. If I include date(Survey_Date) in my SELECT statement, it will sort the data by year -- but I can't override the DESC sort, and it doesn't actually display the date.
SELECT Survey_Date FROM Surveys WHERE Loc_ID = 32 ORDER BY Survey_Date;
yields results like:
01/10/2009
01/20/2013
02/05/2010
...
SELECT date(Survey_Date), Survey_Date FROM Surveys WHERE Loc_ID = 32 ORDER BY 1 ASC;
yields results like:
|01/20/2013
|02/05/2010
|01/10/2009
...
It's clearly sorting on the date correctly now, but it doesn't display the formatted date, and doesn't recognize the ASC command.
Can anyone explain what it's doing?
As already explained at least million times over this site, SQLite does not have date/timestamp type.
So if you want to sort by date, you must store them in format that sorts correctly either numerically, or lexicographically (asciibetically/unicodebetically).
The recommended format is ISO-8601, that is yyyy-mm-dd (yyyy-mm-ddTHH:MM:SS if you want a time too). This is what SQLite has some utility functions to work with, too.
Other possible formats are numeric, either number of seconds since some specific point, e.g. Unix time or number of days, e.g. (Modified) Julian day.
01/10/2009
01/20/2013
02/05/2010
They are strings, so this is correctly sorted.
SELECT date(Survey_Date), Survey_Date FROM Surveys WHERE Loc_ID = 32 ORDER BY 1 ASC;
The date function expects ISO-8601 format. It does not understand what you are giving it and returns NULL. Fully expected.
Sorting by all NULL values effectively does nothing. The rows came out sorted by accident.