I've just started using SQLite, instead of SQL Server, and it doesn't seem to want to do ORDER BY, MAX() or MIN() on dates.
The Survey_Date column is a text field, so ordering it sorts it from January to December, instead of by the year. If I include date(Survey_Date) in my SELECT statement, it will sort the data by year -- but I can't override the DESC sort, and it doesn't actually display the date.
SELECT Survey_Date FROM Surveys WHERE Loc_ID = 32 ORDER BY Survey_Date;
yields results like:
01/10/2009
01/20/2013
02/05/2010
...
SELECT date(Survey_Date), Survey_Date FROM Surveys WHERE Loc_ID = 32 ORDER BY 1 ASC;
yields results like:
|01/20/2013
|02/05/2010
|01/10/2009
...
It's clearly sorting on the date correctly now, but it doesn't display the formatted date, and doesn't recognize the ASC command.
Can anyone explain what it's doing?
As already explained at least million times over this site, SQLite does not have date/timestamp type.
So if you want to sort by date, you must store them in format that sorts correctly either numerically, or lexicographically (asciibetically/unicodebetically).
The recommended format is ISO-8601, that is yyyy-mm-dd (yyyy-mm-ddTHH:MM:SS if you want a time too). This is what SQLite has some utility functions to work with, too.
Other possible formats are numeric, either number of seconds since some specific point, e.g. Unix time or number of days, e.g. (Modified) Julian day.
01/10/2009
01/20/2013
02/05/2010
They are strings, so this is correctly sorted.
SELECT date(Survey_Date), Survey_Date FROM Surveys WHERE Loc_ID = 32 ORDER BY 1 ASC;
The date function expects ISO-8601 format. It does not understand what you are giving it and returns NULL. Fully expected.
Sorting by all NULL values effectively does nothing. The rows came out sorted by accident.
Related
I am working locally with an sqllite DB. I have imported some records from teradata where there was a date field in the format of 'YYYY-MM-DD'. When i imported the records the date switched from a date to a number. I know this is a feature of sqllite and that one can access it via date(sqllite_date) when selecting it in a where clause.
My problem is that the dates now appear to be a bit odd. For example the year appears to be negative.
Is there anyway to recover this to the correct format?
Below is an example of converting a number in the database into a date
SELECT date(18386)
# -4662-03-28
SELECT datetime('now')
# 2021-02-11 10:41:52
SELECT date(sqllite_date) FROM mydb
# Returns -4662-03-28
# Should return 2020-05-04
I am very new to this area so apologies if this is a basic question. Thank you very much for your time
In SQLite you can store dates as TEXT, REAL or INTEGER.
It seems that you stored the dates in a column with INTEGER or REAL affinity.
In this case, if you use the function date(), it considers a value like 18386 as a Julian day, meaning the number of days since noon in Greenwich on November 24, 4714 B.C.
This is why date(18386) returns 4662-03-28B.C.
But I suspect that the date values that you have are the number of days since '1970-01-01'.
In this case, 18386 days after '1970-01-01' is '2020-05-04'.
So you can get the dates in the format YYYY-MM-DD if you add the value of your column as days to '1970-01-01':
SELECT date('1970-01-01', datecolumn || ' day') FROM tablename
Or by transforming your date values to seconds and treat them as UNIX time (the number of seconds since '1970-01-01 00:00:00 UTC'):
SELECT date(datecolumn * 24 * 3600, 'unixepoch') FROM tablename
Replace datecolumn with the name of your column.
I tried to group my daily data by week (given a reference date) to generate a smaller panel data set.
I used postgres before and there it was quite easy:
CREATE TABLE videos_weekly AS SELECT channel_id,
CEIL(DATE_PART('day', observation_date - '2016-02-10')/7) AS week
FROM videos GROUP BY channel_id, week;
But it seems like it is not possible to subtract a timestamp with a date string in Drill. I found the AGE function, which returns an interval between two dates, but how to convert this into an integer (number of days or weeks)?
DATE_SUB may help you here. Following is an example:
SELECT extract(day from date_sub('2016-11-13', cast('2015-01-01' as timestamp)))/7 FROM (VALUES(1));
This will return number of weeks between 2015-01-01 and 2016-11-13.
Click here for documentation
I have been trying with no success to to count how many values were created in a specific week day:
SELECT count(*) as count FROM packets WHERE strftime("%w", timeIn) = '1';
I have this values in timeIn
1472434822.60033
1472434829.12632
1472434962.34593
I don't know what I am doing wrong here.
furthermore, if I use this:
SELECT count(*) as count FROM packets WHERE strftime("%w", timeIn) = '6';
I get
2
which makes no sense. Thank you in advance.
You appear to be storing the date as the number of seconds since 1970 (the Unix epoch) - a common representation. The time strings accepted by the SQLite date functions (see the Time Strings section) default to interpreting numeric time strings as a Julian day numbers:
Similarly, format 12 is shown with 10 significant digits, but the date/time functions will really accept as many or as few digits as are necessary to represent the Julian day number.
You can see this with the following SELECT:
SELECT strftime('%Y-%m-%d', 1472428800.6) AS t
the result of which is:
4026-48-26
For your date representation to be interpreted as a Unix epoch, you need to include 'unixepoch' in the strftime call:
SELECT strftime('%Y-%m-%d', 1472428800.6, 'unixepoch') AS t
which returns:
2016-08-29
If you modify your SELECT to be:
SELECT count(*) as count FROM packets WHERE strftime("%w", timeIn, 'unixepoch') = '6'
you should see results more inline with your expectations.
I am needing to count month between collect dates. I need to know if the test was run in the last 3 months. Below is the code I used but it is giving me a count of zero, but I know they had 3 of the same tests run in a year because I can see the dates. I understand the first one have a count of zero, because there is no test before that, but the count for the other should be 3, 5 respectively.
DATEDIFF('month',[Collect Date],[Collect Date])
Dates of the Tests.
1/8/2015
4/23/2015
9/30/2015
What you are looking for is possible using the LOOKUP function in Tableau. Keep in mind, that the result relies heavily on the data that is displayed and how it is displayed (sorted, etc).
You can create a calculated field like this:
DATEDIFF("month",LOOKUP(ATTR([Test Date]),-1),ATTR([Test Date]))
Which calculates the number of months between the date in the current row and the date from the prior row.
Your result will look something like this:
I'm trying to re-partition some table using week number counting from some day:
my_fact table contains a field called time_stamp of type TIMESTAMPTZ
Unfortunately, re-partition doesn't work, and I'm getting the error:
MyDB=> ALTER TABLE my_fact PARTITION BY MOD(TIMESTAMPDIFF('day', time_stamp::TIMESTAMP, TIMESTAMP '2013-09-23'), 156) REORGANIZE;
NOTICE 4954: The new partitioning scheme will produce 12 partitions
ROLLBACK 2552: Cannot use meta function or non-deterministic function in PARTITION BY expression
Should the cast of time_stamp to TIMESTAMP strip any time zone related info from this field thus making it deterministic?
Thanks!
Take a look at the date_part() function, you can use the TIMESTAMPTZ as its source column:
Example :
**The number of the week of the calendar year that the day is in.**
SELECT EXTRACT(WEEK FROM TIMESTAMP '2001-02-16 20:38:40');
Result: 7
SELECT EXTRACT(WEEK FROM DATE '2001-02-16');
Result: 7
Since I got no answer, I'm writing here what I've ended up with:
ALTER TABLE my_fact PARTITION BY
MOD(
TIMESTAMPDIFF(
'day',
'2013-09-23'::timestamptz AT TIME ZONE 'UTC',
time_stamp AT TIME ZONE 'UTC'),
156)
REORGANIZE;
This solution works.