I was asked to create a query to pull a near-real-time report from an Informix database (I have select access only, I cannot create a SP) and I felt like I succeeded pretty well until I realized that there was a discrepancy in a datetime field. it seems that the program that is populating the db is hard-coded to enter the time in the datetime field in UTC (five hours off of the local time. When the time was 2:30 it entered a row in the database saying John Doe completed the task at 7:30). In my report I am supposed to calculate the number of seconds (as an int) since the user completed the task (field is "completionTime") and I was originally just using:
sysdate - completionTime interval seconds(9) to seconds cast to char then cast to int
When I realized the mistake in the timezone of the completionTime field I just subtracted the offset as an integer (I was already converting the interval to an integer, so I just adjusted the answer by 18000). This worked just fine until Daylight Saving started. Then all of a sudden local time was 4 hours (14400 seconds instead of 18000) off of UTC.
Since I can only select from the db, I next tried using an inefficient case statement (my query went from <0.5 seconds to 3-5 seconds for only 25 rows). Following a suggestion from another forum I changed the time to an integer of seconds from the unix epoch, then used the dbinfo('utc_to_datetime') sp to convert it back to a datetime in the right timezone.
This approach works, but the calculation looks terrible to me:
cast(cast(cast((sysdate - dbinfo("utc_to_datetime", cast(cast(cast((completionTime - TO_DATE('Friday January 1, 2010 0:00', '%A %B %d, %Y %R')) as interval second(9) to second) as char(10)) as int) +1262304000)) as interval second(9) to second) as char(10)) as int)
notice that I am calculating the length of time from the completiontime to 1-1-2010 then adding 12 billion seconds (going all the way back to the unix epoch is too big for Informix's interval seconds(9) to second, hence the two-steps) so that I can then plug it into the dbinfo("utc_to_datetime") sp to convert it back to a datetime in the right timezone, then subtracting it from sysdate. The worst part (besides the six casts) is that the completiontimes that I am dealing with are all within 24 hours of sysdate, most are within 10 minutes, yet I am adding on 12 billion seconds so that I can use the only function I can find that converts between timezones.
My question is, Is this really the best way to do this? By the way, this works very quickly, and my query is back down to a reasonable execution time (<0.5 seconds), I'm just looking at this query and thinking that there has got to be a better way.
Jared
Maybe instead of sysdate you can use DBINFO('utc_current'):
SELECT DBINFO('utc_current') - (completionTime interval seconds(9) to seconds) FROM ...
Related
I need to save a time interval in a column in a table. based on: http://docs.sqlalchemy.org/en/rel_0_8/core/types.html
I can use Interval type for that. My database is SQLite, and I don't quite understand this description in the document:
"The Interval type deals with datetime.timedelta objects. In PostgreSQL, the
native INTERVAL type is used; for others, the value is stored as a date which
is relative to the “epoch” (Jan. 1, 1970)."
Can anybody tell me how should I do that?
So from what I get in the question, you want to just store an interval and take it out of the database to use it again? But you want to understand how it is stored?
Concerning the storage: This is probably easier with Unix timestamps than with DateTimes. Suppose you want to store timedelta(1), i.e. a delta of one day. What is stored in the database is the time since the "epoch", i.e. second "0" in Unix timestamps and as a date: 1970-01-01 00:00:00 (this is where Unix timestamps start counting the seconds). If you don't know about epoch or timestamp, then read Wikipedia on Unix time.
So we want to store one day of difference? The documentation claims it stored "time since epoch". We just learned "epoch" is "second 0", so a day later would be 60 seconds per minute, 60 minutes per hour, 24 hours per day: 60 * 60 * 24 = 86400. So stored as an integer this is easy to understand: If you find the value 86400 in your database, then it means 1 day, 0 hours, 0 minutes, 0 seconds.
Reality is a bit different: It does not store an integer but a DateTime object. Speaking from this perspective, the epoch is 1970-01-01 00:00:00. So what is a delta of one day since the epoch? That is easy: it's 1970-01-02 00:00:00. You can see, it is a day later.
An hour later? 1970-01-01 01:00:00.
Two days, four hours, 30 seconds?: 1970-01-03 04:00:30.
And you could even do it yourself:
epoch = datetime.utcfromtimestamp(0)
delta = timedelta(1)
one_day = datetime.utcfromtimestamp(86400)
print "Date to be stored in database:", epoch + delta
print "Timedelta from date:", one_day - epoch
As you can see, the calculation is easy and this is all that is done behind the scenes. Take a look at this full example:
interval = IntervalItem(interval=delta)
session.add(interval)
i = session.query(IntervalItem).first()
print "Timedelta from database:", i.interval
You can see it is no different from the above example except it goes through the database. The only thing to keep in mind with this, is this note:
Note that the Interval type does not currently provide date arithmetic operations
on platforms which do not support interval types natively.
That means you should be careful how you use it, for example addition in the query might not be a good idea, but you should just play around with it.
I kind of assumed it was a string, so I compared it as a string, but not surprisingly it failed. I believe thats how it works in Mysql. I could be wrong as I haven't worked on it in a while. In either case, how can I check if dates are equal in SQLite? I will be using it in a WHERE clause.
SELECT a._id, b._id, b.start_date,a.event_name, b.start_time,
b.end_date, b.end_time, b.location FROM events_info b INNER JOIN events a ON
a._id=b.event_id WHERE b.start_time = '6:00';
(added space to make it easier to look at)
SQLite doesn't have a dedicated DATETIME type. Normally what people do is make sure they store the date as a formatted string that is consistent; for example, YYYY-MM-DD hh:mm:ss. If you do so, as long as you're consistent, then you can compare dates directly:
SELECT * FROM a WHERE q_date < '2013-01-01 00:00:00';
This works because even though the comparison is technically an alphabetical comparison and not a numeric one, dates in a consistent format like this sort alphabetically as well as numerically.
For such a schema, I would suggest storing dates in 24-hour format (the above example is midnight). Pad months, days, and hours with zeros. If your dates will span multiple timezones, store them all in UTC and do whatever conversion you need client-side to convert them to the local time zone.
Normally dates and times are stored all in one column. If you have to have them separated for whatever reason, just make sure you dates are all consistent and your times are all consistent. For example, dates should all be YYYY-MM-DD and times should all be hh:mm:ss.
The reason that YYYY-MM-DD hh:mm:ss is the preferred format is because when you go from the largest date interval (years) to the smallest (seconds), you can index and sort them very easily and with high performance.
SELECT * FROM a WHERE q_date = '2012-06-04 05:06:00';
would use the index to hone in on the date/time instead of having to do a full table scan. Or if they're in two separate rows:
SELECT * FROM a WHERE q_date = '2012-06-04' AND q_time = '05:06:00';
The key is to make sure that the dates and times are in a consistent format going into the database. For user-friendly presentation, do all conversion client-side, not in the database. (For example, convert '2012-06-04 05:06:00' to "1:06am Eastern 6/4/2012".)
If this doesn't answer question, could you please post the exact format that you're using to store your dates and times, and two example dates that you're trying to compare that aren't working the way you expect them to?
Sqlite can not compare dates directly. we need to convert them in seconds as well as integer also.
Example
SELECT * FROM Table
WHERE
CAST(strftime('%s', date_field) AS integer) <=CAST(strftime('%s', '2015-01-01') AS integer) ;
From Datatypes In SQLite Version 3:
1.2 Date and Time Datatype
SQLite does not have a storage class set aside for storing dates and/or times. Instead, the built-in Date And Time Functions of SQLite are capable of storing dates and times as TEXT, REAL, or INTEGER values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS").
REAL as Julian day numbers, the number of days since noon in Greenwich on November 24, 4714 B.C. according to the proleptic Gregorian calendar.
INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
Applications can chose to store dates and times in any of these formats and freely convert between formats using the built-in date and time functions.
If you look at the examples in Date And Time Functions, something like this should get you close to what you want (which, I'm assuming, is 6:00 of the current day):
WHERE b.start_time = date('now', 'start of day', '+6 hours')
I'm using the following syntax
TIMESTAMPDIFF(2, CHAR(CREATED - TIMESTAMP('1970-01-01 00:00:00'))
where CREATED is of type TIMESTAMP and the database is DB2. The intension is to get the timestamp converted to millis from epoch. If there is a better function that would be more helpful.
Sample data:
For 2011-10-04 13:54:50 returned value is 1316613290 but actual value should be 1317732890 (got from http://www.epochconverter.com)
Query to run
SELECT TIMESTAMPDIFF(2, CHAR(TIMESTAMP('2011-10-04 13:54:50') - TIMESTAMP('1970-01-01 00:00:00'))) FROM SYSIBM.SYSDUMMY1;
This is the result of the fact that TIMESTAMPDIFF returns an estimate of the difference between the timestamps, not the actual value, as expected.
From the reference, page 435 (assuming for iSeries):
The following assumptions are used when converting the element values
to the requested interval type:
One year has 365 days.
One year has 52 weeks.
One year has 12 months.
One quarter has 3 months.
One month has 30 days.
One week has 7 days.
One day has 24 hours.
One hour has 60 minutes.
One minute has 60 seconds.
One second has 1000000 microseconds.
And the actual calculation used is:
seconds + (minutes + (hours + ((days + (months * 30) + (years * 365)) * 24)) * 60) * 60
This is, for obvious reasons, inexact. Not helpful.
This appears to be a direct consequence of the way the timestamp arithmetic results are returned.
That is;
SELECT
TIMESTAMP('1971-03-02 00:00:00') - TIMESTAMP('1970-01-01 00:00:00')
FROM sysibm/sysdummy1
returns:
10,201,000,000.000000
Which can be divided into:
1 year
02 months
01 days
00 hours
00 minutes
00 seconds
000000 microseconds
Which is imprecise period/duration information. While there are a multitude of situations where this type of data is useful, this isn't one of them.
Short answer: The exact answer cannot be correctly calculated in the database, and in fact should not.
Long answer:
The calculations are possible, but rather complex, and definitely not suited for in-database calculation. I'm not going to reproduce them here (look up JodaTime if you're interested, specifically the various Chronology subclasses). Your biggest problem is going to be the fact that months aren't all the same length. Also, you're going to run into major problems if your timestamps are anything other than UTC - more specifically, Daylight Savings time is going to play havoc with the calculation. Why? Because the offsets can change at any time, for any country.
Maybe you could explain why you need the number of milliseconds? Hopefully you're using Java (or able to do so), and can use java.time. But if you're on an iSeries, it's probably RPG...
According to the v9.7 info center, TIMESTAMPDIFF returns an estimated time difference, based on 365 days in a year (not true ~25% of the time), 30 days in a month (not true 75% of the time, though averages out a bit better than that), 24 hours in a day (not true a couple days of the year in some timezones), 60 minutes in an hour (hooray, one right!), and 60 seconds in a minute (true >99.9% of the time - we do get leap seconds).
So, no, this is not the way to get epoch time in DB2. Thus far, I've resorted to getting the time as a timestamp, and converting it in the client.
Part of your error occurs because of the inaccuracy of the TIMESTAMPDIFF function, as others have pointed out.
The other source of error occurs because the Epoch is based on GMT – so you have to take your local timezone into account.
So, you can do this with the following expression:
(DAYS(timestamp('2011-10-04-13.54.50.000000') - current timezone) - DAYS('1970-01-01-00.00.00.000000')) * 86400 + MIDNIGHT_SECONDS(timestamp('2011-10-04-13.54.50.000000') - current timezone)
You can write a simple UDF to simplify this:
create or replace function epoch (in db2ts timestamp)
returns bigint
language sql
deterministic
no external action
return (days(db2ts - current timezone) - days('1970-01-01-00.00.00.000000')) * 86400 + midnight_seconds(db2ts - current timezone);
Good luck,
I'm creating a report from a stored procedure that pulls two date/times (CreatedDate and ClosedDate). I need a column on the report that shows the difference (i.e. time it took to go from open to close). First, I just subtracted CreatedDate from ClosedDate (in the report [SQL Server Reporting Services], not in the stored procedure) and got a time that looks like this: 72.20:34:18.6230000 (day.hour:minute:second). I need to shrink this down, if possible, to just day.hour...
I was experimenting with some of the functions found on MSDN (http://msdn.microsoft.com/en-us/library/ms186724.aspx). DATEDIFF almost gives me what I need, but I can only specify days or hours, and ideally (as I said), I need it to show the 'time to close' as both (day.hour).
Is this possible?
In SSRS, you can apply a custom format to that column to show only days.hours. Right click the column in design mode -->text box properties-->Number(on left hand side). If you don't see one of the formats for date, time or number that fits what you need, create a custom one at the bottom.
Convert to minutes
divide by 1440 gives whole days
modulo 1440 gives remaining minutes, divide by 60 for hours
Something like (not tested):
SELECT
CAST(DATEDIFF(minute, CreatedDate, ClosedDate) / 1440 AS varchar(20)) + '.'
CAST((DATEDIFF(minute, CreatedDate, ClosedDate) % 1440) / 60 AS varchar(20))
FROM
MyTable
You may need to fiddle with the hours representation in cade I've misunderstood
DATEDIFF for day and hour go by boundaries: that is if there are just 3 minutes between the 2 values spanning midnight, there will be one hour/day difference. So I used minutes
Edit:
To overflow the int from DATEDIFF requires a difference of 4000+ years
Thoughts:
Using datetime2 and have CreatedDate of lowest 0001-01-01
ClosedDate is a sentinel value like 9999-12-31 say for "open" items
CreatedDate and ClosedDate are varchar and conversion to datetime is faulty
Your example shows 72 days difference which would be around 104k minutes.
I would try this to see where you have more than 1000 year differences which would be a mere half billion or so minutes:
SELECT * FROM mytables
WHERE DATEDIFF(year, CreatedDate, ClosedDate) > 1000
How do I get the difference in days between 2 dates in SQLite? I have already tried something like this:
SELECT Date('now') - DateCreated FROM Payment
It returns 0 every time.
SELECT julianday('now') - julianday(DateCreated) FROM Payment;
Difference In Days
Select Cast ((
JulianDay(ToDate) - JulianDay(FromDate)
) As Integer)
Difference In Hours
Select Cast ((
JulianDay(ToDate) - JulianDay(FromDate)
) * 24 As Integer)
Difference In Minutes
Select Cast ((
JulianDay(ToDate) - JulianDay(FromDate)
) * 24 * 60 As Integer)
Difference In Seconds
Select Cast ((
JulianDay(ToDate) - JulianDay(FromDate)
) * 24 * 60 * 60 As Integer)
Both answers provide solutions a bit more complex, as they
need to be. Say the payment was created on January 6, 2013.
And we want to know the difference between this date and today.
sqlite> SELECT julianday() - julianday('2013-01-06');
34.7978485878557
The difference is 34 days. We can use julianday('now') for
better clarity. In other words, we do not need to put
date() or datetime() functions as parameters to julianday()
function.
The SQLite documentation is a great reference and the DateAndTimeFunctions page is a good one to bookmark.
It's also helpful to remember that it's pretty easy to play with queries with the sqlite command line utility:
sqlite> select julianday(datetime('now'));
2454788.09219907
sqlite> select datetime(julianday(datetime('now')));
2008-11-17 14:13:55
This answer is a little long-winded, and the documentation will not tell you this (because they assume you are storing your dates as UTC dates in the database), but the answer to this question depends largely on the timezone that your dates are stored in. You also don't use Date('now'), but use the julianday() function, to calculate both dates back against a common date, then subtract the difference of those results from each other.
If your dates are stored in UTC:
SELECT julianday('now') - julianday(DateCreated) FROM Payment;
This is what the top-ranked answer has, and is also in the documentation. It is only part of the picture, and a very simplistic answer, if you ask me.
If your dates are stored in local time, using the above code will make your answer WRONG by the number of hours your GMT offset is. If you are in the Eastern U.S. like me, which is GMT -5, your result will have 5 hours added onto it. And if you try making DateCreated conform to UTC because julianday('now') goes against a GMT date:
SELECT julianday('now') - julianday(DateCreated, 'utc') FROM Payment;
This has a bug where it will add an hour for a DateCreated that is during Daylight Savings Time (March-November). Say that "now" is at noon on a non-DST day, and you created something back in June (during DST) at noon, your result will give 1 hour apart, instead of 0 hours, for the hours portion. You'd have to write a function in your application's code that is displaying the result to modify the result and subtract an hour from DST dates. I did that, until I realized there's a better solution to that problem that I was having: SQLite vs. Oracle - Calculating date differences - hours
Instead, as was pointed out to me, for dates stored in local time, make both match to local time:
SELECT julianday('now', 'localtime') - julianday(DateCreated) FROM Payment;
Or append 'Z' to local time:
julianday(datetime('now', 'localtime')||'Z') - julianday(CREATED_DATE||'Z')
Both of these seem to compensate and do not add the extra hour for DST dates and do straight subtraction - so that item created at noon on a DST day, when checking at noon on a non-DST day, will not get an extra hour when performing the calculation.
And while I recognize most will say don't store dates in local time in your database, and to store them in UTC so you don't run into this, well not every application has a world-wide audience, and not every programmer wants to go through the conversion of EVERY date in their system to UTC and back again every time they do a GET or SET in the database and deal with figuring out if something is local or in UTC.
Just a note for writing timeclock functions. For those looking for hours worked, a very simple change of this gets the hours plus the minutes are shown as a percentage of 60 as most payroll companies want it.
CAST ((julianday(clockOUT) - julianday(clockIN)) * 24 AS REAL) AS HoursWorked
Clock In Clock Out HoursWorked
2016-08-07 11:56 2016-08-07 18:46 6.83333332836628
Given that your date format follows : "YYYY-MM-DD HH:MM:SS",
if you need to find the difference between two dates in number of months :
(strftime('%m', date1) + 12*strftime('%Y', date1)) -
(strftime('%m', date2) + 12*strftime('%Y', date2))
Firstly, it's not clear what your date format is.
There already is an answer involving strftime("%s").
I like to expand on that answer.
SQLite has only the following storage classes: NULL, INTEGER, REAL, TEXT or BLOB.
To simplify things, I'm going to assume dates are REAL containing the seconds since 1970-01-01.
Here's a sample schema for which I will put in the sample data of "1st December 2018":
CREATE TABLE Payment (DateCreated REAL);
INSERT INTO Payment VALUES (strftime("%s", "2018-12-01"));
Now let's work out the date difference between "1st December 2018" and now (as I write this, it is midday 12th December 2018):
Date difference in days:
SELECT (strftime("%s", "now") - DateCreated) / 86400.0 FROM Payment;
-- Output: 11.066875
Date difference in hours:
SELECT (strftime("%s", "now") - DateCreated) / 3600.0 FROM Payment;
-- Output: 265.606388888889
Date difference in minutes:
SELECT (strftime("%s", "now") - DateCreated) / 60.0 FROM Payment;
-- Output: 15936.4833333333
Date difference in seconds:
SELECT (strftime("%s", "now") - DateCreated) FROM Payment;
-- Output: 956195.0
If you want time in 00:00 format:
I solved it like that:
SELECT strftime('%H:%M',
CAST((julianday(FinishTime) - julianday(StartTime)) AS REAL),
'12:00')
FROM something;
If you want difference in seconds
SELECT strftime('%s', '2019-12-02 12:32:53') - strftime('%s', '2019-12-02 11:32:53')
If you want records in between days,
select count(col_Name) from dataset where cast(julianday("now")- julianday(_Last_updated) as int)<=0;
In my case, I have to calculate the difference in minutes and julianday() does not give an accurate value. Instead, I use strftime():
SELECT (strftime('%s', [UserEnd]) - strftime('%s', [UserStart])) / 60
Both dates are converted to unixtime (seconds), then subtracted to get
value in seconds between the two dates. Next, divide it by 60.
https://www.sqlite.org/cvstrac/wiki?p=DateAndTimeFunctions