I'm synchronizing historical data between two systems and I've found a small clock problem between their logs.
I've loaded the data into an SqlLite and need one of the sets by a small amount (~40 milliseconds). However, I'm unable to do so as it seems to always round the time to the nearest second.
For example, attempting something like the following
UPDATE my_table SET my_datetime = DATETIME(my_datetime, '+0.04 seconds') rounds up to the nearest second and I can't find any fractional/millisecond modifier option.
Is there a way to do this that I'm overlooking?
Thanks.
SQLite hasn't a type for datetime. See http://sqlite.org/datatype3.html
Using datetime(...) you are storing your dates as strings. This is equivalent to strftime('%Y-%m-%d %H:%M:%S', ...).
One option is to use a strftime with fractions of seconds:
UPDATE my_table SET my_datetime = STRFTIME('%Y-%m-%d %H:%M:%f',my_datetime, '+0.04 seconds')
Related
I am having an issue in using the RPostgres package to connect with Redshift. I am unsure if it is an issue with our database set up, or if it is a known issue with the package.
I am getting different results when I use the same exact query in Redshift vs. using it in R with the RPostgres package.
It appears entirely due to the date math, as my overall row counts match, as well as everything else, when not using dates.
As an example, this may be a query I run in Redshift (using Metabase). If I run the same exact query in R with the RPostgres package, I will get completely different results.
SELECT
orders.*
FROM
orders
WHERE
orders.date >= current_date-3
AND
orders.date < current_date-2
The dates in Metabase make sense. It shows only one day, three days ago. However, in R, it shows 2 days.
For the purposes of this example, consider date a timestamp.
Has anyone ran into this, or know of an existing issue and work around?
It's usually better to be absolutely explicit with dates to make sure that session settings are not unintentionally affecting the query. Try this:
SELECT
orders.*
FROM
orders
WHERE
orders.date >= date_trunc('day', current_timestamp at time zone 'utc') - '3 days'::interval
AND
orders.date < date_trunc('day', current_timestamp at time zone 'utc') - '2 days'::interval
You may have to change the time zone from utc if your dates are implicitly stored in a different time zone.
How can I compute in SQLite the minimum interval of time in milliseconds ?
Ok to give some context,
Here is how my table look like:
link_budget table
So there is this time column, I would like to make a request that gives me the intervals between two consecutive rows in milliseconds.
I don't know how to make my requests.
Any suggestions ?
Since SQLite does not have a native datetime data type, instead forcing you to either store as a string or as a number, and you have picked a string, you need to convert it into something that can be calculated upon.
You can use the julianday function for this.
To calculate the difference between two values you would simply do this:
SELECT julianday('2006-07-01 12:08:15.310')-julianday('2006-07-01 12:08:14.141')
This, however, will give you the difference in days. One day contains 86400 seconds and thus 86400000 milliseconds, which gives you this:
SELECT (julianday('2006-07-01 12:08:15.310')-julianday('2006-07-01 12:08:14.141'))*86400000
Note that the precision of the floating point types used internally by SQLite does not have enough precision to get the above value accurate but it should likely be close enough to give you millisecond-precision.
For instance, this:
select (julianday('2006-07-01 12:08:14.141')-julianday('2006-07-01 12:08:14.140'))*(86400000)
Which should give 1 millisecond of difference gives this result:
1.00582838058472
try this:
SELECT (STRFTIME('%s', '2021-10-20 18:22:43') - STRFTIME('%s', '2021-10-20 18:22:42'))*1000;
for timestamps that contain milliseconds try this:
SELECT STRFTIME('%s', '2021-10-20 18:22:43.325')*1000 + SUBSTR(STRFTIME('%f', '2021-10-20 18:22:43.325'), 4) -
( STRFTIME('%s', '2021-10-20 18:22:42.921')*1000 + SUBSTR(STRFTIME('%f', '2021-10-20 18:22:42.921'), 4) );
I was asked to create a query to pull a near-real-time report from an Informix database (I have select access only, I cannot create a SP) and I felt like I succeeded pretty well until I realized that there was a discrepancy in a datetime field. it seems that the program that is populating the db is hard-coded to enter the time in the datetime field in UTC (five hours off of the local time. When the time was 2:30 it entered a row in the database saying John Doe completed the task at 7:30). In my report I am supposed to calculate the number of seconds (as an int) since the user completed the task (field is "completionTime") and I was originally just using:
sysdate - completionTime interval seconds(9) to seconds cast to char then cast to int
When I realized the mistake in the timezone of the completionTime field I just subtracted the offset as an integer (I was already converting the interval to an integer, so I just adjusted the answer by 18000). This worked just fine until Daylight Saving started. Then all of a sudden local time was 4 hours (14400 seconds instead of 18000) off of UTC.
Since I can only select from the db, I next tried using an inefficient case statement (my query went from <0.5 seconds to 3-5 seconds for only 25 rows). Following a suggestion from another forum I changed the time to an integer of seconds from the unix epoch, then used the dbinfo('utc_to_datetime') sp to convert it back to a datetime in the right timezone.
This approach works, but the calculation looks terrible to me:
cast(cast(cast((sysdate - dbinfo("utc_to_datetime", cast(cast(cast((completionTime - TO_DATE('Friday January 1, 2010 0:00', '%A %B %d, %Y %R')) as interval second(9) to second) as char(10)) as int) +1262304000)) as interval second(9) to second) as char(10)) as int)
notice that I am calculating the length of time from the completiontime to 1-1-2010 then adding 12 billion seconds (going all the way back to the unix epoch is too big for Informix's interval seconds(9) to second, hence the two-steps) so that I can then plug it into the dbinfo("utc_to_datetime") sp to convert it back to a datetime in the right timezone, then subtracting it from sysdate. The worst part (besides the six casts) is that the completiontimes that I am dealing with are all within 24 hours of sysdate, most are within 10 minutes, yet I am adding on 12 billion seconds so that I can use the only function I can find that converts between timezones.
My question is, Is this really the best way to do this? By the way, this works very quickly, and my query is back down to a reasonable execution time (<0.5 seconds), I'm just looking at this query and thinking that there has got to be a better way.
Jared
Maybe instead of sysdate you can use DBINFO('utc_current'):
SELECT DBINFO('utc_current') - (completionTime interval seconds(9) to seconds) FROM ...
I have a table that includes a 'LastUpdated' column that is generated when the row is inserted using Sqlite's datetime('now') function.
How do I write a Select statement that finds all rows with 'LastUpdated' more than 100 days old?
I think it's a variant of:
SELECT * FROM Table WHERE (DATETIME('Now')-100 Days) > LastUpdated
But I'm unsure of:
a) How to specify the 100 Days?
b) Whether I can actually compare datetimes like this or if I first have to convert DATETIME('Now') to a string?
c) DATETIME('Now') results in UTC time, correct? I think so from my reading of the documentation, but it was a little confusing...
Figured it out--I didn't see all the handy modifiers at the bottom of the SQLite Datetime Documentation.
A bunch of helpful examples there demonstrating addition/subtraction of any datetime unit (years, months, hours, seconds, etc)
SELECT * FROM Table WHERE (DATETIME('Now','-100 Days') > LastUpdated
I kind of assumed it was a string, so I compared it as a string, but not surprisingly it failed. I believe thats how it works in Mysql. I could be wrong as I haven't worked on it in a while. In either case, how can I check if dates are equal in SQLite? I will be using it in a WHERE clause.
SELECT a._id, b._id, b.start_date,a.event_name, b.start_time,
b.end_date, b.end_time, b.location FROM events_info b INNER JOIN events a ON
a._id=b.event_id WHERE b.start_time = '6:00';
(added space to make it easier to look at)
SQLite doesn't have a dedicated DATETIME type. Normally what people do is make sure they store the date as a formatted string that is consistent; for example, YYYY-MM-DD hh:mm:ss. If you do so, as long as you're consistent, then you can compare dates directly:
SELECT * FROM a WHERE q_date < '2013-01-01 00:00:00';
This works because even though the comparison is technically an alphabetical comparison and not a numeric one, dates in a consistent format like this sort alphabetically as well as numerically.
For such a schema, I would suggest storing dates in 24-hour format (the above example is midnight). Pad months, days, and hours with zeros. If your dates will span multiple timezones, store them all in UTC and do whatever conversion you need client-side to convert them to the local time zone.
Normally dates and times are stored all in one column. If you have to have them separated for whatever reason, just make sure you dates are all consistent and your times are all consistent. For example, dates should all be YYYY-MM-DD and times should all be hh:mm:ss.
The reason that YYYY-MM-DD hh:mm:ss is the preferred format is because when you go from the largest date interval (years) to the smallest (seconds), you can index and sort them very easily and with high performance.
SELECT * FROM a WHERE q_date = '2012-06-04 05:06:00';
would use the index to hone in on the date/time instead of having to do a full table scan. Or if they're in two separate rows:
SELECT * FROM a WHERE q_date = '2012-06-04' AND q_time = '05:06:00';
The key is to make sure that the dates and times are in a consistent format going into the database. For user-friendly presentation, do all conversion client-side, not in the database. (For example, convert '2012-06-04 05:06:00' to "1:06am Eastern 6/4/2012".)
If this doesn't answer question, could you please post the exact format that you're using to store your dates and times, and two example dates that you're trying to compare that aren't working the way you expect them to?
Sqlite can not compare dates directly. we need to convert them in seconds as well as integer also.
Example
SELECT * FROM Table
WHERE
CAST(strftime('%s', date_field) AS integer) <=CAST(strftime('%s', '2015-01-01') AS integer) ;
From Datatypes In SQLite Version 3:
1.2 Date and Time Datatype
SQLite does not have a storage class set aside for storing dates and/or times. Instead, the built-in Date And Time Functions of SQLite are capable of storing dates and times as TEXT, REAL, or INTEGER values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS").
REAL as Julian day numbers, the number of days since noon in Greenwich on November 24, 4714 B.C. according to the proleptic Gregorian calendar.
INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
Applications can chose to store dates and times in any of these formats and freely convert between formats using the built-in date and time functions.
If you look at the examples in Date And Time Functions, something like this should get you close to what you want (which, I'm assuming, is 6:00 of the current day):
WHERE b.start_time = date('now', 'start of day', '+6 hours')