I had issue with function as.POSIXlt that appears very mysterious for me. I have in data.frame over 100 000 datetimes but 3 of them didn't behave like they should and changed all datetimes when these 3 dates were included.
time=c("2008-03-30 03:07:44","2008-03-30 03:48:56","2012-03-25 03:22:20")
d=as.POSIXlt(time)
d
## [1] "2008-03-30" "2008-03-30" "2012-03-25"
but it should be:
## [1] "2008-03-30 03:07:44" "2008-03-30 03:48:56" "2012-03-25 03:22:20"
Changing minutes and seconds of these dates didn't force this function to work in right way but changing hours, days, months and years then it works fine. So the problem occurs only in combination of these certain dates and hours.
Any idea what could be the cause of such rather mysterious problem?
This is happening because of the switch from Standard Time to Daylight Saving Time. Based on your timezone (GMT+2), the switch probably took place on March 30, 2008 between 3:00 and 4:00 AM, and likewise on March 25, 2012. So anything between 3:00 and 4:00 AM never existed. R thinks it's not possible for those timestamps to exist, and defaults to the day.
I think you can get around this by setting the timezone in your POSIXlt call.
Related
Calling an API, I need to specify time in milliseconds. I use DateTime in Python 3 to convert from and to human readable dates and times. But when debugging, I get different results depending on what website I use to convert, so I'm having a hard time debugging when the timestamps are (maybe) wrong. The API doesn't combine time and date in their predicates, but uses milliseconds for both.
Let's look at the date predicate 1656547200000
If I go to https://currentmillis.com it says it's June 30th. All good.
If I go to https://www.epochconverter.com it says it's June 30th. All good.
Let's look at the time predicate from 12600000 to 26700000.
12600000, 26700000
If I go to https://currentmillis.com it says it's UTC (24h) 03:30 - 07:25
If I go to https://www.epochconverter.com it says it's UTC (24h) 20:00 - 00:40
Why that different results?
Historically, UNIX systems reckon time as temporal units (seconds, millis, nanos, etc.) elapsed since an absolute origin instant termed "the Epoch": January 1st, 1970 00:00:00 UTC. This is formally recognized in POSIX.
So "UNIX time" is that system of reckoning, and "Epoch timestamps" are points in time in that system.
Now, you appear to me to be conflating temporal units in your use of Epoch timestamps.
In the case of your "short" timestamp, 12600000 seconds since the Epoch is a different point in time than 12600000 milliseconds since the Epoch. That's why you see them resolve to different times of day, as your converters are interpreting them differently. If you'd included the date in your output, you'd have seen the two points in time are almost six months apart.
today there is November (DST). I set File date time with :
TFile.SetCreationTime(FilePath, ADateTime);
If I set date time to: '2021-11-11 02:27:00' read in Windows Explorer is as set.
If I set date time to: '2021-05-14 02:27:00' then date is OK but hour read in Windows Explorer is 03:27.
How to set hour to 02:27:00 independently of DST?
The documentation on TFile.SetCreationTime() isn't great. However, since also a method TFile.SetCreationTimeUtc() exists the former will most likely interpret the TDateTime as local time (the currently chosen timezone in an operating account's settings), while the latter will interpret it as system time (UTC = GMT = timezone 0000).
Long story short: keep yourself from trouble and operate on UTC times, which also doesn't know daylight saving. Examples:
Germany is in timezone +0100, with DST +0200 during summer (which means 1 or 2 hours ahead of UTC).
Most of Brazil is in timezone -0300, with DST -0200 during summer up to the recent past (which means 3 or 2 behind UTC).
I have a SQLite 3 database on MacOS with timestamp data (typically something like 279020203.539467).
The documentation says that dates can be stored as
REAL - as Julian day numbers, the number of days since noon in Greenwich
on November 24, 4714 B.C. according to the proleptic Gregorian calendar.
And it looks like this is what I've got.
I want to export this and import it into other databases.
I am assuming the timestamp datatype is not compatible between database engines, so some conversion has to happen somewhere, and I am working under the assumption that it'd be best to do that while exporting the data from the SQLite database.
But I can't figure out how to do this conversion.
I've looked at this answer which refers to this forum post which indicates that in SQLite you could do something like
select datetime('40660.9454658044', '+2415018 days', '+12 hours', 'localtime');
(Maybe 2415018 is the number of days between November 24, 4714 B.C. and some other magical date...)
However, replacing the timestamp string in this example with what I have results in null. Presumably because '279020203.539467' is some other kind of timestamp. It is also some magnitudes larger than the example.
But how to convert this to a usable date? I know it should be around 2011/2012.
Interpreting the data as an "integer" (seconds since 1970-01-01) gives 1978 so that is not correct either.
UPDATE: I've found that
288773834.371606 should be 2010-02-25 07.57
296636121.950064 should be 2010-05-27 08.55
(CET if that matters).
The good news is: To convert a Julian date to "regular" date format you could use datetime(strftime('%J',jtime)). FYI Here's the doc for sqlite date and time functions. But there's bad news.
A NASA Calculator computes the Julian date of 2010-02-25 to be 2455246. It computes the civil date of 288773834 as Sept 2, 785907 A.D. sqlite doesn't give that same result using the above notation, but it doesn't give a "date".
Even though the numbers look like Julian date notation, they are not any dates in our lifetimes.
DATEDIF("2010-02-25 07.57"; "2010-05-27 08.55"; "d")
This gives 91 days which works out to be 7862400 seconds, almost exactly the same as the difference between the two timestamps (296636121.950064 - 288773834.371606 = 7862287.578458).
On MacOS native timestamps are seconds from 2001-01-01 (Jan 1 2001).
And
DATEDIF("2001-01-01 00:00"; "2010-02-25 07.57"; "d")
gives 3342 days which is 288748800 seconds, near, but not exactly the timestamp but that difference is caused by
DATEDIF only caters for whole days
CET is one hour off
Correcting for that we get around 25000 seconds more to add, which takes us almost exactly to the timestamp for 2010-02-25 07.57.
So the gist of this is that SQLite on MacOS stores timestamps as MacOS native timestamps which are seconds since start of MacOS epoch (2001-01-01 00.00). This is probably caused by the application that created the data was not using SQLite timestamps but MacOS native dates and storing them in the database as any other data.
Converting this to some other format should be trivial, either during the export, a conversion of the exported file or during imports.
Possibly it would be easiest to convert the date on export from the original database using
select ...
datetime(table.timestamp_field, 'unixepoch', '+31 years')
from ...
Why 7/18/2013 11:33 is different in GMT timezone and in my local Time Zone (Asia/kolkata)?
As Unix time-stamp are the ticks being calculated since epoch time 1/1/1970 00:00:00 GMT so i know that there the epoch time had occurred at different interval in different timezone but still. the number of second elapsed should have been same
For example if I(+5:30 GMT) and My friend(+5:00 GMT) starts counting the ticks from 00:00 Hrs respectively so at 18:00 Hrs in both timezone number of ticks should be same. So why same thing is not true with the Unix time-stamp.
Need to Understand the concept fully.
For example if I(+5:30 GMT) and My friend(+5:00 GMT) starts counting the ticks from 00:00 Hrs respectively so at 18:00 Hrs in both timezone number of ticks should be same.
No, because both of you start counting from 00:00 UTC. That's the definition. So for you, that will mean the number of ticks since 18:30, and for your friend it will mean the number of ticks since 19:00.
The idea is that a single instant in time has the same timestamp value everywhere. So if I were calling you now (and ignoring phone delays) we could both agree that "now" is a Unix timestamp of 1374130418. You may have a different local time to me, but we can express "now" in a common format.
See the "core concepts" part of the Noda Time user guide for more discussion of local time vs "global" time.
When your system is configured correctly, the unix time stamp shows you the ticks since 1.1.1970 in Greenwich. So when you an your friend read the timestamp at 18:00 at your local time, you do it with 30 minutes delay and therefore have 1800 seconds difference.
If this isn't the case, one or both of the system clocks isn't set correctly to use GMT as base. Usually this kind of problem doesn't have negative impact as long as you're just working on one system or all systems are set incorrectly the same way.
The Unix time is given as seconds since the epoch: the number of seconds (not counting leap seconds) that have passed since 00:00:00 Coordinated Universal Time (UTC), or Thursday, January 1st, 1970,
The GNU date command has some very nice features that allow you to translate between different time formats. These are explained very nicely in man date so I will only give you some examples here:
### "Normal" format
$ date
Thu Jun 12 11:44:23 CEST 2014
### Unix time
$ date +%s
1402566271
To convert, you can give date a specific date using the -d flag. However, to get a Unix date, this needs to be a full date. You can't convert 3:00PM to Unix time since Unix time refers to an entire date (year,month,day,time). So, for example, to get the Unix date for the 12th of September 1987, you would do:
$ date -d "3 PM 12 September 1987" +%s
558450000
And to convert that back to a "normal" date:
$ date -d "#558450000"
Sat Sep 12 15:00:00 CEST 1987
How to represent the start and end times for one day?
Using October 23, 2008 as an example, is it start 2008-10-23 12:00:00 AM and end 2008-10-23 11:59:59 PM?
I would like to cite a site, http://www.cl.cam.ac.uk/~mgk25/iso-time.html
The international standard notation for the time of day is
hh:mm:ss
where hh is the number of complete hours that have passed since midnight (00-24), mm is the number of complete minutes that have passed since the start of the hour (00-59), and ss is the number of complete seconds since the start of the minute (00-60). If the hour value is 24, then the minute and second values must be zero.
Note: The value 60 for ss might sometimes be needed during an inserted leap second in an atomic time scale like Coordinated Universal Time (UTC). A single leap second 23:59:60 is inserted into the UTC time scale every few years as announced by the International Earth Rotation Service in Paris, to keep UTC from wandering away more than 0.9 s from the less constant astronomical time scale UT1, which is defined by the actual rotation of the earth. In practice you are not very likely to see a clock showing 23:59:60. Most synchronized clocks resynchronize again to UTC some time after a leap second has happened, or they temporarily slow down near the time of a leap seconds, to avoid any disruption that an out-of-range timestamp might otherwise cause.
An example time is
23:59:59
which represents the time one second before midnight.
As with the date notation, the separating colons can also be omitted as in
235959
and the precision can be reduced by omitting the seconds or both the seconds and minutes as in
23:59, 2359, or 23
It is also possible to add fractions of a second after a decimal dot or comma, for instance the time 5.8 ms before midnight can be written as
23:59:59.9942 or 235959.9942
As every day both starts and ends with midnight, the two notations 00:00 and 24:00 are available to distinguish the two midnights that can be associated with one date. This means that the following two notations refer to exactly the same point in time:
1995-02-04 24:00 = 1995-02-05 00:00
In case an unambiguous representation of time is required, 00:00 is usually the preferred notation for midnight and not 24:00. Digital clocks display 00:00 and not 24:00.
ISO 8601 does not specify, whether its notations specify a point in time or a time period. This means for example that ISO 8601 does not define whether 09:00 refers to the exact end of the ninth hour of the day or the period from 09:00 to 09:01 or anything else. The users of the standard must somehow agree on the exact interpretation of the time notation if this should be of any concern.
If a date and a time are displayed on the same line, then always write the date in front of the time. If a date and a time value are stored together in a single data field, then ISO 8601 suggests that they should be separated by a latin capital letter T, as in 19951231T235959.
A remark for readers from the U.S.:
The 24h time notation specified here has already been the de-facto standard all over the world in written language for decades. The only exception are a few English speaking countries, where still notations with hours between 1 and 12 and additions like “a.m.” and “p.m.” are in wide use. The common 24h international standard notation is widely used now even in England (e.g. at airports, cinemas, bus/train timetables, etc.). Most other languages do not even have abbreviations like “a.m.” and “p.m.” and the 12h notation is certainly hardly ever used on Continental Europe to write or display a time. Even in the U.S., the military and computer programmers have been using the 24h notation for a long time.
The old English 12h notation has many disadvantages like:
It is longer than the normal 24h notation.
It takes somewhat more time for humans to compare two times in 12h notation.
It is not clear, how 00:00, 12:00 and 24:00 are represented. Even encyclopedias and style manuals contain contradicting descriptions and a common quick fix seems to be to avoid “12:00 a.m./p.m.” altogether and write “noon”, “midnight”, or “12:01 a.m./p.m.” instead, although the word “midnight” still does not distinguish between 00:00 and 24:00 (midnight at the start or end of a given day).
It makes people often believe that the next day starts at the overflow from “12:59 a.m.” to “1:00 a.m.”, which is a common problem not only when people try to program the timer of VCRs shortly after midnight.
It is not easily comparable with a string compare operation.
It is not immediately clear for the unaware, whether the time between “12:00 a.m./p.m.” and “1:00 a.m./p.m.” starts at 00:00 or at 12:00, i.e. the English 12h notation is more difficult to understand.
Please consider the 12h time to be a relic from the dark ages when Roman numerals were used, the number zero had not yet been invented and analog clocks were the only known form of displaying a time. Please avoid using it today, especially in technical applications! Even in the U.S., the widely respected Chicago Manual of Style now recommends using the international standard time notation in publications.
Surely, if you just want to represent 1 day, you don't need to include the time at all - especially as this raises such a level of discussion about when a day starts or ends. In my experience, date handling is usually complicated enough, without introducing any extra complexity.
Today is Oct 20th 2008 - no more information is necessary.
Or am I missing something?
Oct. 23 starts at 2008-10-23 12AM and finishes at 2008-10-24 12AM—a day ends at the exact same point the next one begins. The very last second begins at 11:59:59 PM but you still have a whole second before the day is over.
Your notation is strange. I'd suggest the standard way of specifying a full day range is:
2008-10-23 00:00:00 and finish 2008-10-23 23:59:59
The definition of the day you mention in the question is any time that is >= 2008-10-23 00:00:00 and < 2008-10-24 00:00:00 .
If you were to use mathematical interval notation, you would write it as
[2008-10-23 00:00:00 , 2008-10-24 00:00:00)
The [ means inclusion, and ) means up to but not including.
In this way you make it easier for users and for yourself.
It depends what you mean by a day. Date handling is, sadly, always more complex than it looks.
To convert your example into 24h ISO date format, you'd say:
2008-10-23 00:00:00 - 2008-10-23 23:59:59
However depending on whether you interpret a range as inclusive or exclusive that could omit the last second of the day*. Programmers normally prefer to keep the upper bound exclusive, so you'd go for:
2008-10-23 00:00:00 - 2008-10-24 00:00:00
Or, as a slightly nicer way of stating the same, if supported (or for human readability):
2008-10-23 00:00:00 - 2008-10-23 24:00:00
*: actually possibly more than one. If the span represents a day in the UTC timezone (or another timezone aligned to it, which is quite likely), there can occasionally be an extra 'leap second' at the end of the day, 23:59:60, in a pointless attempt to keep UTC in line with sidereal time. The next leap second is planned for the end of this year. However, many systems ignore leap seconds because they're silly and annoying.
If you want to represent a 'calendar day' independently of timezones your best bet is the Julian day number. Today is 2454760.