Ive noticed a change in the way R presents some timezone attributes. For example:
date/time objects with the timezone "America/New_York" present the time zone attribute "EST" (consistent with the format code %Z). This has always been the case and is still the case
other timezones, such as "Asia/Qatar" no longer present the familiar attribute "AST" (for Arabic standard time) as seen previously, but now present "+03".
Why is that and what has changed? When I try to format date/time objects to use the %Z format, I still get "+03" not "AST".
This line shows the issue in practice:
> format(as.POSIXct(Sys.time()), format = "%Z", tz = "Asia/Qatar")
[1] "+03"
This might seem trivial, but legacy code that expects "AST" will no longer work. Any feedback is appreciated.
The source for time zone data in R, and most other computing platforms, is the IANA time zone database. The maintainers of the this data started in 2016, and continuing into 2017, meticulously going through every time zone abbreviation, removing any that they deem "invented", and replacing it with a fixed numeric offset.
By "invented", understand that at earlier times in its history, it was acceptable for the database maintainers to just make up something that seemed reasonable for a time zone abbreviation when there wasn't one naturally used in the real world. This practice has now stopped, and is being reverted, because it was recognized that the database is there to record time zone information from the world, not invent it.
In other words, while the Asia/Qatar time zone entry previously had "GST" as an abbreviation for times before 1972, and "AST" as an abbreviation for times from there to present, neither abbreviation is actually used in Qatar. At least, not to the best knowledge of the database maintainers.
That particular change was made in tzdb 2017a. Announcement here.
Related
I always had conceptual troubles with date values being represented as objects without time (SQL for instance has the type DATE beside DATETIME). The same thinking often creates issues in languages that always store dates along with a time (e.g. C#'s DateTime). Suddenly timezone bugs pop up and the developers insist that all that timezone hullabaloo can be ignored, because their DateTime is actually "just a date". But what does that even mean?
The way I see it, such a "plain date" can be interpreted in only two ways:
From the perspective of points in time (i.e. "instants") one might infer that "2019-09-17" means the start (end) of the day, which is "2019-09-17 00:00:00" ("2019-09-17 23:59:59"). Hence timezone information is relevant and we are not talking about "just a date".
The other option is that the date is supposed to mean the whole day, so we are actually talking about a timespan from "2019-09-17 00:00:00" to "2019-09-17 23:59:59" and timezone information cannot be ignored either.
I argue that all use cases for "plain dates" fall in either of the two categories and that time/timezone can only be ignored if the software will only run on and communicate with systems configured with the exact same timezone.
Can someone provide a counter example or a third interpretation?
The "holiday" is an example of a date without a time, and even without a time zone.
Most holidays are religious in nature, and I don't want to start a religious war, or get sidetracked by minutiae. So as an example holiday, I'm going to pick Earth Day: April 22 of each year in the civil calendar.
If you write software that deals with holidays, said software isn't necessarily dealing with an instant in time across the entire planet (though it could do that in addition to tracking holidays). Instead it carries the concept that maps April 22 of each year to Earth Day. And it is irrelevant that some people will celebrate Earth Day hours before or after other people.
If a client wants to know exactly when Earth Day starts/ends for him, then he can translate "the date" into specific instants of time using his particular time zone. The need for that translation does not diminish the value of the date (say 2020-04-22).
C++20 will have several types to represent a "timezone-less date" such as 2020-04-22 using different data structures:
local_days // day 18374 == April/22/2020
year_month_day // year 2020, month 4, day 22 == April/22/2020
year_month_weekday // year 2020, month 4, 4th Wed == April/22/2020
I have a date field at front end. And I am saving it from a time zone say 19/04/2018 and I am on +8. When I load it on local datetime.ToLocalTime() works perfectly in +8 offset and it will show 19/04/2018 but a person sitting in +7 would get in 18/04/2018 23:00 and hence it will show 18/04/2018. How to handle this case.
In general: you should not convert date only to UTC, so you should keep it as local date.
But it really depend on the use of data.
If you are using for some synchronization (all invoices until a fix point), a time with timezone is good, but in such case, also hours and minutes should be included.
If you care about the date written in an invoice, you may not transform date to UTC (so keep local date). If you aggregate invoices, you may wait until the day is passed on all timezone (and people filled all invoices into the system).
When people are looking into the data, what they expect? (case #1 or case #2). Then evaluate what kind of data you need. In the first case, you should add also (at minimum) the hours and minute (timezones never have seconds).
In general, if you have only a date, you are mostly on second case.
I have a simple mobile app that schedules future events between people at a specified location. These events may be physical or virtual, so the time specified for the event may or may not be in the same timezone as the 'location' of the event. For example, a physical meeting may be scheduled for two people in London at local time 10am on a specified date. Alternatively, a Skype call may be scheduled for two people in different timezones at 4pm (in one person's timezone) on a specified date though the 'location' of the event is simply 'office' which means two different places in different timezones.
I wonder the following design is going to work for this application:
On the client, it asks user to input the local date and time and specify the timezone local to the event.
On the server, it converts the local date and time with the provided timezone into UTC timestamp, and store this timestamp only.
When a client retrieves these details, it receives the UTC timestamp only and converts it into local time in the same timezone as the client's current timezone. The client's current timezone is determined by the current system timezone setting, which I think is automatically adjusted based on the client's location (of course, assuming the client is connected to a mobile network).
My main motivations for this design are:
UTC is an absolute and universal time standard, and you can convert to/from it from/to any timezone.
Users only care about the local date and time in the timezone they are currently in.
Is this a feasible design? If not, what specific scenarios would break the application or severely affect user experience? Critiques welcome.
For a single event, knowing the UTC instant on which it occurs is usually enough, so long as you have the right UTC instant (see later).
For repeated events, you need to know the time zone in which it repeats... not just the UTC offset, but the actual time zone. For example, if I schedule a weekly meeting at 5pm in Europe/London with colleagues in America/Los_Angeles, then for most of the year it will occur at 9am for them... but for a couple of weeks in the year it will occur at 8am and for a couple of weeks in the year it will occur at 10am, due to differences in when DST is observed.
Even for a single event, you might want to consider what happens if time zone rules change. Suppose I schedule a meeting for 4pm on March 20th 2018, in the Europe/London time zone. Currently that will occur with a UTC offset of 0... but suppose between now and the meeting, the time zone rules change to bring British Summer Time in one hour earlier. If I've written it in my diary as 4pm, I probably don't want the software to think that it's actually at 5pm because that's the UTC instant we originally predicted.
We don't know your exact application requirements, but the above situations at least provide an argument for potentially storing the local time and time zone instead of the UTC instant... but you'll also need to work out what to do if the local time ends up being skipped or being ambiguous due to DST changes. (When the clocks fall back, some local times occur twice. When the clocks skip forward, some local times are skipped. A time that was unambiguous may become invalid or ambiguous if the rules change between the original planning time and the actual event. You should probably account for this in your design.)
To keep it simple, my answers are:
Timezone info is redundant if you want to define a single moment. A
UTC/Unix timestamp completely defines a moment.
Your design seems feasible but on point 2: i would convert to the UTC/Unix timestamp on the client-side and already give this timestamp
in its final form to the server. Reason: the client-side already has the info necessary to convert (see this time-keeping
client-server-db
architecture
example - it works based exactly on the principles you describe).
One possible problem (as described by Jon Skeet in his answer) are recurring events, but this should be reflected in the way you model
time. The difference between recurring events and fixed events is
that the latter completely define a moment (like a UTC/Unix
timestamp) while the first are only a 'function' which can be applied
to the current time to get the next trigger time of the recurring
event. But this might entirely be a different problem than what
you ask - in any case, somehow distinguishing between recurring
events (if you need them) and fixed events in your model is a good
idea.
One decision to make is: PULL or PUSH? Or both? Do you want the server to be able to send emails for example, when an event comes to
pass? Or do you want client-side alerts only when your client-side
app is running? The answers to these questions will help you come
towards a design suitable for you.
We have a really strange problem in xpages regarding dates, the Notesdocument we have contain a date field and the value is only a date, there is no time portion.
In the xpage I have specified to display the date as a date/time value. the date display correctly on the webpage but we are now getting reports from users who login at night (around midnight) and see the date as adjusted by one day. if the same people login at daytime the date is correct so this only seem to happend around midnight
I have tried to change my clock on my client to around midnight but that does not reproduce it so I assume this is a server issue.
The domino server have correct date/time and we are using the latest version of Domino
any ideas?
we encountered the same problem recently and, I believe, found a very nice solution.
system treats the pure date as a date in UTC time zone. Date value is automatically converted into server's time zone. So the question is how to prevent conversion?
this code prevents conversion:
<xp:this.converter>
<xp:convertDateTime
type="date"
ignoreUserTimeZone="true"
dateStyle="long"
timeZone="UTC">
</xp:convertDateTime>
</xp:this.converter>
pay attention at "timeZone" attribute.
The issue is related to not having the TimeZone specified in the date / time field. We ran into this just yesterday. If you don't have the TZ specified, it seems to assume UTC and will adjust accordingly. Include the time zone and your field will stop adjusting erroneously.
Perhaps it has to do with this ?
http://www-304.ibm.com/support/docview.wss?uid=swg21508734
I've seen some reports about XPages Dates and TimeZone Issues
I suspect your Domino version was 8.5.3, because there were 2 APAR, LO72278 and LO67745, on similar problems against 8.5.3. Fixpack 3 addresses them.
The root cause is that Lotus Notes allows you to save a "Date" with no time or zone and the Notes server has a default Time Zone setting to interpret these incomplete things called dates. XPages doesn't play by the same rules, and its master Java wants to know what zone you're using, and looks to the system for some clue, and generally will use midnight within some TZ to refer to a "Date". There is a whole region on Stack Overflow on the "how to store/represent a date" topic - [datetime] - since languages and DBMS each have their own approach.
Nice legacy Notes focused article about it.
http://www-10.lotus.com/ldd/ddwiki.nsf/dx/05022009100728PMAGU5MB.htm
XPages article about it
http://www-10.lotus.com/ldd/ddwiki.nsf/dx/XPagesTimeZones.htm
Too bad they are not totally on the same page, I assume each release gets closer.
What's the recommended timestamp format for a REST GET API like this:
http://api.example.com/start_date/{timestamp}
I think the actual date format should be ISO 8601 format, such as YYYY-MM-DDThh:mm:ssZ for UTC time.
Should we use the ISO 8601 version without hyphens and colons, such as:
http://api.example.com/start_date/YYYYMMDDThhmmssZ
or should we encode the ISO 8601 format, using for example base64 encoding?
Check this article for the 5 laws of API dates and times HERE:
Law #1: Use ISO-8601 for your dates
Law #2: Accept any timezone
Law #3: Store it in UTC
Law #4: Return it in UTC
Law #5: Don’t use time if you don’t need it
More info in the docs.
REST doesn't have a recommended date format. Really it boils down to what works best for your end user and your system. Personally, I would want to stick to a standard like you have for ISO 8601 (url encoded).
If not having ugly URI is a concern (e.g. not including the url encoded version of :, -, in you URI) and (human) addressability is not as important, you could also consider epoch time (e.g. http://example.com/start/1331162374). The URL looks a little cleaner, but you certainly lose readability.
The /2012/03/07 is another format you see a lot. You could expand upon that I suppose. If you go this route, just make sure you're either always in GMT time (and make that clear in your documentation) or you might also want to include some sort of timezone indicator.
Ultimately it boils down to what works for your API and your end user. Your API should work for you, not you for it ;-).
Every datetime field in input/output needs to be in UNIX/epoch format. This avoids the confusion between developers across different sides of the API.
Pros:
Epoch format does not have a timezone.
Epoch has a single format (Unix time is a single signed number).
Epoch time is not effected by daylight saving.
Most of the Backend frameworks and all native ios/android APIs support epoch conversion.
Local time conversion part can be done entirely in application side depends on the timezone setting of user's device/browser.
Cons:
Extra processing for converting to UTC for storing in UTC format in the database.
Readability of input/output.
Readability of GET URLs.
Notes:
Timezones are a presentation-layer problem! Most of your code shouldn't be dealing with timezones or local time, it should be passing Unix time around.
If you want to store a humanly-readable time (e.g. logs), consider storing it along with Unix time, not instead of Unix time.
RFC6690 - Constrained RESTful Environments (CoRE) Link Format Does not explicitly state what Date format should be however in section 2. Link Format it points to RFC 3986. This implies that recommendation for date type in RFC 3986 should be used.
Basically RFC 3339 Date and Time on the Internet is the document to look at that says:
date and time format for use in Internet protocols that is a
profile of the ISO 8601 standard for representation of dates and
times using the Gregorian calendar.
what this boils down to : YYYY-MM-ddTHH:mm:ss.ss±hh:mm
(e.g 1937-01-01T12:00:27.87+00:20)
Is the safest bet.
Always use UTC:
For example I have a schedule component that takes in one parameter DATETIME.
When I call this using a GET verb I use the following format where my incoming parameter name is scheduleDate.
Example:
https://localhost/api/getScheduleForDate?scheduleDate=2003-11-21T01:11:11Z
For JavaScript,
I normally save date as ISO format by writing: new Date().toISOString();.
Hopefully, it helps with your project.