I am curious what would be the best way to handle an ambiguous date string in any given language. When pre-validating your user input isn't an option, how should MM/dd/YYYY dates be parsed?
How would you parse the following ambiguous date and for what reason (statistical, cultural, etc)?
'1111900' as Jan 11, 1900 [M/dd/YYYY] or Nov 1, 1900 [MM/d/YYYY]?
Unless you know exactly what the language/culture the format is coming from, you need to establish a common date format.
There is something called locale-neutral date format that I would recommend. (YYYY-MM-DD)
It's either use that or be clear as to what part is the year, month and day. (DD MON YYYY or 22 Apr 2003)
See: the w3's view on date formatting.
Edit: mistyped the locale-neutral date format
Depending on how important the software is, I would treat any ambiguous date entry as invalid input. You should be ensuring (at the source) that the date input you get is in a sensible, non-ambiguous format. If you still manage to get something like "1111900" then the input is not correct, someone has obviously bypassed the validity checking code somehow, and probably the most correct thing you can do is to discard the data.
Of course, if this isn't an option and getting the date spot on isn't critical, you could always guess - but it will be a guess. I would definitely avoid this if possible though. Accepting unsanitised input is not the best idea in general.
The only way to know the difference between Jan 11 and Nov 1 in such a system would be through context. Otherwise, you need to go through some sort of disambiguation. That particular date format would be a perfect example of pathologically destructive compression.
My preference when the date in important is use offer drop-downs or a calendar that way it always comes in the expected format.
Related
I have a simple system, which on node creation form, allows user to select start and end dates along with hours and minutes:
The problem is that the date gets messed up somewhere in the middle, or I don't know how to convert it.
In the database, two values gets saved as follows:
start: "2019-08-01T14:00:00.000Z"
end: "2019-08-01T16:00:00.000Z"
Seems correct, as in the datepicker I chose 2pm and 4pm. Then, using momentjs and react-big-calendar, I try to put that event into calendar. It seems that it puts +3 hours every time. I assume, it is because it saves it in UTC format, and I live in Eastern Europe, which is +3 hours from UTC.
What is strange, is that I already get back time converted to my own when I make a request to the database:
Thu Aug 01 2019 17:00:00 GMT+0300 (Eastern European Summer Time)
Could someone help me out? I would expect to have the same hours returned and displayed, as I save them, meaning if I select 2pm, it should be 2pm. I think the solution here would be to ignore timezones, and always display everything in UTC? But how to do that? I have access to momentjs if that helps. I tried something like this:
moment.utc(event.start);
But it still returns the same value, which is:
Thu Aug 01 2019 17:00:00 GMT+0300 (Eastern European Summer Time)
When you setup React-Big-Calendar, you define a localizer. By default, the calendar will display in the local timezone of the browser the user is using. Any and all dates given to the calendar will be converted to that local timezone. If your dates, coming from the db, use an offset of +5, but you're sitting in +3, it will automatically convert those times to display +3 (part of the beauty of UTC).
This is where it can get tricky. Say I want to get all values from my db between Sep 19th and Sep 21st. I have to remember those offsets when requesting data from the system. I could use moment and say moment('2019-09-19').startOf('day'), and it would give me 2019-09-19T00:00:00Z, but this isn't exactly correct for me, as I'm at an offset of -5, which means I actually needed 2019-09-18T19:00:00Z to get the start of my day. The same holds true for my end datetime, where I would say moment('2019-09-21').endOf('day') and it would give me 2019-09-21T23:59:59Z, when I would need 2019-09-21T18:59:59Z.
Now, I'm not positive of this (you'll have to do some testing), but if you use moment-timezone instead of moment it may handle all of this for you. You'll have to play around to know for sure.
FINAL EDIT: I worked out timezone handling with moment-timezone as my localizer. I put the entire thing in a GitHub project, as well as creating a CodeSandbox where people can play with it.
TRULY FINAL EDIT: The (0.36.0) latest version of RBC has full timezone support now, when using either the momentLocalizer or the new luxonLocalizer. Doc site isn't up to date yet, but if you run the 'examples' locally you'll find all the info.
Ive noticed a change in the way R presents some timezone attributes. For example:
date/time objects with the timezone "America/New_York" present the time zone attribute "EST" (consistent with the format code %Z). This has always been the case and is still the case
other timezones, such as "Asia/Qatar" no longer present the familiar attribute "AST" (for Arabic standard time) as seen previously, but now present "+03".
Why is that and what has changed? When I try to format date/time objects to use the %Z format, I still get "+03" not "AST".
This line shows the issue in practice:
> format(as.POSIXct(Sys.time()), format = "%Z", tz = "Asia/Qatar")
[1] "+03"
This might seem trivial, but legacy code that expects "AST" will no longer work. Any feedback is appreciated.
The source for time zone data in R, and most other computing platforms, is the IANA time zone database. The maintainers of the this data started in 2016, and continuing into 2017, meticulously going through every time zone abbreviation, removing any that they deem "invented", and replacing it with a fixed numeric offset.
By "invented", understand that at earlier times in its history, it was acceptable for the database maintainers to just make up something that seemed reasonable for a time zone abbreviation when there wasn't one naturally used in the real world. This practice has now stopped, and is being reverted, because it was recognized that the database is there to record time zone information from the world, not invent it.
In other words, while the Asia/Qatar time zone entry previously had "GST" as an abbreviation for times before 1972, and "AST" as an abbreviation for times from there to present, neither abbreviation is actually used in Qatar. At least, not to the best knowledge of the database maintainers.
That particular change was made in tzdb 2017a. Announcement here.
We have a really strange problem in xpages regarding dates, the Notesdocument we have contain a date field and the value is only a date, there is no time portion.
In the xpage I have specified to display the date as a date/time value. the date display correctly on the webpage but we are now getting reports from users who login at night (around midnight) and see the date as adjusted by one day. if the same people login at daytime the date is correct so this only seem to happend around midnight
I have tried to change my clock on my client to around midnight but that does not reproduce it so I assume this is a server issue.
The domino server have correct date/time and we are using the latest version of Domino
any ideas?
we encountered the same problem recently and, I believe, found a very nice solution.
system treats the pure date as a date in UTC time zone. Date value is automatically converted into server's time zone. So the question is how to prevent conversion?
this code prevents conversion:
<xp:this.converter>
<xp:convertDateTime
type="date"
ignoreUserTimeZone="true"
dateStyle="long"
timeZone="UTC">
</xp:convertDateTime>
</xp:this.converter>
pay attention at "timeZone" attribute.
The issue is related to not having the TimeZone specified in the date / time field. We ran into this just yesterday. If you don't have the TZ specified, it seems to assume UTC and will adjust accordingly. Include the time zone and your field will stop adjusting erroneously.
Perhaps it has to do with this ?
http://www-304.ibm.com/support/docview.wss?uid=swg21508734
I've seen some reports about XPages Dates and TimeZone Issues
I suspect your Domino version was 8.5.3, because there were 2 APAR, LO72278 and LO67745, on similar problems against 8.5.3. Fixpack 3 addresses them.
The root cause is that Lotus Notes allows you to save a "Date" with no time or zone and the Notes server has a default Time Zone setting to interpret these incomplete things called dates. XPages doesn't play by the same rules, and its master Java wants to know what zone you're using, and looks to the system for some clue, and generally will use midnight within some TZ to refer to a "Date". There is a whole region on Stack Overflow on the "how to store/represent a date" topic - [datetime] - since languages and DBMS each have their own approach.
Nice legacy Notes focused article about it.
http://www-10.lotus.com/ldd/ddwiki.nsf/dx/05022009100728PMAGU5MB.htm
XPages article about it
http://www-10.lotus.com/ldd/ddwiki.nsf/dx/XPagesTimeZones.htm
Too bad they are not totally on the same page, I assume each release gets closer.
What's the recommended timestamp format for a REST GET API like this:
http://api.example.com/start_date/{timestamp}
I think the actual date format should be ISO 8601 format, such as YYYY-MM-DDThh:mm:ssZ for UTC time.
Should we use the ISO 8601 version without hyphens and colons, such as:
http://api.example.com/start_date/YYYYMMDDThhmmssZ
or should we encode the ISO 8601 format, using for example base64 encoding?
Check this article for the 5 laws of API dates and times HERE:
Law #1: Use ISO-8601 for your dates
Law #2: Accept any timezone
Law #3: Store it in UTC
Law #4: Return it in UTC
Law #5: Don’t use time if you don’t need it
More info in the docs.
REST doesn't have a recommended date format. Really it boils down to what works best for your end user and your system. Personally, I would want to stick to a standard like you have for ISO 8601 (url encoded).
If not having ugly URI is a concern (e.g. not including the url encoded version of :, -, in you URI) and (human) addressability is not as important, you could also consider epoch time (e.g. http://example.com/start/1331162374). The URL looks a little cleaner, but you certainly lose readability.
The /2012/03/07 is another format you see a lot. You could expand upon that I suppose. If you go this route, just make sure you're either always in GMT time (and make that clear in your documentation) or you might also want to include some sort of timezone indicator.
Ultimately it boils down to what works for your API and your end user. Your API should work for you, not you for it ;-).
Every datetime field in input/output needs to be in UNIX/epoch format. This avoids the confusion between developers across different sides of the API.
Pros:
Epoch format does not have a timezone.
Epoch has a single format (Unix time is a single signed number).
Epoch time is not effected by daylight saving.
Most of the Backend frameworks and all native ios/android APIs support epoch conversion.
Local time conversion part can be done entirely in application side depends on the timezone setting of user's device/browser.
Cons:
Extra processing for converting to UTC for storing in UTC format in the database.
Readability of input/output.
Readability of GET URLs.
Notes:
Timezones are a presentation-layer problem! Most of your code shouldn't be dealing with timezones or local time, it should be passing Unix time around.
If you want to store a humanly-readable time (e.g. logs), consider storing it along with Unix time, not instead of Unix time.
RFC6690 - Constrained RESTful Environments (CoRE) Link Format Does not explicitly state what Date format should be however in section 2. Link Format it points to RFC 3986. This implies that recommendation for date type in RFC 3986 should be used.
Basically RFC 3339 Date and Time on the Internet is the document to look at that says:
date and time format for use in Internet protocols that is a
profile of the ISO 8601 standard for representation of dates and
times using the Gregorian calendar.
what this boils down to : YYYY-MM-ddTHH:mm:ss.ss±hh:mm
(e.g 1937-01-01T12:00:27.87+00:20)
Is the safest bet.
Always use UTC:
For example I have a schedule component that takes in one parameter DATETIME.
When I call this using a GET verb I use the following format where my incoming parameter name is scheduleDate.
Example:
https://localhost/api/getScheduleForDate?scheduleDate=2003-11-21T01:11:11Z
For JavaScript,
I normally save date as ISO format by writing: new Date().toISOString();.
Hopefully, it helps with your project.
GWT doesn't serialize Java Date properly. When I tried sending Date created in Javascript through the wire, I found out that dates between April 1st (funny) and 25th October for years before year 1983 get subtracted by one day.
That means that, say, both 1982-04-01 and 1982-03-31 become 1982-03-31 on the Java side.
Given the dates in question, I would guess that this is some kind of DST problem. I've tried googling, and found only one other reference that describes similar problem.
I also tried submitting bug to the GWT team, but curiously wasn't able to find bugtracker for GWT.
So, my questions are:
Anyone else run into this? I'm on GWT 1.7, and would like to confirm if this happens on 2.0 as well.
My workaround was to send dates as Strings, and parse them on server. Anyone knows better workaround?
Assuming that you are using a java.util.Date
Question 1: It seems that it is fixed in 2.0. I've created both Dates above (1982-04-01 and 1982-03-31) and they come through correctly to the server (both represent on the server as 1982-04-01 and 1982-03-31 respectively). My setup is:
GWT 2.0
Java 1.6
OSX 10.6.2
Question 2: You could always pass the 'milliseconds since January 1, 1970, 00:00:00 GMT' over the async service-which you can get using getTime() on the date object. On the server side you can then instantiate a new Date passing this value in on the constructor:
Date date = new Date(millis);
This saves fiddling around with formatters and parsers.
Dates and times are a complicated subject. The conversion can depend on the locale that the browser is running in and wether both you JVM on the server and the locales of the clients are up-to-date.
In some cases there can be differences because in some regions they switched timezones in the past.
GWT sends dates using just millis since epoch time. Since it is using Date objects, you are limited in that the Dates on the server side will be automatically modified to the servers timezone. Are you sure that you take into account the possible difference in timezones between client and server ?
David
I'm pretty certain the FTR library has some date emulation in it.
http://code.google.com/p/ftr-gwt-library
If you don't have to do client-side conversions (adapt to user's timezone) or calculations send it in a String from the server.
Never came across your specific problem though.
The possible problems soure is difference in Client/Server time zones.
We have also run into a similar problem. It was long enough ago that I do not remember the exact details but the gist of it was there were some minor differences between java.util.Date and the way dates were handled in Javascript. Our workaround was effectively the same as yours, where the actual value sent over the wire was generally a String.