I imagined there would be more literature on this, but I'm having trouble finding any. I have a lot of non-algebraically-aggregatable time series data (that is to say, points for which no function exists that I could use to aggregate them to a higher granularity-- stuff like unique active users, unique contributors, etc... where knowing the amount I had every minute of some hour does not tell me how many I had total during the hour). Currently, I'm just storing and presenting all of this data in UTC. The problem is that many of my clients find this confusing-- understandably so. Because the data is non-algebraically-aggregatable, there's no way to get from UTC data for 1 day midnight- midnight to, say, PST data from midnight to midnight. Recalculation would need to be done from raw data.
So:
Recalculation from raw data is prohibitively expensive for some complicated analytics graphs
We could store all data for all time zones, but this would increase the amount of data we store x24.
All of that said, how do other people deal with this issue? Here's how Google Analytics does it, but this seems insufficient for my use case because I know if I open the multiple timezone can of worms, clients will ask for more than one. This will also take a lot of work that doesn't seem worth the effort as just adding timezone support won't be extremely noticeable or a huge win. What I'm really hoping for is some clever design solution that just presents the UTC data in some intuitive enough way that it's no longer confusing for people in other timezones. Has anyone dealt with similar problems and come upon a solution I'm missing?
First of all, you should recognize that there a lot more than 24 time zones. In order to accurately take into account how people actually use time worldwide, you should be using IANA time zones, of which there are over 500. See also Wikipedia and the timezone tag wiki.
If you are dealing with individual points (discreet timestamps), then you can certainly convert from UTC to any time zone you wish, on the fly as you render your graph. You just need to also keep in mind that the range of data you query will also need to be translated to that time zone.
But if you are talking about aggregating data by the "day" of a specific time zone, then there is no magic bullet. You will need to decide ahead of time which time zones you want to support and calculate each one separately. When you do this, recognize that it's not just the view that's changing. Since the day boundaries are different for each time zone, then the data for each time zone could potentially have very different daily totals.
You should also be aware that not every day has 24 hours. If the day happens to be the date of a daylight saving time transition, it could have 23, 23.5, 24.5, or 25 hours. This could potentially affect how you draw your graph.
One approach you might consider is to be time zone ignorant in your aggregations, rather than using UTC or any specific time zone. Of course this depends heavily on the context of your data, but it is appropriate in certain circumstances. For example, on an invoice, you might care less about the specific timestamps, and more about which calendar date the invoice was assigned to. In that case, once a date is assigned, you would just aggregate on that date. Even if the company operates over multiple time zones, you wouldn't care about that in aggregate.
As far as some clever design that abstracts this from the user, I'm afraid I haven't seen much. The only two choices you really have are timezone-adjusted aggregations (UTC or otherwise), and time zone ignorant aggregations for calendar-date contexts.
We had similar issues to roll up the data for Generation in renewable. We went with three options User / Farm / UTC.
If user selects USER then all the data would be based on his browser Time zone. And Yesterday meant 24 hours till last mid night in user local time.
Similarly if it was Farm, then we take the Farm local and derive the same.
UTC is standard similar to what you have implemented.
Related
I have a date field at front end. And I am saving it from a time zone say 19/04/2018 and I am on +8. When I load it on local datetime.ToLocalTime() works perfectly in +8 offset and it will show 19/04/2018 but a person sitting in +7 would get in 18/04/2018 23:00 and hence it will show 18/04/2018. How to handle this case.
In general: you should not convert date only to UTC, so you should keep it as local date.
But it really depend on the use of data.
If you are using for some synchronization (all invoices until a fix point), a time with timezone is good, but in such case, also hours and minutes should be included.
If you care about the date written in an invoice, you may not transform date to UTC (so keep local date). If you aggregate invoices, you may wait until the day is passed on all timezone (and people filled all invoices into the system).
When people are looking into the data, what they expect? (case #1 or case #2). Then evaluate what kind of data you need. In the first case, you should add also (at minimum) the hours and minute (timezones never have seconds).
In general, if you have only a date, you are mostly on second case.
I have a simple mobile app that schedules future events between people at a specified location. These events may be physical or virtual, so the time specified for the event may or may not be in the same timezone as the 'location' of the event. For example, a physical meeting may be scheduled for two people in London at local time 10am on a specified date. Alternatively, a Skype call may be scheduled for two people in different timezones at 4pm (in one person's timezone) on a specified date though the 'location' of the event is simply 'office' which means two different places in different timezones.
I wonder the following design is going to work for this application:
On the client, it asks user to input the local date and time and specify the timezone local to the event.
On the server, it converts the local date and time with the provided timezone into UTC timestamp, and store this timestamp only.
When a client retrieves these details, it receives the UTC timestamp only and converts it into local time in the same timezone as the client's current timezone. The client's current timezone is determined by the current system timezone setting, which I think is automatically adjusted based on the client's location (of course, assuming the client is connected to a mobile network).
My main motivations for this design are:
UTC is an absolute and universal time standard, and you can convert to/from it from/to any timezone.
Users only care about the local date and time in the timezone they are currently in.
Is this a feasible design? If not, what specific scenarios would break the application or severely affect user experience? Critiques welcome.
For a single event, knowing the UTC instant on which it occurs is usually enough, so long as you have the right UTC instant (see later).
For repeated events, you need to know the time zone in which it repeats... not just the UTC offset, but the actual time zone. For example, if I schedule a weekly meeting at 5pm in Europe/London with colleagues in America/Los_Angeles, then for most of the year it will occur at 9am for them... but for a couple of weeks in the year it will occur at 8am and for a couple of weeks in the year it will occur at 10am, due to differences in when DST is observed.
Even for a single event, you might want to consider what happens if time zone rules change. Suppose I schedule a meeting for 4pm on March 20th 2018, in the Europe/London time zone. Currently that will occur with a UTC offset of 0... but suppose between now and the meeting, the time zone rules change to bring British Summer Time in one hour earlier. If I've written it in my diary as 4pm, I probably don't want the software to think that it's actually at 5pm because that's the UTC instant we originally predicted.
We don't know your exact application requirements, but the above situations at least provide an argument for potentially storing the local time and time zone instead of the UTC instant... but you'll also need to work out what to do if the local time ends up being skipped or being ambiguous due to DST changes. (When the clocks fall back, some local times occur twice. When the clocks skip forward, some local times are skipped. A time that was unambiguous may become invalid or ambiguous if the rules change between the original planning time and the actual event. You should probably account for this in your design.)
To keep it simple, my answers are:
Timezone info is redundant if you want to define a single moment. A
UTC/Unix timestamp completely defines a moment.
Your design seems feasible but on point 2: i would convert to the UTC/Unix timestamp on the client-side and already give this timestamp
in its final form to the server. Reason: the client-side already has the info necessary to convert (see this time-keeping
client-server-db
architecture
example - it works based exactly on the principles you describe).
One possible problem (as described by Jon Skeet in his answer) are recurring events, but this should be reflected in the way you model
time. The difference between recurring events and fixed events is
that the latter completely define a moment (like a UTC/Unix
timestamp) while the first are only a 'function' which can be applied
to the current time to get the next trigger time of the recurring
event. But this might entirely be a different problem than what
you ask - in any case, somehow distinguishing between recurring
events (if you need them) and fixed events in your model is a good
idea.
One decision to make is: PULL or PUSH? Or both? Do you want the server to be able to send emails for example, when an event comes to
pass? Or do you want client-side alerts only when your client-side
app is running? The answers to these questions will help you come
towards a design suitable for you.
We have a website that has a large number of events that have dates and times created by admins. Admins choose a time zone for each date time entered, and they are stored in UTC time. We are trying to support a global audience, and be completely localized in terms of dates.
We have a search page, that allows dates to be entered as search criteria.
So users could say, show me all events between "12:01 AM July-1-2011" and "11:59 PM July 10-2011".
I'm trying to figure out what the best approach is to determining what time zone to consider the date filter criteria in.
Force end users to select a time zone when creating a date filters. This is cumbersome, and our designers our pushing back. It is what I would prefer.
Assume the the entered dates are in the users "preferred" time zone, which is set upon logging in.
Store times in Local time, without converting to UTC. This way the end users are searching in the admin created date. I hate this idea, i need help explaining why this is bad.
Please help!
Second option is possible solution to your problem. And it is probably the best.
Possibly you could get current time zone offset from web browser (with JavaScript) but the problem is, there are certain time zones that currently have the same offset but Daylight Saving Time switches on different dates, therefore search result would be inaccurate. By having User to choose his/her prefer time zone and storing that information in the profile, you could always present correct dates and times, as well as use this information for searching. However, I would add an information near search box, so that end User would know what time zone this refers to (with JavaScript that would be obvious: the current one, with profile User might forgot).
BTW. Time zone information is best to show as "UTC+02:00 (Warsaw, Zagreb, Skopje)" instead "Central European Time"...
As for other options:
1. Too much clicking. As well as "don't make me think, I want to have it in my local time zone, isn't that obvious?".
3. Local times will not be comparable against each other. You will soon end up with two different dates referring to the same point in time (at least in terms of the numbers). Really bad idea.
I'm writing a geotagging app and running into headaches with timezones. Basically, my app has the following data:
Images with local timestamps (i.e. relative to a timezone)
GPS track files consisting of entries using UTC timestamps
My problem: I need a way to get all data that belongs to a give day, based on the timezone where the data was acquired. For the images, that is easy (I ask the timezone from the user upon import and save it in the EXIF data), but I'm not sure how to do it for the GPS tracks (there usually are multiple tracks per day, and assigning them timezones is not easy for the user when importing data that spans sever days and timezones). I can think of two possible solutions:
Use a heuristic based on the fact that the tracks are recorded at the same time and place as the images - but there can be tracks before a day's first image or after its last one that still need to be included - I'm not sure how to realiably handle such edge cases
Determine the timezone from the GPS coordinates - this would be an ideal solution, but is there an open source library that does this (ideally one that works offline)?
The heuristic method I don't think will work well.
Firstly always store times as UTC and timezone of origin, otherwise time is less meaningful.
After some thought I think that it would be sufficient to resolve down to the country code and from that lookup the timezone.
Depending on how much detail you want I think GeoTog may help you to locate a city and therefore a country from a lat/long (although it will need changing to work the other way).
If not that then Gisgraphy will work with the larger GeoNames database. You could use the web service or extract the data.
If none of these are good enough then I think you'll need to get a some GIS data, possibly boundaries from VMAP0 and process it into polygons or something searchable.
Option two: you could start by checking this site: http://www.twinsun.com/tz/tz-link.htm
Option one (less complicated, but I am not sure I accurately understand your need...)
So you have as input:
A target day defined in a known timezone TZ, starting at t0 and ending at t1 (excluded)
Images with timestamps ti in the same timezone TZ (is this hypothesis true?)
GPS tracks with UTC timestamps tg which can span over several time zones
We also know that there is at least one GPS track for each image.
Here's something that should work:
Convert your target day into UTC. You get the values t0/UTC and t1/UTC
Convert images' timestamps into UTC (you get ti/UTC from known ti/TZ)
process image if (t0/UTC <= ti/UTC < t1/UTC) i.e. it was taken during your target day.
find a GPS track including ti/UTC (no problem since tracks are timestamped in UTC), and then the closest timestamp within the list of points in this track. This point is the most likely position of your image.
I want to know if there can be 2 or more GMT timezones for one city or state. I know there can be more then one GMT timezone for a country, but not sure if it's for state and city too. Share your knowledge please.
Interpreting the question to mean 'are there any cities which are in more than one time zone', then the answer is 'yes'. And there are American states with multiple time zones (Indiana and Arizona being two of them).
There has been recent discussion on the TZ mailing list about the area of China known as Xinjiang, which has a mixed population of Han Chinese and of Uyghurs. It seems that the Han use the standard Chinese time zone (Asia/Beijing), but the Uyghurs often use a local time zone. This is now encapsulated in the Olson database, with the name Asia/Urumqi for the Uyghur time zone.
So, for example, the zone.tab file in tzdata2010b.tar.gz, available from ftp://elsie.nci.nih.gov/pub/tzdata2010b.tar.gz (the code is ftp://elsie.nci.nih.gov/pub/tzcode2009t.tar.gz). There is an extensive description of how and why the change was made in the asia file.
Note that the Olson (Time Zone) database is now (2016-09-19) available from IANA at https://www.iana.org/time-zones rather than from NIH. You can get the current release easily enough; getting historical releases may be harder.
Yes, time zones really do change 20 times a year around the world, and sometimes at essentially no notice (that is, the government legislates the changes only a day or two before the change).
#basit asks:
Wow about the 20 times a year around the world. I'm trying to log the timezone for latitude and longitude, so now my question would be, how long should I log the data for? 6 months? 1 month? 2.. 3..?
And also, how long does it take for daylight savings to change in a year, because I need to log timezone with daylight saving and refresh the data after certain given period.
What I mean is that during the course of 2009, there were 20 issues of the time zone database, because of changes in rules in at least that many places. However, any given country usually only changes their rules once - though with Argentina, different states were changing their rules at different times and compounding the problems.
I'm not clear that we have enough information to tell you how long to log the data for. I'd be inclined to say at least 12 months, but it depends what you are going to do with it. At one level, all you need to do is keep up with the Olson database - that will tell you the time zone rules for essentially everywhere in the world. If you are interested in tracking the time zones of your visitors, then you can keep the data for as long as you like. Since not everyone uses the canonical Continent/City notation for their time zone (I tend to use the older US/Pacific notation, for instance - which is still supported, but is equivalent to America/Los_Angeles). The classical notations such as TZ=EST5EDT are ambiguous; both the USA and Australia have timezones that use EST as an abbreviation, and the dates when the switch between standard and daylight saving time occurs varies (witness the mass of data in the Olson database).
You also ask 'how long does it take for a time zone to change'. I'm not sure what you mean. In terms of 'when the clocks change (between standard and daylight saving time)', it is 'instantaneous'; one second it is one time zone offset; the next second it is the other. If you mean 'how long does it take for governments to change their mind', it varies radically. For example, both Europe and the USA have relatively fixed rules that change every few years; the rule in the USA had been stable for about 20 years, then they changed the rules about 3 years ago. Europe is similar. On the other hand, some countries change their rules yearly. My impression is that some of the Islamic countries adjust when they switch between standard and daylight saving time (or vice versa) depending in part on when Ramadan falls - if the change would occur during Ramadan, then they bring it forward, or delay it, so that the rule does not change during Ramadan. Other countries have different reasons for the brinksmanship that goes on - maybe it is the political equivalent of a release deadline. So it may take quite a while for people to decide what the 'final' (meaning 'next edition') of the rules will be for a given year.
The web site http://worldtimezone.com/ does a pretty good job of keeping track of most of these idiosyncracies.
I think you mean "Can one city or state span two time zones?". Yes. Mexico Beach, FL sits on the border between CST and EST with parts of the town in both time zones.
As for how you could tell a computer that, no idea.
There is only one gmt for the whole world. As for timezones, see here, showing variation of observance e.g. within Kansas.
Any arbitrary jurisdiction may have multiple timezones, though the majority do not.
Have a look at http://www.worldtimezone.com/faq.html