We have a database of cities with its geo coordinates. Once we filled it with corresponding time zones using tzworld. User sets location including city, city has time zone - here how we know user's timezone (we need to render date and time on server). But time zones are being changed: some new are appearing, some old are being removed.
Is there any best practices or tools to handle that kind of changes?
I.e. there is a city Foo with time zone Foo/Bar. One day tzdata was changed, and Foo/Bar was split into Foo/Old_Bar and Foo/New_Bar time zones with the same UTC offsets. We still have Foo/Bar in our db. Actually, it's a BC break, but it's ok since, say, we can handle those BC breaks. But then tzdata was changed again, and now Foo/New_Bar has different offset. And here comes troubles. Some users from Foo city see wrong local time since that moment.
Just to be sure you understand me right: it's not about DST, it's about the fact that time zones (their names) are being changed.
As far as I can see, wee need a kind of machine-readable tzdata diff. Like
split: Foo/Bar Foo/Old_Bar,Foo/New_Bar
move: Foo/New_Bar -05:00
This issue makes me feel that storing time zones is a bad idea. Is there a better one?
With specific regard to the IANA/Olson TZ database, the location identifiers do not change once established. The history of each identifier is always consistent for that location.
However, if you are using tz_world or some other map source to determine the time zone for some other location - one that doesn't necessarily have it's own identifier, then yes - it's possible that a zone split will cause the zone to change. Though, when it does, the new zone should be consistent with the old zone, up to the point of the change.
As a real world example, consider America/Fort_Nelson, which was added in tzdb 2015g for Fort Nelson, British Columbia, Canada, and the surrounding region of the Northern Rockies Regional Municipality. Previously, this area would have been resolved to America/Vancouver, but the zone was split due to their March 2015 time change. The tz_world maps were updated on November 7, 2015 to account for this change.
If you had previously resolved a user in Fort Nelson to America/Vancouver, then they will have incorrect times from November 1st, 2015 forward, as that's when Vancouver switched back to UTC-8, while Fort Nelson remained at UTC-7.
If you update to the latest tzdb and tz_world, you can use the original information to re-determine the time zone - which would now be America/Fort_Nelson.
The new time zone will accurately reflect all of the same information as Vancouver before the split, and the correct information for Fort Nelson after the split.
All of this should just work, assuming you update time zones after each update of tz_world, and recalculate future events after updating the tzdb.
The question remains, how do you know which zones have split and changed so you don't have to recalculate everything? For a small amount of data, you might as well recalculate everything. But for larger datasets, this might be impractical. Unfortunately, there's no machine-readable standardized format for the differences. I believe this has been talked about before in the tz discussion list, but I can't find it at the moment. You can ask there if you like.
Currently the only way is to manually read the release notes of each update. You can find them in the tz-announce list archives (or subscribe to the list for future updates). You can also find them in the NEWS file of any given release. You'll also want to review the history of the tz_world shapefile, which is on that web site.
Also, recognize that time zone IDs will never be removed from the tzdb. A split may create a new zone (Foo/New_Bar), but the original zone will remain (Foo/Bar, not Foo/Old_Bar). If a zone is determined unnecessary, its Zone entry might be replaced with a Link entry, but it will never be removed entirely.
Related
I have a Redshift data table where all time values are stored in CST and I convert the time values to the respective timezone based on the zip code (location).
While I do that, I understand that all time values are in Standard time and hence my function usage is
CASE WHEN **** convert_timezone('CST', 'EST', time_column)
WHEN **** convert_timezone('CST', 'MST', time_column)
....
END
This may not be applicable once we enter into Daylight Savings time. How can I take care of this such that I do not modify the SQL query again in 2018 March and in future?
Don't use time zone abbreviations. The are somewhat ambiguous, and can only refer to one aspect of the time zone. Instead, use a full IANA time zone identifier, such as America/Chicago for US Central time.
This is explained well in the Redshift docs:
Using a Time Zone Name
If you specify a time zone using a time zone name, CONVERT_TIMEZONE automatically adjusts for Daylight Saving Time (DST), or any other local seasonal protocol, such as Summer Time, Standard Time, or Winter Time, that is in force for that time zone during the date and time specified by 'timestamp'. For example, 'Europe/London' represents UTC in the winter and UTC+1 in the summer.
As far as the "...based on the zip code" part of your question, understand that not every ZIP code is locality-based. There are also technical assignments, overseas APO/FPO addresses, US territories, and other edge cases. Additionally, some zip codes may straddle more than one time zone.
A better approach, when possible, is to:
Get an approximation of latitude/longitude coordinates - using a variety of techniques depending on your source data. For example, geocoding APIs can take a street address and give a lat/lon.
Then determine the time zone identifier for that location, using one of the techniques listed here.
I have web application which controls the stores locate din different time zones.
In this web application user will set rules that, any particular product is in discount for any given period of time i.e. 1-Aug 2014 to 5-Aug-2014. so how can this rule be executed in web application. this info can be inserted from anytime zone. but store in Europe and Store in US should have to make this product available from exact 1-Aug-2014, Europe will be early compare to US.
SO how we can handle this kind of scenarios.
In general:
Store the time zone for each store, such as America/Los_Angeles or Europe/London.
When checking for discounts, get the current UTC time and use the store's time zone to determine the appropriate local time.
Compare that local time against the expiration date for the discount.
This assumes that the business rule is to rely upon the store location's time zone. This might not necessarily be the case - as many online stores serve customers worldwide without regard to where the physical store or product is located. In that case, you need to determine who's time zone is applicable. Is it a single fixed time zone for the whole company? Or perhaps it's aligned the end-user's time zone, such that those in different time zones would have the offer expire at different times. You'll need to decide what is appropriate for your particular business.
Sorry I can't be more specific, but you didn't provide many details to go on. If you need further assistance, please consider editing your question to include details such as what language and platform you're using, what code you have tried, and what the specific business rules are.
When getting information from Twitter's API for a user, they provide two fields related to the user's time zone:
utc_offset: -14400,
time_zone: "Indiana (East)"
Unfortunately, this doesn't tell the full story because I don't know if that UTC offset was calculated during standard time or daylight savings time. After dividing by 3600 seconds, I get -4 hours, which is valid during the summer months, but in the winter the correct value would be -5 hours.
If the value was ALWAYS determined by the daylight savings time value then I could write an algorithm for that, however after some searching on the subject I've seen several pasted outputs that contradict that assumption. (as a quick example, this question shows his/her offset as -21600 and then he/she says he/she is on central time, which if calculated during daylight savings time would be -18000).
It would make sense to me that the value would be calculated as of Jan 1 and the several pasted outputs I've found online fall into that category, but my own Twitter account shows the values listed above for which this assumption is invalid. My next thought was maybe it was calculated at the time I created my account, but then that seems erroneous as well because I can change my time zone at any later point (and even so, I created my account in November when I would have been on standard and not daylight time!).
My last thought was that maybe the value is being calculated by the date of the API request. This makes a lot of sense and the Twitter accounts I own all seem to validate this. BUT, the SA question I linked to earlier shows that the person answered the question on June 2nd, which is daylight savings time and his/her value of -21600 reflects a standard time for the Central time zone.
Anyone out there solve this problem? Thanks so much!
Twitter's front end uses Ruby on Rails. If you go to your own twitter account settings and look at the possible options for time zones (view source on the dropdown list), you will find that they match up with those provided by ActiveSupport::TimeZone, shown in this documentation. Although there appears to be some zones understood by Rails that Twitter has omitted, all of the Twitter zone key names are in that list.
I have asked Twitter to use standard time zone names in the future, in this developer request.
Why does Rails limit this list and use their own key values? Who knows. I have asked before, and gotten very little response. Read here.
But you can certainly use their mapping dictionary to turn the time_zone value into a standard IANA time zone identifier. For example:
"Indiana (East)" => "America/Indiana/Indianapolis"
"Central Time (US & Canada)" => "America/Chicago"
This can be found in the Rails documentation, and in the source code. (Scroll down to MAPPING.)
Then you can use any standard IANA/Olson/TZDB implementation you wish. They exist for just about every language and platform. For further details, see the timezone tag wiki. If you need help with a specific implementation, you'll need to expand your question to tell us what language you are using and what you have tried so far. (Or consider asking a new question about just that part of it.)
In regards to the utc_offset field, twitter does not make it clear what basis they use to calculate it. My guess is that it is the user's current offset, based on the time that you call the API.
Update 1
I have added support for converting Rails time zone names to both IANA and Windows standard time zone identifiers in my TimeZoneConverter library for .NET. If you are using .NET, you can use this library to simplify your conversions and stay on top of updates more easily.
Update 2
Twitter's API now returns the time zone in this format:
"time_zone": {
"name": "Pacific Time (US & Canada)",
"tzinfo_name": "America/Los_Angeles",
"utc_offset": -28800
},
Use the tzinfo_name field. Done. :)
I'm writing a geotagging app and running into headaches with timezones. Basically, my app has the following data:
Images with local timestamps (i.e. relative to a timezone)
GPS track files consisting of entries using UTC timestamps
My problem: I need a way to get all data that belongs to a give day, based on the timezone where the data was acquired. For the images, that is easy (I ask the timezone from the user upon import and save it in the EXIF data), but I'm not sure how to do it for the GPS tracks (there usually are multiple tracks per day, and assigning them timezones is not easy for the user when importing data that spans sever days and timezones). I can think of two possible solutions:
Use a heuristic based on the fact that the tracks are recorded at the same time and place as the images - but there can be tracks before a day's first image or after its last one that still need to be included - I'm not sure how to realiably handle such edge cases
Determine the timezone from the GPS coordinates - this would be an ideal solution, but is there an open source library that does this (ideally one that works offline)?
The heuristic method I don't think will work well.
Firstly always store times as UTC and timezone of origin, otherwise time is less meaningful.
After some thought I think that it would be sufficient to resolve down to the country code and from that lookup the timezone.
Depending on how much detail you want I think GeoTog may help you to locate a city and therefore a country from a lat/long (although it will need changing to work the other way).
If not that then Gisgraphy will work with the larger GeoNames database. You could use the web service or extract the data.
If none of these are good enough then I think you'll need to get a some GIS data, possibly boundaries from VMAP0 and process it into polygons or something searchable.
Option two: you could start by checking this site: http://www.twinsun.com/tz/tz-link.htm
Option one (less complicated, but I am not sure I accurately understand your need...)
So you have as input:
A target day defined in a known timezone TZ, starting at t0 and ending at t1 (excluded)
Images with timestamps ti in the same timezone TZ (is this hypothesis true?)
GPS tracks with UTC timestamps tg which can span over several time zones
We also know that there is at least one GPS track for each image.
Here's something that should work:
Convert your target day into UTC. You get the values t0/UTC and t1/UTC
Convert images' timestamps into UTC (you get ti/UTC from known ti/TZ)
process image if (t0/UTC <= ti/UTC < t1/UTC) i.e. it was taken during your target day.
find a GPS track including ti/UTC (no problem since tracks are timestamped in UTC), and then the closest timestamp within the list of points in this track. This point is the most likely position of your image.
I want to know if there can be 2 or more GMT timezones for one city or state. I know there can be more then one GMT timezone for a country, but not sure if it's for state and city too. Share your knowledge please.
Interpreting the question to mean 'are there any cities which are in more than one time zone', then the answer is 'yes'. And there are American states with multiple time zones (Indiana and Arizona being two of them).
There has been recent discussion on the TZ mailing list about the area of China known as Xinjiang, which has a mixed population of Han Chinese and of Uyghurs. It seems that the Han use the standard Chinese time zone (Asia/Beijing), but the Uyghurs often use a local time zone. This is now encapsulated in the Olson database, with the name Asia/Urumqi for the Uyghur time zone.
So, for example, the zone.tab file in tzdata2010b.tar.gz, available from ftp://elsie.nci.nih.gov/pub/tzdata2010b.tar.gz (the code is ftp://elsie.nci.nih.gov/pub/tzcode2009t.tar.gz). There is an extensive description of how and why the change was made in the asia file.
Note that the Olson (Time Zone) database is now (2016-09-19) available from IANA at https://www.iana.org/time-zones rather than from NIH. You can get the current release easily enough; getting historical releases may be harder.
Yes, time zones really do change 20 times a year around the world, and sometimes at essentially no notice (that is, the government legislates the changes only a day or two before the change).
#basit asks:
Wow about the 20 times a year around the world. I'm trying to log the timezone for latitude and longitude, so now my question would be, how long should I log the data for? 6 months? 1 month? 2.. 3..?
And also, how long does it take for daylight savings to change in a year, because I need to log timezone with daylight saving and refresh the data after certain given period.
What I mean is that during the course of 2009, there were 20 issues of the time zone database, because of changes in rules in at least that many places. However, any given country usually only changes their rules once - though with Argentina, different states were changing their rules at different times and compounding the problems.
I'm not clear that we have enough information to tell you how long to log the data for. I'd be inclined to say at least 12 months, but it depends what you are going to do with it. At one level, all you need to do is keep up with the Olson database - that will tell you the time zone rules for essentially everywhere in the world. If you are interested in tracking the time zones of your visitors, then you can keep the data for as long as you like. Since not everyone uses the canonical Continent/City notation for their time zone (I tend to use the older US/Pacific notation, for instance - which is still supported, but is equivalent to America/Los_Angeles). The classical notations such as TZ=EST5EDT are ambiguous; both the USA and Australia have timezones that use EST as an abbreviation, and the dates when the switch between standard and daylight saving time occurs varies (witness the mass of data in the Olson database).
You also ask 'how long does it take for a time zone to change'. I'm not sure what you mean. In terms of 'when the clocks change (between standard and daylight saving time)', it is 'instantaneous'; one second it is one time zone offset; the next second it is the other. If you mean 'how long does it take for governments to change their mind', it varies radically. For example, both Europe and the USA have relatively fixed rules that change every few years; the rule in the USA had been stable for about 20 years, then they changed the rules about 3 years ago. Europe is similar. On the other hand, some countries change their rules yearly. My impression is that some of the Islamic countries adjust when they switch between standard and daylight saving time (or vice versa) depending in part on when Ramadan falls - if the change would occur during Ramadan, then they bring it forward, or delay it, so that the rule does not change during Ramadan. Other countries have different reasons for the brinksmanship that goes on - maybe it is the political equivalent of a release deadline. So it may take quite a while for people to decide what the 'final' (meaning 'next edition') of the rules will be for a given year.
The web site http://worldtimezone.com/ does a pretty good job of keeping track of most of these idiosyncracies.
I think you mean "Can one city or state span two time zones?". Yes. Mexico Beach, FL sits on the border between CST and EST with parts of the town in both time zones.
As for how you could tell a computer that, no idea.
There is only one gmt for the whole world. As for timezones, see here, showing variation of observance e.g. within Kansas.
Any arbitrary jurisdiction may have multiple timezones, though the majority do not.
Have a look at http://www.worldtimezone.com/faq.html