I would like to think that some of the software I'm writing today will be used in 30 years. But I am also aware that a lot of it is based upon the UNIX tradition of exposing time as the number of seconds since 1970.
#include <stdio.h>
#include <time.h>
#include <limits.h>
void print(time_t rt) {
struct tm * t = gmtime(&rt);
puts(asctime(t));
}
int main() {
print(0);
print(time(0));
print(LONG_MAX);
print(LONG_MAX+1);
}
Execution results in:
Thu Jan 1 00:00:00 1970
Sat Aug 30 18:37:08 2008
Tue Jan 19 03:14:07 2038
Fri Dec 13 20:45:52 1901
The functions ctime(), gmtime(), and localtime() all take as an argument a time value representing the time in seconds since the Epoch (00:00:00 UTC, January 1, 1970; see time(3) ).
I wonder if there is anything proactive to do in this area as a programmer, or are we to trust that all software systems (aka Operating Systems) will some how be magically upgraded in the future?
Update It would seem that indeed 64-bit systems are safe from this:
import java.util.*;
class TimeTest {
public static void main(String[] args) {
print(0);
print(System.currentTimeMillis());
print(Long.MAX_VALUE);
print(Long.MAX_VALUE + 1);
}
static void print(long l) {
System.out.println(new Date(l));
}
}
Wed Dec 31 16:00:00 PST 1969
Sat Aug 30 12:02:40 PDT 2008
Sat Aug 16 23:12:55 PST 292278994
Sun Dec 02 08:47:04 PST 292269055
But what about the year 292278994?
I have written portable replacement for time.h (currently just localtime(), gmtime(), mktime() and timegm()) which uses 64 bit time even on 32 bit machines. It is intended to be dropped into C projects as a replacement for time.h. It is being used in Perl and I intend to fix Ruby and Python's 2038 problems with it as well. This gives you a safe range of +/- 292 million years.
You can find the code at the y2038 project. Please feel free to post any questions to the issue tracker.
As to the "this isn't going to be a problem for another 29 years", peruse this list of standard answers to that. In short, stuff happens in the future and sometimes you need to know when. I also have a presentation on the problem, what is not a solution, and what is.
Oh, and don't forget that many time systems don't handle dates before 1970. Stuff happened before 1970, sometimes you need to know when.
You can always implement RFC 2550 and be safe forever ;-)
The known universe has a finite past and future. The current age of
the universe is estimated in [Zebu] as between 10 ** 10 and 2 * 10 **
10 years. The death of the universe is estimated in [Nigel] to occur
in 10 ** 11 - years and in [Drake] as occurring either in 10 ** 12
years for a closed universe (the big crunch) or 10 ** 14 years for an
open universe (the heat death of the universe).
Y10K compliant programs MAY choose to limit the range of dates they
support to those consistent with the expected life of the universe.
Y10K compliant systems MUST accept Y10K dates from 10 ** 12 years in
the past to 10 ** 20 years into the future. Y10K compliant systems
SHOULD accept dates for at least 10 ** 29 years in the past and
future.
Visual Studio moved to a 64 bit representation of time_t in Visual Studio 2005 (whilst still leaving _time32_t for backwards compatibility).
As long as you are careful to always write code in terms of time_t and don't assume anything about the size then as sysrqb points out the problem will be solved by your compiler.
Given my age, I think I should pay a lot into my pension and pay of all my depts, so someone else will have to fit the software!
Sorry, if you think about the “net present value” of any software you write today, it has no effect what the software does in 2038. A “return on investment” of more than a few years is uncommon for any software project, so you make a lot more money for your employer by getting the software shipped quicker, rather than thinking that far ahead.
The only common exception is software that has to predict future, 2038 is already a problem for mortgage quotation systems.
I work in embedded and I thought I would post our solution here. Our systems are on 32 bits, and what we sell right now has a warantee of 30 years which means that they will encounter the year 2038 bug. Upgrading in the future was not a solution.
To fix this, we set the kernel date 28 years earlier that the current date. It's not a random offset, 28 years is excatly the time it will take for the days of the week to match again. For instance I'm writing this on a thursday and the next time march 7 will be a thursday is in 28 years.
Furthermore, all the applications that interact with dates on our systems will take the system date (time_t) convert it to a custom time64_t and apply the 28 years offset to the right date.
We made a custom library to handle this. The code we're using is based off this: https://github.com/android/platform_bionic
Thus, with this solution you can buy yourself an extra 28 years easily.
I think that we should leave the bug in. Then about 2036 we can start selling consultancy for large sums of money to test everything. After all isn't that how we successfully managed the 1999-2000 rollover.
I'm only joking!
I was sat in a bank in London in 1999 and was quite amazed when I saw a consultant start Y2K testing the coffee machine. I think if we learnt anything from that fiasco, it was that the vast majority of software will just work and most of the rest won't cause a melt down if it fails and can be fixed after the event if needed. As such, I wouldn't take any special precautions until much nearer the time, unless you are dealing with a very critical piece of software.
Keep good documentation, and include a description of your time dependencies. I don't think many people have thought about how hard this transition might be, for example HTTP cookies are going to break on that date.
What should we do to prepare for 2038?
Hide, because the apocalypse is coming.
But seriously, I hope that compilers (or the people who write them, to be precise) can handle this. They've got almost 30 years. I hope that's enough time.
At what point do we start preparing for Y10K? Have any hardware manufacturers / research labs looked into the easiest way to move to whatever new technology we'll have to have because of it?
By 2038, time libraries should all be using 64-bit integers, so this won't actually be that big of a deal (on software that isn't completely unmaintained).
COBOL programs might be fun though.
Operative word being "should".
If you need to ensure futureproofing then you can construct your own date/time class and use that but I'd only do that if you think that what you write will be used on legacy OS'
Related
Note: For convenience, PowerShell is used to demonstrate the behavior, but the question is about the behavior of the System.DateTimeOffset .NET type.
Note: As Matt Johnson points out, the behavior in question only happens on Unix-like platforms (macOS, Linux, using .NET Core).
Seemingly, when System.DateTimeOffset converts a local date, the resulting time-zone offset (offset from UTC) differs for dates before 19 Nov 1883.
Consider the following PowerShell command:
([datetime] '1883-11-18'), ([datetime] '1883-11-19') |
ForEach-Object { ([datetimeoffset] $_).Offset.ToString() }
which, when invoked in the Eastern Time Zone yields:
-04:57:00
-05:00:00
(DST (daylight-saving time) didn't come into play until 1918.)
I suspect that this is related to the following, excerpted from Wikipedia, emphasis added:
[...] had brought the US railway companies to an agreement which led to standard railway time being introduced at noon on 18 November 1883 across the nation.
Can anyone confirm this and provide more information?
Why, specifically, did the offset change by 3 minutes?
Searching for 1883 in the entire .NET documentation yields just the following, passing reference:
[...] the U.S. Eastern Standard Time zone from 1883 to 1917, before the introduction of daylight saving time in the United States
When running on Unix-like platforms (Linux, OSX, etc.), the time zone data in the operating system originates from the IANA time zone database. Early dates in this data set are referenced by their Local Mean Time (LMT), which is calculated from the latitude and longitude of the reference city. In this case, US Eastern Time being represented by America/New_York has an LMT entry that aligns with your reported findings.
From the tzdb:
# Zone NAME GMTOFF RULES FORMAT [UNTIL]
Zone America/New_York -4:56:02 - LMT 1883 Nov 18 12:03:58
-5:00 US E%sT 1920
-5:00 NYC E%sT 1942
-5:00 US E%sT 1946
-5:00 NYC E%sT 1967
-5:00 US E%sT
Note that seconds in the offset are truncated on import due to the precision of offsets supported by DateTimeOffset.
You can read more about LMT (and lots more) in the theory file in the tz database.
On Windows, you would not see this behavior, because Windows doesn't have time zone data for the Eastern Standard Time time zone earlier than the 2006-2007 transition. Thus dates earlier than 1987 may be converted incorrectly. If that matters to you, you'll need another source of time zone information, such as the copy of tzdb included with Noda Time.
Also note that local time is implied with casting from DateTime to DateTimeOffset, but only when the input value's .Kind is DateTimeKind.Local or DateTimeKind.Unspecified. If instead it's DateTimeKind.Utc, then the resulting DateTimeOffset will have a zero offset.
Okay so I've figured out why timestamps can't go over 2038 because of the 32 bit integers, that's cool. But I couldn't help but wonder what will be done to overcome this problem. I can't seem to find a straight answer to that question anywhere and it's bugging me so bad. Like if they introduce 64 bit timestamps are we gonna have to refactor our systems? Or if they decide to split timestamps into chunks like 1970-2038, 2038-2076 etc.. ?
Time is often converted into numeric parameter (e.g., to miliseconds or other units) elapsed from a reference date (epoch time)
The overview on wikipedia is very incomplete:
http://en.wikipedia.org/wiki/Epoch_%28reference_date%29
What is is the list of epoch times for all possible OS platforms and major programming languages?
(e.g., R (running on different OS platforms,unix, windows, Android, Apple, Perl, Python, Ruby, C++, Java).
In most modern frameworks, it's the Unix/POSIX standard of 1/1/1970.
You asked about R - it's 1/1/1970. Refrence Here
Most languages/frameworks that are cross platform either do this internally, or they abstract it. It would be too painful otherwise. Imagine having to compensate for a different epoch every time you re-targeted. That would be aweful.
BTW - There is another list here that may be more interesting to you.
I have got the following data:
In a computing context, an epoch is the date and time relative to which a computer's clock and timestamp values are determined. The epoch traditionally corresponds to 0 hours, 0 minutes, and 0 seconds (00:00:00) Coordinated Universal Time (UTC) on a specific date, which varies from system to system. Most versions of Unix, for example, use January 1, 1970 as the epoch date; Windows uses January 1, 1601; Macintosh systems use January 1, 1904, and Digital Equipment Corporation's Virtual Memory System (VMS) uses November 17, 1858.
Reference : Epoch
also you can see this : Epoch Computer
In Java epoch is like unix i.e. January 1 1970 Midnight which is used in programming widely.
I made a small test benchmark comparing .NET's System.Security.Cryptography AES implementation vs BouncyCastle.Org's AES.
Link to GitHub code: https://github.com/sidshetye/BouncyBench
I'm particularly interested in AES-GCM since it's a 'better' crypto algorithm and .NET is missing it. What I noticed was that while the AES implementations are very comparable between .NET an BouncyCastle, the GCM performance is quite poor (see extra background below for more). I suspect it's due to many buffer copies or something. To look deeper, I tried profiling the code (VS2012 => Analyze menu bar option => Launch performance wizard) and noticed that there was a LOT of CPU burn inside mscorlib.dll
Question: How can I figure out what's eating most of the CPU in such a case? Right now all I know is "some lines/calls in Init() burn 47% of CPU inside mscorlib.ni.dll" - but without knowing what specific lines, I don't know where to (try and) optimize. Any clues?
Extra background:
Based on the "The Galois/Counter Mode of Operation (GCM)" paper by David A. McGrew, I read "Multiplication in a binary field can use a variety of time-memory tradeoffs. It can be implemented with no key-dependent memory, in which case it will generally run several times slower than AES. Implementations that are willing to sacrifice modest amounts of memory can easily realize speeds greater than that of AES."
If you look at the results, the basic AES-CBC engine performances are very comparable. AES-GCM adds the GCM and reuses the AES engine beneath it in CTR mode (faster than CBC). However, GCM also adds multiplication in the GF(2^128) field in addition to the CTR mode, so there could be other areas of slowdown. Anyway, that's why I tried profiling the code.
For the interested, where is my quick test performance benchmark. It's inside a Windows 8 VM and YMMV. The test is configurable but currently it's to simulate crypto overhead in encrypting many cells of a database (=> many but small plaintext input)
Creating initial random bytes ...
Benchmark test is : Encrypt=>Decrypt 10 bytes 100 times
Name time (ms) plain(bytes) encypted(bytes) byte overhead
.NET ciphers
AES128 1.5969 10 32 220 %
AES256 1.4131 10 32 220 %
AES128-HMACSHA256 2.5834 10 64 540 %
AES256-HMACSHA256 2.6029 10 64 540 %
BouncyCastle Ciphers
AES128/CBC 1.3691 10 32 220 %
AES256/CBC 1.5798 10 32 220 %
AES128-GCM 26.5225 10 42 320 %
AES256-GCM 26.3741 10 42 320 %
R - Rerun tests
C - Change size(10) and iterations(100)
Q - Quit
This is a rather lame move from Microsoft as they obviously broke a feature that worked well before Windows 8, but no longer, as explained in this MSDN blog post:
:
On Windows 8 the profiler uses a different underlying technology than
what it does on previous versions of Windows, which is why the
behavior is different on Windows 8. With the new technology, the
profiler needs the symbol file (PDB) to know what function is
currently executing inside NGEN’d images.
(...)
It is however on our backlog to implement in the next version of Visual Studio.
The post gives directions to generate the PDB files yourself (thanks!).
"Geolocation is the identification of the real-world geographic location of an object. Geolocation may refer to the practice of assessing the location, or to the actual assessed location." -- http://en.wikipedia.org/wiki/Geolocation
Is there a standard way to describe temporal locations/coordinates that extend beyond Unix timestamps? If not, please suggest or describe an outline for one. Such a system would formalize times like:
-13,750,000,000 ± 11,000,000 (Big Bang)
1970-01-01:00:00:00 (Unix Epoch)
1 (Year 1 CE)
For example: both Geolocations and Chronolocations frequently lack precision -- this is just one consideration but I'm sure there are more.
My goal is to formalize a way to store and retrieve temporal locations of all kinds. As you might imagine this is more complex than it sounds.
I have never heard of such a system, but it would be fairly trivial to write a class where a structured data type like this exists:
struct bigTime{
signed long int millenium;
int decade;
signed long int seconds;
}time;
You could store milennia before/after an arbitrary point (even 1970 for simplicity) for long range, decades for mid range, then use seconds and milliseconds as short term.
You could create a class where adding +/- $X seconds, minutes, hours, days, weeks, months, years, decades, centuries, millenia would be straightforward.
Say you wanted to go 156 years back. that's -15 decades and -189 341 556 seconds.
Or 3205 years and 2 weeks and a day back. That's -3 millenia, -20 decades, -159 080 630 seconds.
Or even 67,000,012 years (from jonathan's offtopic joke). That's -67000 millenia, -1 decade -63 113 851.9 seconds.
All of those are from today, but would be from whatever arbitrary point you chose.
The system I describe would give you 4.2 Trillion years to work with either way down to the millisecond, and more or less minimize memory required. (I'm sure it could be brought down more if you tried)