I run discrete event simulations which cover more than a year. All events need to be tracked on a second time scale.
Now, I want to schedule the events according to a timetable, calculate overtime, ... and compare it to the real labor time in the end.
I thought whether there is an elegant way to implement schedules in R such that the following tasks/questions can be answered within one line of code:
Schedule work according to a timetable.
Given a timetable, how many labor hours do we expect per day/week/month/... Calculate the overtime per day/week/month/...
Create simple plots to visualize working time, overtime, ... and compare it to the planned work load.
On an abstract level, assume all of the timetables are given more or less in the following way:
Name Days_of_year Available Time
Name1 Jan-07 – Oct-18 Mon, Tues, Wed, Thurs, Fri, Sat, Sun 8:45 - 18:00
Name2 Jan-01 – Dec-31 Mon, Tues, Wed, Thurs, Fri 20:00 - 7:00
I am not looking for obvious answers like "create an arry and somehow you get everything out of that" but an implementation that:
Makes schedules independent of time resolution (days, hours, minutes, seconds).
Schedules/timetables are not stored with redundant information.
Any result (e.g. total year labor, ...) should be easily transferred to any given unit.
Easy filter possibilities by month, wday, ...
Ideally, calendar functionalities (holidays, ...) can be easily used.
Perhaps as part of lubridate?
Use case view ("schedule functions" indicated by s_):
Time: s_hm(„7:45“), s_md(„Jan-23“) or s_md(„01-23“). Please note that those "times" do not make sence as dates or date-times, but in the framework of schedules they perfectly make sence!
Interval: s_interval. See first thoughts below.
Duration: s_interval/dhours(1) => Duration in hours
Functions:
Max(), Min() on any s_interval
%s_in%
Given a date, when does somebody work:
date(s) %s_in% s_interval
=> return TRUE/FALSE (array)
Given a schedule, when is somebody available in an interval:
s_interval %s_in% interval
=> return class interval - a subset of interval!
Given an interval, was the labor according to schedule? Calculate the idle-, over- and working time:
interval %s_in% s_interval
=> return class s_interval
Given two schedules, consider them as one unit. How is idle-, over- and working time distributed:
(s_interval1 %+% s_interval2)
=> return class s_interval
Some thoughts:
schedule in case of planning labor time:
s_interval $tzone UTC
$daysofyear 286 days (Better alternative: format month-day e.g. Jan-07 – Oct-18)
$wday Mon, Tues, ... Sun (According to the start time of a shift/lecture...)
$time 12 h (Better alternative: format time span* e.g. 8:00 – 18:00. NOT period which results in hours)
$overtime 0 h (A priori, a schedule does not plan overtime)
$idle 0 h (A priori, a schedule only plans labor time)
schedule when real time-data is available:
s_interval$tzone UTC
$daysofyear 286 days (Better alternative: format month-day e.g. Jan-07 – Oct-18)
$wday c(Mon, Tues, Wed, ...) (According to the start time of a shift/lecture...)
$time c(12 h, 1 h, 0.5 h, ...) (Better alternative: format time span* e.g. 8:00 – 18:00. NOT period which results in hours)
$overtime c(2 h, 0 h, -1 h, ...) total time = time + overtime. Array of lubridate intervals*?
$idle c(4 h, 8 h, 0h, ...) pure working_time = time – idle + overtime. Array of lubridate intervals*?
* Use intervals not periods such that interval/dhours(1) possible.
If the tzone is the same we could calculate e.g.
s_interval1 %+% s_interval2 =
s_interval$tzone UTC
$daysofyear 286 days, 123 days
$wday c(Mon, Tues, Wed, ...), c(Sat, Sun)
$time c(12 h, 1 h, 0.5 h, ...), c(-1 h, 2.5 h)
$overtime c(2 h, 0 h, -1 h, ...), c(0 h, 5 h)
$idle c(4 h, 8 h, 0h, ...), c(4 h, 8 h)
There are related posts about this topic visualization of schedules with some interesting answers concerning timeline and gantt packages and how to filter dates.
However, they are not very conclusive.
As I am quite new to R, I don't know, how to start such a task the best way and understanding the structure of a package like lubridate is quite advanced...
I developed a package which gives the desired functionality (even with plots). At the moment, the package is private, but let me know if you are interested.
Related
I am trying to determine the interval between two dates that I create using DateComponents. If I make the first date 1 year prior to the second, I get 365 days, 0 hours, 0 minutes and 0 seconds. If I make the dates further apart (400 years here), suddenly my date is off by 11 minutes 56 seconds. Here is the code:
import Foundation
var mycal = Calendar(identifier: .iso8601)
var datum = DateComponents(year:1600, month:1, day:1, hour:12, minute:0,
second:0)
let j2000 = DateComponents(year:2000, month:1, day:1, hour:12, minute:0,
second:0)
let datum_date = mycal.date(from: datum)
let j2000_date = mycal.date(from: j2000)
let interval = mycal.dateComponents([.day, .hour, .minute, .second], from:j2000_date!, to:datum_date!)
print("Datum: \(datum_date!)") //1600-01-01 19:48:04 +0000
print("j2000: \(j2000_date!)") //2000-01-01 20:00:00 +0000
Note the next-to-last line: Comments show what the print produces. I've tried it with the Gregorian calendar too, same problem. I'm not sure exactly how far back the inconsistency occurs, I've gone back far enough to produce and it sometimes seems to "stick" as I change the code moving closer in time again. Strangely, the "interval" appears to show the correct amount of days(here -146097), but the date shown is incorrect and I will likely need that in my calculations. Anyone have any ideas?
The difference could be related to leep year adjustments but that would give a difference of 11 minutes and 14 seconds (there'd still be 40 seconds unaccounted for, 26 of which could be leep seconds).
see: https://www.infoplease.com/leap-year-101-next-when-list-days-calendar-years-calculation-last-rules
In Theory, if you compute a multi-year time difference with a precision of minutes and seconds, you should get variations of 5 hours 48 minutes and 46 seconds every 3 out of four years and get within 11 minutes and 14 seconds on the fourth year. I don't know how macOS (Unix) deals with that there there is probably a bunch of considerations that they need to take into account (especially beyond 400 year where that 11 minutes 14 seconds gets adjusted).
If that level of precision is required by your use case, I would suggest reading up on the minute details of time calculations. Given that dates are stored internally as a number of seconds, going back to a precise day and time over these long periods must require some special math acrobatics.
See Apple's documentation here: https://developer.apple.com/reference/foundation/nscalendar
If I have everyday datetime - how to find out, the event has already occurred or not, by subtraction with datetime.now()
Let we had everyday meeting at 15:35. Today John came earlier - at 12:45, but Alex was late for 2 h. and 15 min. (came at 17:40).
meet_dt = datetime(year=2015, month=8, day=19, hour=15, minute=35)
john_dt = datetime(year=2015, month=8, day=19, hour=12, minute=45)
alex_dt = datetime(year=2015, month=8, day=19, hour=17, minute=40)
print(meat_dt - john_dt) # came before > 2:50:00
print(meat_dt - alex_dt) # came after > -1 day, 21:55:00
If I take away from the big date less - then everything is fine, but conversely I recive -1 day, 21:55:00 why not -2:15:00, what a minus day?
Because timedeltas are normalized
All of the parts of the timedelta other than the days field are always nonnegative, as described in the documentation.
Incidentally, if you want to see what happened first, don't do this subtraction. Just compare directly with <:
if then < datetime.datetime.now():
# then is in the past
I am interacting with a Remote Server. This Remote Server is in a different Time Zone. Part of the Authentication requires me to produce the:
"The number of seconds since January 1, 1970 00:00:00 GMT
The server will only accept requests where the timestamp
is within 600s of the current time"
The documentation of erlang:now(). reveals that it can get me the the elapsed time since 00:00 GMT, January 1, 1970 (zero hour)
on the assumption that the underlying OS supports this. It returns a size=3 tuple, {MegaSecs, Secs, MicroSecs}. I tried using element(2,erlang:now()) but the remote server sends me this message:
Timestamp expired: Given timestamp (1970-01-07T14:44:42Z)
not within 600s of server time (2012-01-26T09:51:26Z)
Which of these 3 parameters is the required number of seconds since Jan 1, 1970 ? What aren't i doing right ? Is there something i have to do with the universal time as in calendar:universal_time() ? UPDATEAs an update, i managed to switch off the time-expired problem by using this:
seconds_1970()->
T1 = {{1970,1,1},{0,0,0}},
T2 = calendar:universal_time(),
{Days,{HH,Mins,Secs}} = calendar:time_difference(T1,T2),
(Days * 24 * 60 * 60) + (HH * 60 * 60) + (Mins * 60) + Secs.
However, the question still remains. There must be a way, a fundamental Erlang way of getting this, probably a BIF, right ?
You have to calculate the UNIX time (seconds since 1970) from the results of now(), like this:
{MegaSecs, Secs, MicroSecs} = now().
UnixTime = MegaSecs * 1000000 + Secs.
Just using the second entry of the tuple will tell you the time in seconds since the last decimal trillionellium (in seconds since the UNIX epoch).
[2017 Edit]
now is deprecated, but erlang:timestamp() is not and returns the same format as now did.
Which of these 3 parameters is the required number of seconds since Jan 1, 1970 ?
All three of them, collectively. Look at the given timestamp. It's January 7, 1970. Presumably Secs will be between 0 (inclusive) and 1,000,000 (exclusive). One million seconds is only 11.574 days. You need to use the megaseconds as well as the seconds. Since the error tolerance is 600 seconds you can ignore the microseconds part of the response from erlang:now().
I am writing a program in Fortran and I need a way of calculating the duration of the program down to milliseconds. I have been using the function "date_and_time", which leaves me with an array containing the system's time in hours, minutes, seconds, and milliseconds.
I believe that I can call this function at the start of my program to store the current time, then call the function again at the end of my program to store the latest time. But after that, how would I computer the duration? I tried just subtracting the values, but the milliseconds reset when one second passes, just like the seconds reset when one minute passes. How would be best way to approach this be?
Here is the program:
PROGRAM TEST_TIME_AND_DATE
INTEGER I
REAL J
INTEGER TIME_A(8), TIME_B(8)
CALL DATE_AND_TIME(VALUES=TIME_A)
PRINT '(8I5))', TIME_A
DO I = 0, 400000000
J = I * I - J
END DO
CALL DATE_AND_TIME(VALUES=TIME_B)
print '(8I5))', TIME_B
END PROGRAM TEST_TIME_AND_DATE
And here is the result:
2011 6 11 -300 9 14 49 304
2011 6 11 -300 9 14 50 688
I'm not sure what to do here, thanks.
If you want elapsed clock time, it would be simpler to use the intrinsic procedure system_clock since it provides a single time-value output. (There are additional arguments to provide information about the procedure, which is why it is a procedure instead of a function.) See, for example, http://gcc.gnu.org/onlinedocs/gfortran/SYSTEM_005fCLOCK.html. If you want to time the CPU usage, then use cpu_time. For either, two calls, at the start and end of the program, then a simple difference. You can use the COUNT_RATE argument to convert to integer count of time into seconds.
You can subtract the numbers, then convert everything into milliseconds and sum up the ms, sec in ms, min in ms, hrs in ms, ...
In your case this would be
0 + 0 + 0 + 0 + 0 + 1*1000 + 384 = 1384 [ms]
This approach works fine also with overflows since a positive number in a left-more column outweights negative numbers if they are all converted to the same basis. E.g. 0:58.000 to 1:02.200 yields
1 * 60000 + (-56) * 1000 + 200 = 4200
Please note that this does work up to days but not with months since they do not share a common length.
You could calculate the offset from some starting time (Jan 1, 1970 for UNIX) in seconds or milliseconds. The difference in those numbers is your elapsed time.
(2011 - 1970) * (number of seconds in a year) +
(month of the year - 1) * (number of seconds in a month) +
(day of the month - 1) * (number of seconds in a day) +
( ... )
In my dataset I have two timestamp columns. The first is microseconds since application was started - e.g., 1400805323. The second is described as 64bit timestamp which I'm hoping will indicate clock time, using NTP format of number of seconds from 1/1/1901.
Example of '64bit' timestamps:
129518309081725000
129518309082059000
129518309082393000
129518309082727000
129518309083060000
129518309083394000
129518309083727000
Is there any matlab/python code that could convert this into a readable format?
Any help much appreciated,
Steve
Assuming that these values were generated today, June 6th 2011, these values look like number of 100-nanosecond intervals since Jan 1st year 1601. This is how Windows NT stores FILETIME. For more concentrated info on this read this blog post of Raymond Chen. These articles also show how to convert it to anything else
See edit below for updated answer:
For NTP time, the 64bits are broken in to seconds and fraction of seconds. The top 32 bits is the seconds. The bottom 32 bits is the fraction of seconds. You get the fraction by dividing the fraction part by 2^32.
So step one, convert to a double.
If you like python that's easy enough, I didn't add any bounds checking:
def to_seconds(h):
return (h>>32) + ((float)(h&0xffffffff))/pow(2,32)
>>> to_seconds(129518309081725000)
30155831.26845886
The time module can covert that float to a readable time format.
import time
time.ctime(to_seconds(ntp_timestamp))
You'll need to worry about where the timestamp originated though. time.ctime assumes reletive to the Jan 1, 1970. So if your program is basing the ntp formats of time since program run, you'd need to add to the seconds to normalize the timestamp for ctime.
>>> time.ctime(to_seconds(129518309081725000))
'Tue Dec 15 17:37:11 1970'
EDIT:
PyGuy is right, the original timestamps are not ntp time numbers, they are Windows 64 bit timestamps.
Here is the new to_seconds method to convert the 100ns interval based on 1/1/1601 to the 1970 seconds interval:
def to_seconds(h):
s=float(h)/1e7 # convert to seconds
return s-11644473600 # number of seconds from 1601 to 1970
And the new output:
import time
time.ctime(to_seconds(129518309081725000))
'Mon Jun 6 04:48:28 2011'