If I have everyday datetime - how to find out, the event has already occurred or not, by subtraction with datetime.now()
Let we had everyday meeting at 15:35. Today John came earlier - at 12:45, but Alex was late for 2 h. and 15 min. (came at 17:40).
meet_dt = datetime(year=2015, month=8, day=19, hour=15, minute=35)
john_dt = datetime(year=2015, month=8, day=19, hour=12, minute=45)
alex_dt = datetime(year=2015, month=8, day=19, hour=17, minute=40)
print(meat_dt - john_dt) # came before > 2:50:00
print(meat_dt - alex_dt) # came after > -1 day, 21:55:00
If I take away from the big date less - then everything is fine, but conversely I recive -1 day, 21:55:00 why not -2:15:00, what a minus day?
Because timedeltas are normalized
All of the parts of the timedelta other than the days field are always nonnegative, as described in the documentation.
Incidentally, if you want to see what happened first, don't do this subtraction. Just compare directly with <:
if then < datetime.datetime.now():
# then is in the past
Related
I have been able to read the blogs and understand somewhat how the calculations are taking place, but need further clarity.
Case: Ingestion_Time_Logged will output the CreationTime event on the 7th min or the 37th min regardless of what time the actual CreationTime event occurred. Example:
If CreationTime Event occured at "2021-03-06 07:38:59.000" Then the Ingestion_Time_Logged will be the closest 7th min or 37th min. In this case the Ingestion_Time_Logged will be
"2021-03-06 08:07:59.000". The code works fine. I am just trying to understand how it is calculating it.
What I understand so far:
% is the modulo operator which is a way to determine the remainder of a division operation. Instead of returning the result of the division, the modulo operation returns the whole number remainder.
latestComposed_min%30 will divide the minutes by 30 and result in the reminder. I used it to find if the minute is between 0 - 7 or 30 - 37. It is the same as below but much easier and efficient.
if(latestComposed_min < 7 OR (latestComposed_min>30 AND latestComposed_min < 37))
Excluded 37 and 7 to keep them as they are since they are already ok, if we do not exclude them, they will be added 37 minutes. Since both are NOT comparisons AND should be used between them otherwise with OR the result will be always true (which is wrong)
The piece I am struggling with:
| eval Ingestion_Time_Logged=strftime(case(latestCreated_min%30 < 7, CreationTime_epoch-CreationTime_epoch%1800+420+latestCreated_sec, latestCreated_min!=37 AND latestCreated_min!=7, CreationTime_epoch-CreationTime_epoch%1800+2220+latestCreated_sec,1=1,CreationTime_epoch),"%Y-%m-%d %H:%M:%S.%6N")
Full Code:
| makeresults
| eval CreationTime="2021-03-06 07:38:59.000"
| eval CreationTime_epoch=strptime(CreationTime, "%Y-%m-%d %H:%M:%S.%6N")
| eval latestCreated_hour=tonumber(strftime(CreationTime_epoch, "%H"))
| eval latestCreated_min=tonumber(strftime(CreationTime_epoch, "%M"))
| eval latestCreated_sec=round(CreationTime_epoch%60,6)
| eval Ingestion_Time_Logged=strftime(case(latestCreated_min%30 < 7, CreationTime_epoch-CreationTime_epoch%1800+420+latestCreated_sec, latestCreated_min!=37 AND latestCreated_min!=7, CreationTime_epoch-CreationTime_epoch%1800+2220+latestCreated_sec,1=1,CreationTime_epoch),"%Y-%m-%d %H:%M:%S.%6N")
| table Ingestion_Time_Logged, CreationTime, CreationTime_epoch, latestCreated_hour, latestCreated_min
Let's break it down
Here, we're setting the value of the Ingestion_Time_logged field to the result of the strftime function. That is, we're converting a epoch time into a string.
| eval Ingestion_Time_Logged=strftime(
The epoch to convert is determined by a case statement.
case(
If the created minute (38 in the example) is 0-6 or 30-36
latestCreated_min%30 < 7,
then round down. The %1800 is the same as %30 above, only in seconds rather than minutes. Subtracting that from the epoch time shifts the epoch time to the top or bottom of the hour. The 420 adds in 7 minutes then we add in seconds.
CreationTime_epoch-CreationTime_epoch%1800+420+latestCreated_sec,
If the created minute is neither 7 nor 37 (our case)
latestCreated_min!=37 AND latestCreated_min!=7,
then we do the same as above, except using 37 minutes, advancing the time to the next :07 or :37.
CreationTime_epoch-CreationTime_epoch%1800+2220+latestCreated_sec,
This is the default (catch-all) case, which takes the CreationTime_epoch field as-is.
1=1,CreationTime_epoch),
This is how the timestamp string will be formatted.
"%Y-%m-%d %H:%M:%S.%6N")
I've looked at both [1] and [2] and I'm completely confused (and since the dbf file is a version
4 file, [1] should apply well). For one thing why does [1] state that the timestamp's date portion is the # of days since 1/1/4713 BC? That's just very puzzling. Secondly, assuming that it is the # of days since 4713 BC, I'm having some trouble with the value I am getting.
First off, my dbf file has a timestamp field which has an 8 byte long value. The actual
date is 2000/8/16 17:21:41. In the dbf file, the 8 byte sequence is as follows
0x42ccb20e0340df00.
From [1], it says the first 4 bytes are for the date, and 2nd 4 bytes for the time. If the original
byte sequence is actually little-endian (0x42ccb20e) then that should be 0x0eb2cc42 which
comes to the value of 246598722. So date is 0x0eb2cc42 (246598722) and time is 0x00df4003
(14630915).
I must be missing something here or calculating something wrong. 246598722 is equivalent to 675612 years(assuming 1yr = 365 days, as adding leap years would confuse me..and shouldn't really be that much off).
From [2], I shouldn't use 01/01/4173bc as the basis but 12/31/1899 (well, 1/1/1900). But then, the date value I have isn't even in the range of what [2] shows.
Now if I take the actual value (2000/8/16) and use [1] and [2], I get the following:
method [1]: 2450501 days : (2000 - -4713) * 365 + (8 * 30) + 16
method [2]: 36756 days : [100 * 365 + 8 * 30 + 16] (over counting the # of days)
The dbf file isn't corrupted (otherwise, if I look at the timestamp in dBase, it'd crap out
and display something crazy).
I've thought of using big-endian, but that makes even less sense as the values are even larger. I've even thought of the possibility that it's actually the # of seconds elapsed since either date, but that makes the values with even less sense. i.e. 246598722 = # of seconds elapsed (counting back from 2000/8/16) will make the base year as 1812. (calculations: 246898722 / (3600 * 365) = 187.8985, so 2000 - 187.8985 = 1812.1015)
Can someone point out where I'm doing this wrong?
Thanks!
[1] - https://www.dbase.com/Knowledgebase/INT/db7_file_fmt.htm
[2] - Convert dBase Timestamp
For any dBASE questions, I would recommend to go to the dBASE newsgroups, they have a very helpful and knowledgeable community.
I've finally found the answer thanks to [3].
Basically, the timestamp 8 byte sequence is used as a whole with the following notes:
It's stored in big-endian.
The last byte is not used.
It's a Julian Day Number.
So in my case, it's 0x42ccb20e0340df00 and truncating the last byte,
I get 0x42ccb20e0340df.
Then the following python code gets the correct info:
import datetime
base = 0x42cc418ba99a00
frm_date = int('42ccb20e0340df', 16)
final_ts = (frm_date - base) / 500
final_date = datetime.datetime.utcfromtimestamp(final_ts)
which outputs 2000-8-16 17:21:41 and some milliseconds, which I just ignore.
So I'm guessing the theory is that the above code moves the 'base' date to
1970/1/1 from 1/1/1, which helps since utcfromtimestamp() doesn't
work with any value prior to 1970/1/1.
My confusion stems from the fact it doesn't use 4713BC as the
base year, instead it uses 1/1/1, though I'm still trying to figure out how to get the value 0x42cc418ba99a00 for 1970/1/1.
[3] - https://stackoverflow.com/a/60424157/10860403
I am trying to determine the interval between two dates that I create using DateComponents. If I make the first date 1 year prior to the second, I get 365 days, 0 hours, 0 minutes and 0 seconds. If I make the dates further apart (400 years here), suddenly my date is off by 11 minutes 56 seconds. Here is the code:
import Foundation
var mycal = Calendar(identifier: .iso8601)
var datum = DateComponents(year:1600, month:1, day:1, hour:12, minute:0,
second:0)
let j2000 = DateComponents(year:2000, month:1, day:1, hour:12, minute:0,
second:0)
let datum_date = mycal.date(from: datum)
let j2000_date = mycal.date(from: j2000)
let interval = mycal.dateComponents([.day, .hour, .minute, .second], from:j2000_date!, to:datum_date!)
print("Datum: \(datum_date!)") //1600-01-01 19:48:04 +0000
print("j2000: \(j2000_date!)") //2000-01-01 20:00:00 +0000
Note the next-to-last line: Comments show what the print produces. I've tried it with the Gregorian calendar too, same problem. I'm not sure exactly how far back the inconsistency occurs, I've gone back far enough to produce and it sometimes seems to "stick" as I change the code moving closer in time again. Strangely, the "interval" appears to show the correct amount of days(here -146097), but the date shown is incorrect and I will likely need that in my calculations. Anyone have any ideas?
The difference could be related to leep year adjustments but that would give a difference of 11 minutes and 14 seconds (there'd still be 40 seconds unaccounted for, 26 of which could be leep seconds).
see: https://www.infoplease.com/leap-year-101-next-when-list-days-calendar-years-calculation-last-rules
In Theory, if you compute a multi-year time difference with a precision of minutes and seconds, you should get variations of 5 hours 48 minutes and 46 seconds every 3 out of four years and get within 11 minutes and 14 seconds on the fourth year. I don't know how macOS (Unix) deals with that there there is probably a bunch of considerations that they need to take into account (especially beyond 400 year where that 11 minutes 14 seconds gets adjusted).
If that level of precision is required by your use case, I would suggest reading up on the minute details of time calculations. Given that dates are stored internally as a number of seconds, going back to a precise day and time over these long periods must require some special math acrobatics.
See Apple's documentation here: https://developer.apple.com/reference/foundation/nscalendar
I want to compute simple descriptive statistics (mean, etc) of times when people go to bed. I ran into two problems. The original data comes from an Excel file in which just the time that people went to bed, were typed in - in 24 hrs format. My problem is that r so far doesn't recognizes if people went to bed at 1.00 am the next day. Meaning that a person who went to bed at 10 pm is 3 hrs apart from the one at 1.00 am (and not 21 hrs).
In my dataframe the variable in_bed is a POSIXct format so I thought to apply an if-function telling that if the time is before 12:00 than I want to add 24 hrs.
My function is:
Patr$in_bed <- if(Patr$in_bed <= ) {
Patr$in_bed + 24*60*60
}
My data frame looks like this
in_bed
1 1899-12-30 22:13:00
2 1899-12-30 23:44:00
3 1899-12-30 00:08:00
If I run my function my variable gets deleted and the following error message gets printed:
Warning message:
In if (Patr$in_bed < "1899-12-30 12:00") { :
the condition has length > 1 and only the first element will be used
What do I do wrong or does anyone has a better idea? And can I run commands such as mean on variables in POSIXct format and if not how do I do it?
When you compare Patr$in_bed (vector) and "1899-12-30 12:00" (single value), you get a logical vector. But the IF statement requires a single logical, thus it generates a warning and consider only the first element of the vector.
You can try :
Patr$in_bed <- Patr$in_bed + 24*60*60 * (Patr$in_bed < as.POSIXct("1899-12-30 12:00"))
Explanations : the comparison in the parenthesis will return a logical vector, which will be converted to integer (0 for FALSE and 1 for TRUE). Then the dates for which the statement is true will have +24*60*60, and the others dates will have +0.
But since the POSIXct format includes the date, I don't see the purpose of adding 24 hrs. For instance,
as.POSIXct("1899-12-31 01:00:00") - as.POSIXct("1899-12-30 22:00:00")
returns a time difference of 3 hours, not 21.
To answer your last question, yes you can compute the mean of a POSIXct vector, simply with :
mean(Patr$in_bed)
Hope it helps,
Jérémy
In my dataset I have two timestamp columns. The first is microseconds since application was started - e.g., 1400805323. The second is described as 64bit timestamp which I'm hoping will indicate clock time, using NTP format of number of seconds from 1/1/1901.
Example of '64bit' timestamps:
129518309081725000
129518309082059000
129518309082393000
129518309082727000
129518309083060000
129518309083394000
129518309083727000
Is there any matlab/python code that could convert this into a readable format?
Any help much appreciated,
Steve
Assuming that these values were generated today, June 6th 2011, these values look like number of 100-nanosecond intervals since Jan 1st year 1601. This is how Windows NT stores FILETIME. For more concentrated info on this read this blog post of Raymond Chen. These articles also show how to convert it to anything else
See edit below for updated answer:
For NTP time, the 64bits are broken in to seconds and fraction of seconds. The top 32 bits is the seconds. The bottom 32 bits is the fraction of seconds. You get the fraction by dividing the fraction part by 2^32.
So step one, convert to a double.
If you like python that's easy enough, I didn't add any bounds checking:
def to_seconds(h):
return (h>>32) + ((float)(h&0xffffffff))/pow(2,32)
>>> to_seconds(129518309081725000)
30155831.26845886
The time module can covert that float to a readable time format.
import time
time.ctime(to_seconds(ntp_timestamp))
You'll need to worry about where the timestamp originated though. time.ctime assumes reletive to the Jan 1, 1970. So if your program is basing the ntp formats of time since program run, you'd need to add to the seconds to normalize the timestamp for ctime.
>>> time.ctime(to_seconds(129518309081725000))
'Tue Dec 15 17:37:11 1970'
EDIT:
PyGuy is right, the original timestamps are not ntp time numbers, they are Windows 64 bit timestamps.
Here is the new to_seconds method to convert the 100ns interval based on 1/1/1601 to the 1970 seconds interval:
def to_seconds(h):
s=float(h)/1e7 # convert to seconds
return s-11644473600 # number of seconds from 1601 to 1970
And the new output:
import time
time.ctime(to_seconds(129518309081725000))
'Mon Jun 6 04:48:28 2011'