script to get average based on timestamps - unix

I have two fields in my text file which are
timestamp number
The format of timestamp is hh:mm:ss.mmm
some sample records are
18:31:48.345 0.00345
18:31:49.153 0.00123
18.32:23.399 0.33456
I want to print out averages of records which are no more than 30 second apart. what is a good and fast way of doing it

Here is a starting point in awk. I know you can optimize code better.
count == 0 { startTime = timeToSeconds($1) }
{ currentTime = timeToSeconds($1)
elapsedTime = currentTime - startTime
if (elapsedTime > 30.0) {
calculateAverage()
startTime = timeToSeconds($1)
}
print
sum += $2
count++
}
END { calculateAverage() }
function timeToSeconds(timeString) {
# Convert a time string to number of seconds
split(timeString, tokens, ":")
seconds = tokens[1]*3600.0 + tokens[2]*60.0 + tokens[3]
return seconds
}
function calculateAverage() {
# Use & modify global vars: count, sum
average = sum / count
printf "Average: %.4g\n\n", average
sum = 0.0; count = 0
}

I would start by using some scripting language that has built-in date/time 'operations'. For instance, in Ruby you could easily do:
require 'time'
t,n = gets.chomp.split(/\s+/)
ts1 = Time.parse(t)
# ...
t,n = gets.chomp.split(/\s+/)
ts2 = Time.parse(t)
Which now allows you to do things like:
diff = ts2 - ts1
if diff > 30
# difference is greater than 30 seconds
end
Ruby Time objects can be used in context (float, int, String, etc) so it is trivial to start doing calculations as if the parsed dates are actually numeric values.

Related

Time formatting: how to write a While loop that operates for a whole minute?

I have written the following function:
iterations_per_minute = function() {
Sys.setenv(TZ='GMT+5') ## This line is optional; it just sets my timezone
final_instant = as.numeric(format(Sys.time(), "%H.%M")) + 0.01
counter = 0
while(as.numeric(format(Sys.time(), "%H.%M")) < final_instant) {
counter = counter + 1
}
return(counter)
}
You can infer from the code what the function does, but allow me to explain in lay words anyway: what number can you reach by counting as fast as possible during one minute starting from the number 1? Think of the computer doing exactly that same thing. The output of this function is the number that the computer reaches after counting for a whole minute.
Or at least that is how I would like this function to behave. It does work the way I have described if we pair exactly the call to the function with the beginning of a minute in the clock. However, it will count for less than a minute if we execute this function when the second hand of the clock is pointing at any other number besides twelve. How do I fix this?
Also, I figure I probably won't get the desired output if I execute the function between 23:59 and 0:00. How can this other issue be fixed?
Seems to me like you're trying to introduce more moving parts than you need.
Consider the following:
a <- Sys.time()
a
# [1] "2020-07-25 16:21:40 CDT"
a + 60
# [1] "2020-07-25 16:22:40 CDT"
So, we can just add 60 to Sys.time() without worrying about conversions or whatever else:
iterations_per_minute = function() {
counter = 0
end <- Sys.time() + 60
while( Sys.time() < end ) {
counter = counter + 1
}
return(counter)
}
Using this function, apparently my machine can count to 1474572 in one minute.

How to get a slice of tuples (hour, minute)

I've got a problem that I couldn't resolve. I'm using https://github.com/kmanley/golang-tuple to create tuples.
I've got a list of minutes:
minutes := int{0, 30} // Minutes are 0 and 30
And a four parameters: start, startBreak, stop, stopBreak:
start := tuple.NewTupleFromItems(9, 30) // It represents "9:30"
startBreak := tuple.NewTupleFromItems(12, 0) // It represents "12:00"
stop := tuple.NewTupleFromItems(21, 0) // It represents "21:00"
stopBreak := tuple.NewTupleFromItems(14, 30) // It represents "14:30"
I want to get a slice of tuples (hour, minutes) using all the minutes in the minutes slice and they must not be included in the range startBreak-stopBreak (it can be equal to startBreak or stopBreak, so the range will become 12:30, 13:00, 13:30, 14:00) and stop-start (it can be equal to start and stop, so the range will become 21:30, 22:00, 22:30, ..., 8:30, 9:00).
For example, using those four parameters, the final result will be:
9:30, 10:00, 10:30, 11:00, 11:30, 12:00, 14:30, 15:00, 15:30, 16:00, 16:30, 17:00, 17:30, 18:00, 18:30, 19:00, 19:30, 20:00, 20:30, 21:00
Here is a minimal code that demonstrates this, I did not put any data validation.
func periods(minutes, start, startBreak, stopBreak, stop *tuple.Tuple) (out []tuple.Tuple) {
// next() moves current to the next minute interval
i := 0
curr := tuple.NewTupleFromItems(start.Get(0), minutes.Get(0))
next := func() {
i = (i + 1) % minutes.Len()
curr.Set(1, minutes.Get(i))
if i == 0 {
curr.Set(0, curr.Get(0).(int)+1)
}
}
for ; curr.Le(stop); next() {
if (curr.Ge(start) && curr.Le(startBreak)) || (curr.Ge(stopBreak) && curr.Le(stop)) {
out = append(out, *curr.Copy())
}
}
return out
}
Playground

How can I find the next "available" time slot Interval from a duration?

I have a list of JodaTime Intervals. These Intervals contain time specifications from:
(In military time)
00:00:00 - 10:00:00
11:30:00 - 15:00:00 (3:00:00 PM)
15:30:00 - 23:59:59 (11:59:59 PM).
My issue is that I want to use Interval abstractions from the user to detect what time slot fits the user's desired interval.
For instance, with this code I calculate:
optimizeIntervals(generateMeetingIntervals());
generateBetweenIntervals();
ArrayList<Interval> during = getMeetingDuringIntervals();
ArrayList<Interval> between = getMeetingBetweenIntervals();
Interval desiredDuration = new Interval(now,now.plusMinutes(requestedDuration));
for(int i = 0; i<between.size();i++){
Interval current = between.get(i);
if(current.getEnd().isAfter(now)){
if (current.contains(testing)){
setNextAvailableStart(now);
setNextAvailableEnd(current.getEnd());
}
}
}
However I cannot use Intervals as I need to find the next available interval that fits the user's criteria. If I use Interval for desiredDuration that means, if a user wants a timeslot that is "15 minutes long" I want the algorithm to find the next available 15 minute time slot. However, this algorithm does not work because intervals only check if the start and end times are between the containing interval's start and end times.
Example:
now = 10:30 AM
desiredDuration = 30 minutes
And desiredDuration is calculated as an interval from 10:30 AM to 11:00 AM
loop runs:
is 10:30 AM to 11:00 AM contained within 00:00:00 to 10:00:00? - no
is 10:30 AM to 11:00 AM contained within 11:30:00 to 15:00:00? - no should be yes
However, on the second check I want to just see if there is a duration of 30 minutes available within that time frame. How can I do this?
The problem is, you are using the wrong class; you want to use Duration. The question you are asking is: is there a 30 minute window available. You don't care when it starts, just "does it exist?" Therefore, use Duration, which has no concept of a start time or an end time.
Note that you are using if(current.getEnd().isAfter(now)) anyway to answer whether the interval is after the current period.
So, check each period, generate a possible fit, make sure it's long enough, and then save it if it works. Here's some code:
Duration desiredDuration = new Duration(requestedDuration);
Interval validDuration = null;
for(int i = 0; i<between.size();i++) {
Interval current = between.get(i);
if(current.getEnd().isAfter(now)) {
Interval candidateDuration = current.withDurationAfterStart(desiredDuration);
if(current.contains(candidateDuration)) {
validDuration = candidateDuration;
break;
}
}
}
As #durron597 has said, my issue was using Interval instead of Duration. Here is my implementation that works:
this.meetingData = testMeetingData;
availableNow = false;
DateTime now = caseToTest;
//Calculate range abstractions and optimize them.
optimizeIntervals(generateMeetingIntervals());
generateBetweenIntervals();
ArrayList<Interval> during = getMeetingDuringIntervals();
ArrayList<Interval> between = getMeetingBetweenIntervals();
Interval testing = new Interval(now, now.plusMinutes(requestedDuration));
if (testMeetingData.size()>0) {
for (int i = 0; i < between.size(); i++) {
Interval current = between.get(i);
if (!current.getEnd().isBefore(now) && !current.getEnd().isEqual(now)) {
if (current.toDuration().isLongerThan(testing.toDuration())) {
setNextAvailableStart(current.getStart());
setNextAvailableEnd(current.getEnd());
if (current.contains(testing)) {
setNextAvailableStart(now);
setAvailableNow(true);
}
break;
}
}
}
}
else{
setNextAvailableStart(new DateTime().withTimeAtStartOfDay());
setNextAvailableEnd(new DateTime().withTime(23,59,59,999));
setAvailableNow(true);
}

How to do date/time comparison

Is there any options in doing date comparison in Go? I have to sort data based on date and time - independently. So I might allow an object that occurs within a range of dates so long as it also occurs within a range of times. In this model, I could not simply just select the oldest date, youngest time/latest date, latest time and Unix() seconds compare them. I'd really appreciate any suggestions.
Ultimately, I wrote a time parsing string compare module to check if a time is within a range. However, this is not faring to well; I've got some gaping issues. I'll post that here just for fun, but I'm hoping there's a better way to time compare.
package main
import (
"strconv"
"strings"
)
func tryIndex(arr []string, index int, def string) string {
if index <= len(arr)-1 {
return arr[index]
}
return def
}
/*
* Takes two strings of format "hh:mm:ss" and compares them.
* Takes a function to compare individual sections (split by ":").
* Note: strings can actually be formatted like "h", "hh", "hh:m",
* "hh:mm", etc. Any missing parts will be added lazily.
*/
func timeCompare(a, b string, compare func(int, int) (bool, bool)) bool {
aArr := strings.Split(a, ":")
bArr := strings.Split(b, ":")
// Catches margins.
if (b == a) {
return true
}
for i := range aArr {
aI, _ := strconv.Atoi(tryIndex(aArr, i, "00"))
bI, _ := strconv.Atoi(tryIndex(bArr, i, "00"))
res, flag := compare(aI, bI)
if res {
return true
} else if flag { // Needed to catch case where a > b and a is the lower limit
return false
}
}
return false
}
func timeGreaterEqual(a, b int) (bool, bool) {return a > b, a < b}
func timeLesserEqual(a, b int) (bool, bool) {return a < b, a > b}
/*
* Returns true for two strings formmated "hh:mm:ss".
* Note: strings can actually be formatted like "h", "hh", "hh:m",
* "hh:mm", etc. Any missing parts will be added lazily.
*/
func withinTime(timeRange, time string) bool {
rArr := strings.Split(timeRange, "-")
if timeCompare(rArr[0], rArr[1], timeLesserEqual) {
afterStart := timeCompare(rArr[0], time, timeLesserEqual)
beforeEnd := timeCompare(rArr[1], time, timeGreaterEqual)
return afterStart && beforeEnd
}
// Catch things like `timeRange := "22:00:00-04:59:59"` which will happen
// with UTC conversions from local time.
// THIS IS THE BROKEN PART I BELIEVE
afterStart := timeCompare(rArr[0], time, timeLesserEqual)
beforeEnd := timeCompare(rArr[1], time, timeGreaterEqual)
return afterStart || beforeEnd
}
So TLDR, I wrote a withinTimeRange(range, time) function but it's not working totally correctly. (In fact, mostly just the second case, where a time range crosses over days is broken. The original part worked, I just realized I'd need to account for that when making conversions to UTC from local.)
If there's a better (preferably built in) way, I'd love to hear about it!
NOTE:
Just as an example, I solved this issue in Javascript with this function:
function withinTime(start, end, time) {
var s = Date.parse("01/01/2011 "+start);
var e = Date.parse("01/0"+(end=="24:00:00"?"2":"1")+"/2011 "+(end=="24:00:00"?"00:00:00":end));
var t = Date.parse("01/01/2011 "+time);
return s <= t && e >= t;
}
However I really want to do this filter server-side.
Use the time package to work with time information in Go.
Time instants can be compared using the Before, After, and Equal
methods. The Sub method subtracts two instants, producing a Duration.
The Add method adds a Time and a Duration, producing a Time.
Play example:
package main
import (
"fmt"
"time"
)
func inTimeSpan(start, end, check time.Time) bool {
return check.After(start) && check.Before(end)
}
func main() {
start, _ := time.Parse(time.RFC822, "01 Jan 15 10:00 UTC")
end, _ := time.Parse(time.RFC822, "01 Jan 16 10:00 UTC")
in, _ := time.Parse(time.RFC822, "01 Jan 15 20:00 UTC")
out, _ := time.Parse(time.RFC822, "01 Jan 17 10:00 UTC")
if inTimeSpan(start, end, in) {
fmt.Println(in, "is between", start, "and", end, ".")
}
if !inTimeSpan(start, end, out) {
fmt.Println(out, "is not between", start, "and", end, ".")
}
}
For comparison between two times use time.Sub()
// utc life
loc, _ := time.LoadLocation("UTC")
// setup a start and end time
createdAt := time.Now().In(loc).Add(1 * time.Hour)
expiresAt := time.Now().In(loc).Add(4 * time.Hour)
// get the diff
diff := expiresAt.Sub(createdAt)
fmt.Printf("Lifespan is %+v", diff)
The program outputs:
Lifespan is 3h0m0s
http://play.golang.org/p/bbxeTtd4L6
For case when your interval's end date doesn't contains hours like
"from 2017-01-01 to whole day of 2017-01-16" it's better to adjust interval's end to midnight of the next day to include all milliseconds like this:
if now.After(start) && now.Before(end.Add(24 * time.Hour).Truncate(24 * time.Hour)) {
...
}
It's possible to compare date using int64 of Unix epoch with seconds granularity. If you need more exact comparison like milisecons or microseconds etc. I guess that
#Oleg Neumyvakin's answer is perfect.
if expirationDate.Unix() > time.Now().Unix() {
...
}
If you're interested in comparing whether a time is close to another for test purposes, you can use testify assert.WithinDuration for this. For example:
expectedTime := time.Now()
actualTime := expectedTime.Add(100*time.Millisecond)
assert.WithinDuration(t, expectedTime, actualTime, 1*time.Second) // pass
assert.WithinDuration(t, expectedTime, actualTime, 1*time.Millisecond) // fail
Otherwise the implementation of assert.WithinDuration can be re-used in your code to determine how close two times are (subtracting one date from the other gives the time difference):
func WithinDuration(expected, actual time.Time, delta time.Duration) bool {
dt := expected.Sub(actual)
return dt >= -delta && dt <= delta
}
Recent protocols prefer usage of RFC3339 per golang time package documentation.
In general RFC1123Z should be used instead of RFC1123 for servers that insist on that format, and RFC3339 should be preferred for new protocols. RFC822, RFC822Z, RFC1123, and RFC1123Z are useful for formatting; when used with time.Parse they do not accept all the time formats permitted by the RFCs.
cutOffTime, _ := time.Parse(time.RFC3339, "2017-08-30T13:35:00Z")
// POSTDATE is a date time field in DB (datastore)
query := datastore.NewQuery("db").Filter("POSTDATE >=", cutOffTime).
As explained in the theread we could use github.com/google/go-cmp/cmp package for dates comparison in tests.
func TestDates(t *testing.T) {
date, _ := time.Parse(time.RFC3339, "2021-11-05T12:00:00+02:00")
dateEqual, _ := time.Parse(time.RFC3339, "2021-11-05T11:00:00+01:00")
dateNotEqual, _ := time.Parse(time.RFC3339, "2021-11-05T12:00:01+02:00")
assertDates(t, date, dateEqual) //pass
assertDates(t, date, dateNotEqual) //fail
}
func assertDates(t *testing.T, expected, actual time.Time) {
t.Helper()
if diff := cmp.Diff(expected, actual); diff != "" {
t.Errorf("mismatch (-expected +actual):\n%s", diff)
}
}
package main
import (
"fmt"
"time"
)
func main() {
fmt.Println("Hello World")
maxRep := 5
repPeroid := 6
expiry := maxRep * repPeroid
fmt.Println("Expiry: ", expiry)
fmt.Println(time.Now())
CorrIdtime := time.Now().Add(time.Second * time.Duration(expiry)).Format(time.RFC3339)
Notifytime := time.Now().Add(2 * time.Second * time.Duration(expiry)).Format(time.RFC3339)
fmt.Println(CorrIdtime)
fmt.Println(Notifytime)
if CorrIdtime < Notifytime {
fmt.Println("Discarded")
} else {
fmt.Println("Accepted")
}
}
Per proposal time: add Time.Compare and related commit, time.Compare will be added in the new release (Go 1.20)
// Compare compares the time instant t with u. If t is before u, it returns -1;
// if t is after u, it returns +1; if they're the same, it returns 0.
func (t Time) Compare(u Time) int {
Sample
var t1, t2 Time
result := t1.Compare(t2)

Using Groovy comparison operators with Date objects

I'm investigating an issue and ran across some suspicious code involving comparison of Date instances using comparison operators. e.g.
def stamp = ... //Date
def offset = ... //Integer
def d = new Date(stamp.time + offset)
if (d < new Date()) {
...
}
This resource indicates the above is equivalent to the following
def stamp = ... //Date
def offset = ... //Integer
def d = new Date(stamp.time + offset)
if (d.compareTo(new Date()) < 0) {
...
}
However, the GDK documentation on Dates only has examples comparing dates using compareTo, before, and after and I seem to recall specifically avoiding using the comparison operators on Dates due to an experience with unexpected results. Are the above two code examples indeed equivalent (that is, can I safely use comparison operators on Dates in Groovy, or should I only use compareTo, before, and after)?
Thanks!
Well if you plug them into the handy GroovyConsole they have the same result.
If I understand the question correctly:
def stamp = Date.parse("MM/dd/yyyy","02/02/2010")
def offset = 1213123123
def d = new Date(stamp.time+offset)
if(d < new Date() ) {
println "before"
}
if(d.compareTo(new Date()) < 0) {
println "before"
}
Prints "before" twice
If I switched the stamp date to 2011 lets say it would not print.

Resources