My objective is to take a map[string]int containing potentially up to a million entries and chunk it in sizes of up to 500 and POST the map to an external service. I'm newer to golang, so I'm tinkering in the Go Playground for now.
Any tips anyone has on how to improve the efficiency of my code base, please share!
Playground: https://play.golang.org/p/eJ4_Pd9X91c
The CLI output I'm seeing is:
original size 60
chunk bookends 0 20
0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,
chunk bookends 20 40
0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,
chunk bookends 40 60
0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,
The problem here is that while the chunk bookends are being calculated correctly, the x value is starting at 0 each time. I think I should expect it to start at the chunk bookend minimum, which would be 0, 20, 40, etc. How come the range is starting at zero each time?
Source:
package main
import (
"fmt"
"math/rand"
"strconv"
)
func main() {
items := make(map[string]int)
// Generate some fake data for our testing, in reality this could be 1m entries
for i := 0; i < 60; i ++ {
// int as strings are intentional here
items[strconv.FormatInt(int64(rand.Int()), 10)] = rand.Int()
}
// Create a map of just keys so we can easily chunk based on the numeric keys
i := 0
keys := make([]string, len(items))
for k := range items {
keys[i] = k
i++
}
fmt.Println("original size", len(keys))
//batchContents := make(map[string]int)
// Iterate numbers in the size batch we're looking for
chunkSize := 20
for chunkStart := 0; chunkStart < len(keys); chunkStart += chunkSize {
chunkEnd := chunkStart + chunkSize
if chunkEnd > len(items) {
chunkEnd = len(items)
}
// Iterate over the keys
fmt.Println("chunk bookends", chunkStart, chunkEnd)
for x := range keys[chunkStart:chunkEnd] {
fmt.Print(x, ",")
// Build the batch contents with the contents needed from items
// #todo is there a more efficient approach?
//batchContents[keys[i]] = items[keys[i]]
}
fmt.Println()
// #todo POST final batch contents
//fmt.Println(batchContents)
}
}
When you process a chunk:
for x := range keys[chunkStart:chunkEnd] {}
You are iterating over a slice, and having one iteration variable, it will be the slice index, not the element from the slice (at the given index). Hence it will always start at 0. (When you iterate over a map, first iteration variable is the key because there is no index there, and the second is the value associated with that key.)
Instead you want this:
for _, key := range keys[chunkStart:chunkEnd] {}
Also note that it's redundant to first collect the keys in a slice, and then process them. You may do that when iterating over the map once, at first. Just keep a variable counting the iterations to know when you reach the chunk size, which may be implicit if you use data structures that keeps this (e.g. the size of a keys batch slice).
For example (try it on the Go Playground):
chunkSize := 20
batchKeys := make([]string, 0, chunkSize)
process := func() {
fmt.Println("Batch keys:", batchKeys)
batchKeys = batchKeys[:0]
}
for k := range items {
batchKeys = append(batchKeys, k)
if len(batchKeys) == chunkSize {
process()
}
}
// Process last, potentially incomplete batch
if len(batchKeys) > 0 {
process()
}
Is there any options in doing date comparison in Go? I have to sort data based on date and time - independently. So I might allow an object that occurs within a range of dates so long as it also occurs within a range of times. In this model, I could not simply just select the oldest date, youngest time/latest date, latest time and Unix() seconds compare them. I'd really appreciate any suggestions.
Ultimately, I wrote a time parsing string compare module to check if a time is within a range. However, this is not faring to well; I've got some gaping issues. I'll post that here just for fun, but I'm hoping there's a better way to time compare.
package main
import (
"strconv"
"strings"
)
func tryIndex(arr []string, index int, def string) string {
if index <= len(arr)-1 {
return arr[index]
}
return def
}
/*
* Takes two strings of format "hh:mm:ss" and compares them.
* Takes a function to compare individual sections (split by ":").
* Note: strings can actually be formatted like "h", "hh", "hh:m",
* "hh:mm", etc. Any missing parts will be added lazily.
*/
func timeCompare(a, b string, compare func(int, int) (bool, bool)) bool {
aArr := strings.Split(a, ":")
bArr := strings.Split(b, ":")
// Catches margins.
if (b == a) {
return true
}
for i := range aArr {
aI, _ := strconv.Atoi(tryIndex(aArr, i, "00"))
bI, _ := strconv.Atoi(tryIndex(bArr, i, "00"))
res, flag := compare(aI, bI)
if res {
return true
} else if flag { // Needed to catch case where a > b and a is the lower limit
return false
}
}
return false
}
func timeGreaterEqual(a, b int) (bool, bool) {return a > b, a < b}
func timeLesserEqual(a, b int) (bool, bool) {return a < b, a > b}
/*
* Returns true for two strings formmated "hh:mm:ss".
* Note: strings can actually be formatted like "h", "hh", "hh:m",
* "hh:mm", etc. Any missing parts will be added lazily.
*/
func withinTime(timeRange, time string) bool {
rArr := strings.Split(timeRange, "-")
if timeCompare(rArr[0], rArr[1], timeLesserEqual) {
afterStart := timeCompare(rArr[0], time, timeLesserEqual)
beforeEnd := timeCompare(rArr[1], time, timeGreaterEqual)
return afterStart && beforeEnd
}
// Catch things like `timeRange := "22:00:00-04:59:59"` which will happen
// with UTC conversions from local time.
// THIS IS THE BROKEN PART I BELIEVE
afterStart := timeCompare(rArr[0], time, timeLesserEqual)
beforeEnd := timeCompare(rArr[1], time, timeGreaterEqual)
return afterStart || beforeEnd
}
So TLDR, I wrote a withinTimeRange(range, time) function but it's not working totally correctly. (In fact, mostly just the second case, where a time range crosses over days is broken. The original part worked, I just realized I'd need to account for that when making conversions to UTC from local.)
If there's a better (preferably built in) way, I'd love to hear about it!
NOTE:
Just as an example, I solved this issue in Javascript with this function:
function withinTime(start, end, time) {
var s = Date.parse("01/01/2011 "+start);
var e = Date.parse("01/0"+(end=="24:00:00"?"2":"1")+"/2011 "+(end=="24:00:00"?"00:00:00":end));
var t = Date.parse("01/01/2011 "+time);
return s <= t && e >= t;
}
However I really want to do this filter server-side.
Use the time package to work with time information in Go.
Time instants can be compared using the Before, After, and Equal
methods. The Sub method subtracts two instants, producing a Duration.
The Add method adds a Time and a Duration, producing a Time.
Play example:
package main
import (
"fmt"
"time"
)
func inTimeSpan(start, end, check time.Time) bool {
return check.After(start) && check.Before(end)
}
func main() {
start, _ := time.Parse(time.RFC822, "01 Jan 15 10:00 UTC")
end, _ := time.Parse(time.RFC822, "01 Jan 16 10:00 UTC")
in, _ := time.Parse(time.RFC822, "01 Jan 15 20:00 UTC")
out, _ := time.Parse(time.RFC822, "01 Jan 17 10:00 UTC")
if inTimeSpan(start, end, in) {
fmt.Println(in, "is between", start, "and", end, ".")
}
if !inTimeSpan(start, end, out) {
fmt.Println(out, "is not between", start, "and", end, ".")
}
}
For comparison between two times use time.Sub()
// utc life
loc, _ := time.LoadLocation("UTC")
// setup a start and end time
createdAt := time.Now().In(loc).Add(1 * time.Hour)
expiresAt := time.Now().In(loc).Add(4 * time.Hour)
// get the diff
diff := expiresAt.Sub(createdAt)
fmt.Printf("Lifespan is %+v", diff)
The program outputs:
Lifespan is 3h0m0s
http://play.golang.org/p/bbxeTtd4L6
For case when your interval's end date doesn't contains hours like
"from 2017-01-01 to whole day of 2017-01-16" it's better to adjust interval's end to midnight of the next day to include all milliseconds like this:
if now.After(start) && now.Before(end.Add(24 * time.Hour).Truncate(24 * time.Hour)) {
...
}
It's possible to compare date using int64 of Unix epoch with seconds granularity. If you need more exact comparison like milisecons or microseconds etc. I guess that
#Oleg Neumyvakin's answer is perfect.
if expirationDate.Unix() > time.Now().Unix() {
...
}
If you're interested in comparing whether a time is close to another for test purposes, you can use testify assert.WithinDuration for this. For example:
expectedTime := time.Now()
actualTime := expectedTime.Add(100*time.Millisecond)
assert.WithinDuration(t, expectedTime, actualTime, 1*time.Second) // pass
assert.WithinDuration(t, expectedTime, actualTime, 1*time.Millisecond) // fail
Otherwise the implementation of assert.WithinDuration can be re-used in your code to determine how close two times are (subtracting one date from the other gives the time difference):
func WithinDuration(expected, actual time.Time, delta time.Duration) bool {
dt := expected.Sub(actual)
return dt >= -delta && dt <= delta
}
Recent protocols prefer usage of RFC3339 per golang time package documentation.
In general RFC1123Z should be used instead of RFC1123 for servers that insist on that format, and RFC3339 should be preferred for new protocols. RFC822, RFC822Z, RFC1123, and RFC1123Z are useful for formatting; when used with time.Parse they do not accept all the time formats permitted by the RFCs.
cutOffTime, _ := time.Parse(time.RFC3339, "2017-08-30T13:35:00Z")
// POSTDATE is a date time field in DB (datastore)
query := datastore.NewQuery("db").Filter("POSTDATE >=", cutOffTime).
As explained in the theread we could use github.com/google/go-cmp/cmp package for dates comparison in tests.
func TestDates(t *testing.T) {
date, _ := time.Parse(time.RFC3339, "2021-11-05T12:00:00+02:00")
dateEqual, _ := time.Parse(time.RFC3339, "2021-11-05T11:00:00+01:00")
dateNotEqual, _ := time.Parse(time.RFC3339, "2021-11-05T12:00:01+02:00")
assertDates(t, date, dateEqual) //pass
assertDates(t, date, dateNotEqual) //fail
}
func assertDates(t *testing.T, expected, actual time.Time) {
t.Helper()
if diff := cmp.Diff(expected, actual); diff != "" {
t.Errorf("mismatch (-expected +actual):\n%s", diff)
}
}
package main
import (
"fmt"
"time"
)
func main() {
fmt.Println("Hello World")
maxRep := 5
repPeroid := 6
expiry := maxRep * repPeroid
fmt.Println("Expiry: ", expiry)
fmt.Println(time.Now())
CorrIdtime := time.Now().Add(time.Second * time.Duration(expiry)).Format(time.RFC3339)
Notifytime := time.Now().Add(2 * time.Second * time.Duration(expiry)).Format(time.RFC3339)
fmt.Println(CorrIdtime)
fmt.Println(Notifytime)
if CorrIdtime < Notifytime {
fmt.Println("Discarded")
} else {
fmt.Println("Accepted")
}
}
Per proposal time: add Time.Compare and related commit, time.Compare will be added in the new release (Go 1.20)
// Compare compares the time instant t with u. If t is before u, it returns -1;
// if t is after u, it returns +1; if they're the same, it returns 0.
func (t Time) Compare(u Time) int {
Sample
var t1, t2 Time
result := t1.Compare(t2)
Given two date ranges, what is the simplest or most efficient way to determine whether the two date ranges overlap?
As an example, suppose we have ranges denoted by DateTime variables StartDate1 to EndDate1 and StartDate2 to EndDate2.
(StartA <= EndB) and (EndA >= StartB)
Proof:
Let ConditionA Mean that DateRange A Completely After DateRange B
_ |---- DateRange A ------|
|---Date Range B -----| _
(True if StartA > EndB)
Let ConditionB Mean that DateRange A is Completely Before DateRange B
|---- DateRange A -----| _
_ |---Date Range B ----|
(True if EndA < StartB)
Then Overlap exists if Neither A Nor B is true -
(If one range is neither completely after the other,
nor completely before the other,
then they must overlap.)
Now one of De Morgan's laws says that:
Not (A Or B) <=> Not A And Not B
Which translates to: (StartA <= EndB) and (EndA >= StartB)
NOTE: This includes conditions where the edges overlap exactly. If you wish to exclude that,
change the >= operators to >, and <= to <
NOTE2. Thanks to #Baodad, see this blog, the actual overlap is least of:
{ endA-startA, endA - startB, endB-startA, endB - startB }
(StartA <= EndB) and (EndA >= StartB)
(StartA <= EndB) and (StartB <= EndA)
NOTE3. Thanks to #tomosius, a shorter version reads:
DateRangesOverlap = max(start1, start2) < min(end1, end2)
This is actually a syntactical shortcut for what is a longer implementation, which includes extra checks to verify that the start dates are on or before the endDates. Deriving this from above:
If start and end dates can be out of order, i.e., if it is possible that startA > endA or startB > endB, then you also have to check that they are in order, so that means you have to add two additional validity rules:
(StartA <= EndB) and (StartB <= EndA) and (StartA <= EndA) and (StartB <= EndB)
or:
(StartA <= EndB) and (StartA <= EndA) and (StartB <= EndA) and (StartB <= EndB)
or,
(StartA <= Min(EndA, EndB) and (StartB <= Min(EndA, EndB))
or:
(Max(StartA, StartB) <= Min(EndA, EndB)
But to implement Min() and Max(), you have to code, (using C ternary for terseness),:
((StartA > StartB) ? StartA : StartB) <= ((EndA < EndB) ? EndA : EndB)
I believe that it is sufficient to say that the two ranges overlap if:
(StartDate1 <= EndDate2) and (StartDate2 <= EndDate1)
This article Time Period Library for .NET describes the relation of two time periods by the enumeration PeriodRelation:
// ------------------------------------------------------------------------
public enum PeriodRelation
{
After,
StartTouching,
StartInside,
InsideStartTouching,
EnclosingStartTouching,
Enclosing,
EnclosingEndTouching,
ExactMatch,
Inside,
InsideEndTouching,
EndInside,
EndTouching,
Before,
} // enum PeriodRelation
For reasoning about temporal relations (or any other interval relations, come to that), consider Allen's Interval Algebra. It describes the 13 possible relations that two intervals can have with respect to each other. You can find other references — "Allen Interval" seems to be an operative search term. You can also find information about these operations in Snodgrass's Developing Time-Oriented Applications in SQL (PDF available online at URL), and in Date, Darwen and Lorentzos Temporal Data and the Relational Model (2002) or Time and Relational Theory: Temporal Databases in the Relational Model and SQL (2014; effectively the second edition of TD&RM).
The short(ish) answer is: given two date intervals A and B with components .start and .end and the constraint .start <= .end, then two intervals overlap if:
A.end >= B.start AND A.start <= B.end
You can tune the use of >= vs > and <= vs < to meet your requirements for degree of overlap.
ErikE comments:
You can only get 13 if you count things funny... I can get "15 possible relations that two intervals can have" when I go crazy with it. By sensible counting, I get only six, and if you throw out caring whether A or B comes first, I get only three (no intersect, partially intersect, one wholly within other). 15 goes like this: [before:before, start, within, end, after], [start:start, within, end, after], [within:within, end, after], [end:end, after], [after:after].
I think that you cannot count the two entries 'before:before' and 'after:after'. I could see 7 entries if you equate some relations with their inverses (see the diagram in the referenced Wikipedia URL; it has 7 entries, 6 of which have a different inverse, with equals not having a distinct inverse). And whether three is sensible depends on your requirements.
----------------------|-------A-------|----------------------
|----B1----|
|----B2----|
|----B3----|
|----------B4----------|
|----------------B5----------------|
|----B6----|
----------------------|-------A-------|----------------------
|------B7-------|
|----------B8-----------|
|----B9----|
|----B10-----|
|--------B11--------|
|----B12----|
|----B13----|
----------------------|-------A-------|----------------------
If the overlap itself should be calculated as well, you can use the following formula:
overlap = max(0, min(EndDate1, EndDate2) - max(StartDate1, StartDate2))
if (overlap > 0) {
...
}
All the solutions that check a multitude of conditions based on where the ranges are in relation to one another can be greatly simplified by simply ensuring that one range starts before or at the same time as the other. You can do this by swapping the ranges if necessary up front.
Then, you can detect overlap if the second range start is:
less than or equal to the first range end (if ranges are inclusive, containing both the start and end times); or
less than (if ranges are inclusive of start and exclusive of end).
For example (assuming inclusive at both ends), there's only four possibilities for range 2, of which one is a non-overlap (the > at the end of the range means it doesn't matter where the range ends):
|-----| range 1, lines below are all range 2.
|--> : overlap.
|--> : overlap.
|---> overlap (no overlap in exclusive-of-end case).
|---> no overlap.
The endpoint of the second range doesn't affect the result at all. So, in pseudo-code, you can do something like (assuming s <= e holds for all ranges - if not, you may have top swap them as well):
def overlaps(r1, r2):
if r1.s > r2.s:
swap r1, r2
return r2.s <= r1.e
Or, the one-level-limit recursive option:
def overlaps(r1, r2):
if r1.s <= r2.s:
return r2.s <= r1.e
return overlaps(r2, r1)
If the ranges are exclusive at the end, you just have to replace <= with < in the expression you return (in both code snippets).
This greatly limits the number of checks you have to make because you remove half of the problem space early by ensuring the first range never starts after the second.
And, since "code talks", here is some Python code that shows this in action, with quite a few test cases. First, the InclusiveRange class:
class InclusiveRange:
"""InclusiveRange class to represent a lower and upper bound."""
def __init__(self, start, end):
"""Initialisation, ensures start <= end.
Args:
start: The start of the range.
end: The end of the range.
"""
self.start = min(start, end)
self.end = max(start, end)
def __repr__(self):
"""Return representation for f-string."""
return f"({self.start}, {self.end})"
def overlaps(self, other):
"""True if range overlaps with another.
Args:
other: The other InclusiveRange to check against.
"""
# Very limited recursion to ensure start of first range
# isn't after start of second.
if self.start > other.start:
return other.overlaps(self)
# Greatly simplified check for overlap.
return other.start <= self.end
Then a test case handler to allow us to nicely present the result of a single test case:
def test_case(range1, range2):
"""Single test case checker."""
# Get low and high value for "graphic" output.
low = min(range1.start, range2.start)
high = max(range1.end, range2.end)
# Output ranges and graphic.
print(f"r1={range1} r2={range2}: ", end="")
for val in range(low, high + 1):
is_in_first = range1.start <= val <= range1.end
is_in_second = range2.start <= val <= range2.end
if is_in_first and is_in_second:
print("|", end="")
elif is_in_first:
print("'", end="")
elif is_in_second:
print(",", end="")
else:
print(" ", end="")
# Finally, output result of overlap check.
print(f" - {range1.overlaps(range2)}\n")
Then finally, a decent chunk of test cases to which you can add your own if need be:
# Various test cases, add others if you doubt the correctness.
test_case(InclusiveRange(0, 1), InclusiveRange(8, 9))
test_case(InclusiveRange(0, 4), InclusiveRange(5, 9))
test_case(InclusiveRange(0, 4), InclusiveRange(4, 9))
test_case(InclusiveRange(0, 7), InclusiveRange(2, 9))
test_case(InclusiveRange(0, 4), InclusiveRange(0, 9))
test_case(InclusiveRange(0, 9), InclusiveRange(0, 9))
test_case(InclusiveRange(0, 9), InclusiveRange(4, 5))
test_case(InclusiveRange(8, 9), InclusiveRange(0, 1))
test_case(InclusiveRange(5, 9), InclusiveRange(0, 4))
test_case(InclusiveRange(4, 9), InclusiveRange(0, 4))
test_case(InclusiveRange(2, 9), InclusiveRange(0, 7))
test_case(InclusiveRange(0, 9), InclusiveRange(0, 4))
test_case(InclusiveRange(0, 9), InclusiveRange(0, 9))
test_case(InclusiveRange(4, 5), InclusiveRange(0, 9))
Running that produces the output:
r1=(0, 1) r2=(8, 9): '' ,, - False
r1=(0, 4) r2=(5, 9): ''''',,,,, - False
r1=(0, 4) r2=(4, 9): ''''|,,,,, - True
r1=(0, 7) r2=(2, 9): ''||||||,, - True
r1=(0, 4) r2=(0, 9): |||||,,,,, - True
r1=(0, 9) r2=(0, 9): |||||||||| - True
r1=(0, 9) r2=(4, 5): ''''||'''' - True
r1=(8, 9) r2=(0, 1): ,, '' - False
r1=(5, 9) r2=(0, 4): ,,,,,''''' - False
r1=(4, 9) r2=(0, 4): ,,,,|''''' - True
r1=(2, 9) r2=(0, 7): ,,||||||'' - True
r1=(0, 9) r2=(0, 4): |||||''''' - True
r1=(0, 9) r2=(0, 9): |||||||||| - True
r1=(4, 5) r2=(0, 9): ,,,,||,,,, - True
where each line has:
the two ranges being evaluated;
a graphical representation of the "range space" (from lowest start to highest end) where each character is a value in that "range space":
' indicates a value in the first range only;
, indicates a value in the second range only;
| indicates a value in both ranges; and
indicates a value in neither range.
the result of the overlap check.
You can see quite clearly that you only get true in the overlap check when there is at least one value in both ranges (i.e., a | character). Every other case gives false.
Feel free to use any other values if you want to add more test cases.
Here is yet another solution using JavaScript. Specialities of my solution:
Handles null values as infinity
Assumes that the lower bound is inclusive and the upper bound exclusive.
Comes with a bunch of tests
The tests are based on integers but since date objects in JavaScript are comparable you can just throw in two date objects as well. Or you could throw in the millisecond timestamp.
Code:
/**
* Compares to comparable objects to find out whether they overlap.
* It is assumed that the interval is in the format [from,to) (read: from is inclusive, to is exclusive).
* A null value is interpreted as infinity
*/
function intervalsOverlap(from1, to1, from2, to2) {
return (to2 === null || from1 < to2) && (to1 === null || to1 > from2);
}
Tests:
describe('', function() {
function generateTest(firstRange, secondRange, expected) {
it(JSON.stringify(firstRange) + ' and ' + JSON.stringify(secondRange), function() {
expect(intervalsOverlap(firstRange[0], firstRange[1], secondRange[0], secondRange[1])).toBe(expected);
});
}
describe('no overlap (touching ends)', function() {
generateTest([10,20], [20,30], false);
generateTest([20,30], [10,20], false);
generateTest([10,20], [20,null], false);
generateTest([20,null], [10,20], false);
generateTest([null,20], [20,30], false);
generateTest([20,30], [null,20], false);
});
describe('do overlap (one end overlaps)', function() {
generateTest([10,20], [19,30], true);
generateTest([19,30], [10,20], true);
generateTest([10,20], [null,30], true);
generateTest([10,20], [19,null], true);
generateTest([null,30], [10,20], true);
generateTest([19,null], [10,20], true);
});
describe('do overlap (one range included in other range)', function() {
generateTest([10,40], [20,30], true);
generateTest([20,30], [10,40], true);
generateTest([10,40], [null,null], true);
generateTest([null,null], [10,40], true);
});
describe('do overlap (both ranges equal)', function() {
generateTest([10,20], [10,20], true);
generateTest([null,20], [null,20], true);
generateTest([10,null], [10,null], true);
generateTest([null,null], [null,null], true);
});
});
Result when run with karma&jasmine&PhantomJS:
PhantomJS 1.9.8 (Linux): Executed 20 of 20 SUCCESS (0.003 secs / 0.004 secs)
Here is the code that does the magic:
var isOverlapping = ((A == null || D == null || A <= D)
&& (C == null || B == null || C <= B)
&& (A == null || B == null || A <= B)
&& (C == null || D == null || C <= D));
Where..
A -> 1Start
B -> 1End
C -> 2Start
D -> 2End
Proof? Check out this test console code gist.
An easy way to remember the solution would be
min(ends)>max(starts)
I would do
StartDate1.IsBetween(StartDate2, EndDate2) || EndDate1.IsBetween(StartDate2, EndDate2)
Where IsBetween is something like
public static bool IsBetween(this DateTime value, DateTime left, DateTime right) {
return (value > left && value < right) || (value < left && value > right);
}
Here's my solution in Java, which works on unbounded intervals too
private Boolean overlap (Timestamp startA, Timestamp endA,
Timestamp startB, Timestamp endB)
{
return (endB == null || startA == null || !startA.after(endB))
&& (endA == null || startB == null || !endA.before(startB));
}
The solution posted here did not work for all overlapping ranges...
----------------------|-------A-------|----------------------
|----B1----|
|----B2----|
|----B3----|
|----------B4----------|
|----------------B5----------------|
|----B6----|
----------------------|-------A-------|----------------------
|------B7-------|
|----------B8-----------|
|----B9----|
|----B10-----|
|--------B11--------|
|----B12----|
|----B13----|
----------------------|-------A-------|----------------------
my working solution was:
AND (
('start_date' BETWEEN STARTDATE AND ENDDATE) -- caters for inner and end date outer
OR
('end_date' BETWEEN STARTDATE AND ENDDATE) -- caters for inner and start date outer
OR
(STARTDATE BETWEEN 'start_date' AND 'end_date') -- only one needed for outer range where dates are inside.
)
As there have been several answers for different languages and environments, here is one for standard ANSI SQL.
In standard SQL it is as as simple as
(StartDate1, EndDate1) overlaps (StartDate2, EndDate2)
assuming all four columns are DATE or TIMESTAMP columns. It returns true if both ranges have at least one day in common (assuming DATE values)
(However not all DBMS products support that)
In PostgreSQL it's also easy to test for inclusion by using date ranges
daterange(StartDate1, EndDate1) #> daterange(StartDate2, EndDate2)
the above returns true if the second range is completely included in the first (which is different to "overlaps")
This was my javascript solution with moment.js:
// Current row dates
var dateStart = moment("2014-08-01", "YYYY-MM-DD");
var dateEnd = moment("2014-08-30", "YYYY-MM-DD");
// Check with dates above
var rangeUsedStart = moment("2014-08-02", "YYYY-MM-DD");
var rangeUsedEnd = moment("2014-08-015", "YYYY-MM-DD");
// Range covers other ?
if((dateStart <= rangeUsedStart) && (rangeUsedEnd <= dateEnd)) {
return false;
}
// Range intersects with other start ?
if((dateStart <= rangeUsedStart) && (rangeUsedStart <= dateEnd)) {
return false;
}
// Range intersects with other end ?
if((dateStart <= rangeUsedEnd) && (rangeUsedEnd <= dateEnd)) {
return false;
}
// All good
return true;
Short answer using momentjs:
function isOverlapping(startDate1, endDate1, startDate2, endDate2){
return moment(startDate1).isSameOrBefore(endDate2) &&
moment(startDate2).isSameOrBefore(endDate1);
}
the answer is based on above answers, but its shortened.
In Microsoft SQL SERVER - SQL Function
CREATE FUNCTION IsOverlapDates
(
#startDate1 as datetime,
#endDate1 as datetime,
#startDate2 as datetime,
#endDate2 as datetime
)
RETURNS int
AS
BEGIN
DECLARE #Overlap as int
SET #Overlap = (SELECT CASE WHEN (
(#startDate1 BETWEEN #startDate2 AND #endDate2) -- caters for inner and end date outer
OR
(#endDate1 BETWEEN #startDate2 AND #endDate2) -- caters for inner and start date outer
OR
(#startDate2 BETWEEN #startDate1 AND #endDate1) -- only one needed for outer range where dates are inside.
) THEN 1 ELSE 0 END
)
RETURN #Overlap
END
GO
--Execution of the above code
DECLARE #startDate1 as datetime
DECLARE #endDate1 as datetime
DECLARE #startDate2 as datetime
DECLARE #endDate2 as datetime
DECLARE #Overlap as int
SET #startDate1 = '2014-06-01 01:00:00'
SET #endDate1 = '2014-06-01 02:00:00'
SET #startDate2 = '2014-06-01 01:00:00'
SET #endDate2 = '2014-06-01 01:30:00'
SET #Overlap = [dbo].[IsOverlapDates] (#startDate1, #endDate1, #startDate2, #endDate2)
SELECT Overlap = #Overlap
the simplest
The simplest way is to use a well-engineered dedicated library for date-time work.
someInterval.overlaps( anotherInterval )
java.time & ThreeTen-Extra
The best in the business is the java.time framework built into Java 8 and later. Add to that the ThreeTen-Extra project that supplements java.time with additional classes, specifically the Interval class we need here.
As for the language-agnostic tag on this Question, the source code for both projects is available for use in other languages (mind their licenses).
Interval
The org.threeten.extra.Interval class is handy, but requires date-time moments (java.time.Instant objects) rather than date-only values. So we proceed by using the first moment of the day in UTC to represent the date.
Instant start = Instant.parse( "2016-01-01T00:00:00Z" );
Instant stop = Instant.parse( "2016-02-01T00:00:00Z" );
Create an Interval to represent that span of time.
Interval interval_A = Interval.of( start , stop );
We can also define an Interval with a starting moment plus a Duration.
Instant start_B = Instant.parse( "2016-01-03T00:00:00Z" );
Interval interval_B = Interval.of( start_B , Duration.of( 3 , ChronoUnit.DAYS ) );
Comparing to test for overlaps is easy.
Boolean overlaps = interval_A.overlaps( interval_B );
You can compare an Interval against another Interval or Instant:
abuts
contains
encloses
equals
isAfter
isBefore
overlaps
All of these use the Half-Open approach to defining a span of time where the beginning is inclusive and the ending is exclusive.
I had a situation where we had dates instead of datetimes, and the dates could overlap only on start/end. Example below:
(Green is the current interval, blue blocks are valid intervals, red ones are overlapping intervals).
I adapted Ian Nelson's answer to the following solution:
(startB <= startA && endB > startA)
|| (startB >= startA && startB < endA)
This matches all overlap cases but ignores the allowed overlap ones.
The mathematical solution given by #Bretana is good but neglects two specific details:
aspect of closed or half-open intervals
empty intervals
About the closed or open state of interval boundaries, the solution of #Bretana valid for closed intervals
(StartA <= EndB) and (EndA >= StartB)
can be rewritten for half-open intervals to:
(StartA < EndB) and (EndA > StartB)
This correction is necessary because an open interval boundary does not belong to the value range of an interval by definition.
And about empty intervals, well, here the relationship shown above does NOT hold. Empty intervals which do not contain any valid value by definition must be handled as special case. I demonstrate it by my Java time library Time4J via this example:
MomentInterval a = MomentInterval.between(Instant.now(), Instant.now().plusSeconds(2));
MomentInterval b = a.collapse(); // make b an empty interval out of a
System.out.println(a); // [2017-04-10T05:28:11,909000000Z/2017-04-10T05:28:13,909000000Z)
System.out.println(b); // [2017-04-10T05:28:11,909000000Z/2017-04-10T05:28:11,909000000Z)
The leading square bracket "[" indicates a closed start while the last bracket ")" indicates an open end.
System.out.println(
"startA < endB: " + a.getStartAsInstant().isBefore(b.getEndAsInstant())); // false
System.out.println(
"endA > startB: " + a.getEndAsInstant().isAfter(b.getStartAsInstant())); // true
System.out.println("a overlaps b: " + a.intersects(b)); // a overlaps b: false
As shown above, empty intervals violate the overlap condition above (especially startA < endB), so Time4J (and other libraries, too) has to handle it as special edge case in order to guarantee that the overlap of any arbitrary interval with an empty interval does not exist. Of course, date intervals (which are closed by default in Time4J but can be half-open, too, like empty date intervals) are handled in a similar way.
This is an extension to the excellent answer by #charles-bretana.
The answer however does not make a distinction among open, closed, and half-open (or half-closed) intervals.
Case 1: A, B are closed intervals
A = [StartA, EndA]
B = [StartB, EndB]
[---- DateRange A ------] (True if StartA > EndB)
[--- Date Range B -----]
[---- DateRange A -----] (True if EndA < StartB)
[--- Date Range B ----]
Overlap iff: (StartA <= EndB) and (EndA >= StartB)
Case 2: A, B are open intervals
A = (StartA, EndA)
B = (StartB, EndB)
(---- DateRange A ------) (True if StartA >= EndB)
(--- Date Range B -----)
(---- DateRange A -----) (True if EndA <= StartB)
(--- Date Range B ----)
Overlap iff: (StartA < EndB) and (EndA > StartB)
Case 3: A, B right open
A = [StartA, EndA)
B = [StartB, EndB)
[---- DateRange A ------) (True if StartA >= EndB)
[--- Date Range B -----)
[---- DateRange A -----) (True if EndA <= StartB)
[--- Date Range B ----)
Overlap condition: (StartA < EndB) and (EndA > StartB)
Case 4: A, B left open
A = (StartA, EndA]
B = (StartB, EndB]
(---- DateRange A ------] (True if StartA >= EndB)
(--- Date Range B -----]
(---- DateRange A -----] (True if EndA <= StartB)
(--- Date Range B ----]
Overlap condition: (StartA < EndB) and (EndA > StartB)
Case 5: A right open, B closed
A = [StartA, EndA)
B = [StartB, EndB]
[---- DateRange A ------) (True if StartA > EndB)
[--- Date Range B -----]
[---- DateRange A -----) (True if EndA <= StartB)
[--- Date Range B ----]
Overlap condition: (StartA <= EndB) and (EndA > StartB)
etc...
Finally, the general condition for two intervals to overlap is
(StartA <🞐 EndB) and (EndA >🞐 StartB)
where 🞐 turns a strict inequality into a non-strict one whenever the comparison is made between two included endpoint.
In case you're using a date range that has not ended yet (still on going) e.g. not set
endDate = '0000-00-00' you can not use BETWEEN because 0000-00-00 is not a valid date!
I used this solution:
(Startdate BETWEEN '".$startdate2."' AND '".$enddate2."') //overlap: starts between start2/end2
OR (Startdate < '".$startdate2."'
AND (enddate = '0000-00-00' OR enddate >= '".$startdate2."')
) //overlap: starts before start2 and enddate not set 0000-00-00 (still on going) or if enddate is set but higher then startdate2
If startdate2 is higher then enddate there is no overlap!
The answer is too simple for me so I have created a more generic dynamic SQL statement which checks to see if a person has any overlapping dates.
SELECT DISTINCT T1.EmpID
FROM Table1 T1
INNER JOIN Table2 T2 ON T1.EmpID = T2.EmpID
AND T1.JobID <> T2.JobID
AND (
(T1.DateFrom >= T2.DateFrom AND T1.dateFrom <= T2.DateTo)
OR (T1.DateTo >= T2.DateFrom AND T1.DateTo <= T2.DateTo)
OR (T1.DateFrom < T2.DateFrom AND T1.DateTo IS NULL)
)
AND NOT (T1.DateFrom = T2.DateFrom)
Using Java util.Date, here what I did.
public static boolean checkTimeOverlaps(Date startDate1, Date endDate1, Date startDate2, Date endDate2)
{
if (startDate1 == null || endDate1 == null || startDate2 == null || endDate2 == null)
return false;
if ((startDate1.getTime() <= endDate2.getTime()) && (startDate2.getTime() <= endDate1.getTime()))
return true;
return false;
}
For ruby I also found this:
class Interval < ActiveRecord::Base
validates_presence_of :start_date, :end_date
# Check if a given interval overlaps this interval
def overlaps?(other)
(start_date - other.end_date) * (other.start_date - end_date) >= 0
end
# Return a scope for all interval overlapping the given interval, including the given interval itself
named_scope :overlapping, lambda { |interval| {
:conditions => ["id <> ? AND (DATEDIFF(start_date, ?) * DATEDIFF(?, end_date)) >= 0", interval.id, interval.end_date, interval.start_date]
}}
end
Found it here with nice explaination ->
http://makandracards.com/makandra/984-test-if-two-date-ranges-overlap-in-ruby-or-rails
The easiest way to do it in my opinion would be to compare if either EndDate1 is before StartDate2 and EndDate2 is before StartDate1.
That of course if you are considering intervals where StartDate is always before EndDate.
If you provide a date range as input and want to find out if it overlaps with the existing date range in database, the following conditions can successfully meet your demand
Assume you provide a #StartDate and #EndDate from your form input.
conditions are :
If #StartDate is ahead of existingStartDate and behind existingEndDate then we can say #StartDate is in the middle of a existing date range, thus we can conclude it will overlap
#StartDate >=existing.StartDate And #StartDate <= existing.EndDate)
If #StartDate is behind existingStartDate but #EndDate is ahead of existingStartDate we can say that it will overlap
(#StartDate <= existing.StartDate And #EndDate >= existing.StartDate)
If #StartDate is behind existingStartDate And #EndDate is ahead of existingEndDate we can conclude that the provided date range devours a existing date range , thus overlaps
(#StartDate <= existing.StartDate And #EndDate >= existing.EndDate))
If any of the condition stands true, your provided date range overlaps with existing ones in the database.
Split the problem into cases then handle each case.
The situation 'two date ranges intersect' is covered by two cases - the first date range starts within the second, or the second date range starts within the first.
You can try this:
//custom date for example
$d1 = new DateTime("2012-07-08");
$d2 = new DateTime("2012-07-11");
$d3 = new DateTime("2012-07-08");
$d4 = new DateTime("2012-07-15");
//create a date period object
$interval = new DateInterval('P1D');
$daterange = iterator_to_array(new DatePeriod($d1, $interval, $d2));
$daterange1 = iterator_to_array(new DatePeriod($d3, $interval, $d4));
array_map(function($v) use ($daterange1) { if(in_array($v, $daterange1)) print "Bingo!";}, $daterange);
public static class NumberExtensionMethods
{
public static Boolean IsBetween(this Int64 value, Int64 Min, Int64 Max)
{
if (value >= Min && value <= Max) return true;
else return false;
}
public static Boolean IsBetween(this DateTime value, DateTime Min, DateTime Max)
{
Int64 numricValue = value.Ticks;
Int64 numericStartDate = Min.Ticks;
Int64 numericEndDate = Max.Ticks;
if (numricValue.IsBetween(numericStartDate, numericEndDate) )
{
return true;
}
return false;
}
}
public static Boolean IsOverlap(DateTime startDate1, DateTime endDate1, DateTime startDate2, DateTime endDate2)
{
Int64 numericStartDate1 = startDate1.Ticks;
Int64 numericEndDate1 = endDate1.Ticks;
Int64 numericStartDate2 = startDate2.Ticks;
Int64 numericEndDate2 = endDate2.Ticks;
if (numericStartDate2.IsBetween(numericStartDate1, numericEndDate1) ||
numericEndDate2.IsBetween(numericStartDate1, numericEndDate1) ||
numericStartDate1.IsBetween(numericStartDate2, numericEndDate2) ||
numericEndDate1.IsBetween(numericStartDate2, numericEndDate2))
{
return true;
}
return false;
}
if (IsOverlap(startdate1, enddate1, startdate2, enddate2))
{
Console.WriteLine("IsOverlap");
}
This was my solution, it returns true when the values don't overlap:
X START 1
Y END 1
A START 2
B END 2
TEST1: (X <= A || X >= B)
&&
TEST2: (Y >= B || Y <= A)
&&
TEST3: (X >= B || Y <= A)
X-------------Y
A-----B
TEST1: TRUE
TEST2: TRUE
TEST3: FALSE
RESULT: FALSE
---------------------------------------
X---Y
A---B
TEST1: TRUE
TEST2: TRUE
TEST3: TRUE
RESULT: TRUE
---------------------------------------
X---Y
A---B
TEST1: TRUE
TEST2: TRUE
TEST3: TRUE
RESULT: TRUE
---------------------------------------
X----Y
A---------------B
TEST1: FALSE
TEST2: FALSE
TEST3: FALSE
RESULT: FALSE