Currently I am interning at a software company and one of my tasks has been to implement the recognition of mouse gestures. One of the senior developers helped me get started and provided code/projects that uses the $1 Unistroke Recognizer http://depts.washington.edu/aimgroup/proj/dollar/. I get, in a broad way, what the $1 Unistroke Recognizer is doing and how it works but am a bit overwhelmed with trying to understand all of the internals/finer details of it.
My problem is that I am trying to recognize the gesture of moving the mouse downards, then upwards. The $1 Unistroke Recognizer determines that the gesture I created was a downwards gesture, which is infact what it ought to do. What I really would like it to do is say "I recognize a downards gesture AND THEN an upwards gesture."
I do not know if the lack of understanding of the $1 Unistroke Recognizer completely is causing me to scratch my head, but does anyone have any ideas as to how to recognize two different gestures from moving the mouse downwards then upwards?
Here is my idea that I thought might help me but would love for someone who is an expert or even knows just a bit more than me to let me know what you think. Any help or resources that you know of would be greatly appreciated.
How My Application Currently Works:
The way that my current application works is that I capture points from where the mouse cursor is while the user holds down the left mouse button. A list of points then gets feed to a the gesture recognizer and it then spits out what it thinks to be the best shape/gesture that cooresponds to the captured points.
My Idea:
What I wanted to do is before I feed the points to the gesture recognizer is to somehow go through all the points and break them down into separate lines or curves. This way I could feed each line/curve in one at a time and from the basic movements of down, up, left, right, diagonals, and curves I could determine the final shape/gesture.
One way I thought would be good in determining if there are separate lines in my list of points is sampling groups of points and looking at their slope. If the slope of a sampled group of points differed X% from some other group of sampled points then it would be safe to assume that there is indeed a separate line present.
What I Think Are Possible Problems In My Thinking:
Where do I determine the end of a line and the start of a separate line? If I was to use the idea of checking the slope of a group of points and then determined that there was a separate line present that doesn't mean I nessecarily found the slope of a separate line. For example if you were to draw a straight edged "L" with a right angle and sample the slope of the points around the corner of the "L" you would see that the slope would give resonable indication that there is a separate line present but those points don't correspond to the start of a separate line.
How to deal with the ever changing slope of a curved line? The gesture recognizer that I use handles curves already in the way I want it too. But I don't want my method that I use to determine separate lines keep on looking for these so called separate lines in a curve because its slope is changing all the time when I sample groups of points. Would I just stop sampling points once the slope changed more than X% so many times in a row?
I'm not using the correct "type" of math for determining separate lines. Math isn't my strongest subject but I did do some research. I tried to look into Dot Products and see if that would point me in some direction, but I don't know if it will. Has anyone used Dot Prodcuts for doing something like this or some other method?
Final Thoughts, Remarks, And Thanks:
Part of my problem I feel like is that I don't know how to compeletly ask my question. I wouldn't be surprised if this problem has already been asked (in one way or another) and a solution exist that can be Googled. But my search results on Google didn't provide any solutions as I just don't know exactly how to ask my question yet. If you feel like it is confusing please let me know where and why and I will help clarify it. In doing so maybe my searches on Google will become more precise and I will be able to find a solution.
I just want to say thanks again for reading my post. I know its long but didn't really know where else to ask it. Imma talk with some other people around the office but all of my best solutions I have used throughout school have come from the StackOverflow community so I owe much thanks to you.
Edits To This Post:
(7/6 4:00 PM) Another idea I thought about was comparing all the points before a Min/Max point. For example, if I moved the mouse downards then upwards, my starting point would be the current Max point while the point where I start moving the mouse back upwards would be my min point. I could then go ahead and look to see if there are any points after the min point and if so say that there could be a new potential line. I dunno how well this will work on other shapes like stars but thats another thing Im going to look into. Has anyone done something similar to this before?
If your problem can be narrowed down to breaking apart a general curve into straight or smoothly curved partial lines then you could try this.
Comparing the slope of the segments and identifying breaking points where it is greater then some threshold would work in a very simplified case. Imagine a perfectly formed L-shape where you have a right angle between two straight lines. Obviously the corner point would be the only one where the slope difference is above the threshold as long as the threshold is between 0 and 90 degrees, and thus a identifiable breaking point.
However, the vertical and horizontal lines may be slightly curved so the threshold would need to be large enough for these small differences in slope to be ignored as breaking points. You'd also have to decide how sharp a corner the algorithm should pick up as a break. is 90 deg or higher required, or is even 30 deg enough? This is an important question.
Finally, to make this robust I would not be satisfied comparing the slopes of two adjacent segments. Hands may shake, corners may be smoothed out and the ideal conditions to find straight lines and sharp corners will probably never occur. For each point investigated for a break I would take the average slope of the N previous segments and compare it to the average slope of the N following segments. This can be efficiently implemented using a running mean. By choosing a good sample number N (depending on the accuracy of the input, the total number of points, etc) the algorithm can avoid the noise and make better detections.
Basically the algorithm would be:
For each investigated point (beginning N points into the sequence and ending N points before the end.)
Compute average slope of the N previous segments.
Compute average slope of the N next segments.
If the difference of the averages is greater than the Threshold, mark current point as a breaking point.
This is quite off the top of my head. You'd have to try it in your application.
if you work with absolute angles, like upwards and downwards, you can simply take the absolute slope between two points (not necessarily adjacent) to determine if it's RIGHT, LEFT, UP, DOWN (if that is enough of a distinction)
the art is to find a distance between points so that the angle is not random (with 1px, the angle will be a multiple of 45°)
There is a firefox plugin for Navigation using mouse gestures that works very well. I think it's FireGestures, but I'm not sure. I guess you can get some inspiration from that one
Additional thought: If you draw a shape by connectiong successive points, then connecting back to the first point, the ratio between the area and the final line segment's length is also an indicator for the gesture's "edginess"
If you are just interested in up/down/left/right, a first approximation is to check 45 degree segments of a circle. This is easily done by checking the the horizontal difference between (successive) points against the vertical difference between points.
Say you have a greater positive horizontal difference than vertical difference, then that would be 'RIGHT'.
The only difficulty then comes for example, in distinguishing UP/DOWN from UP/RIGHT/DOWN. But this could be done by distances between points. If you determine that the mouse has moved RIGHT for less than 20 pixels say, then you can ignore that movement.
Related
Problem image
I am trying to align/register (>4) 2-D point cloud segments from several laser scanners with high accuracy, producing an perimeter contour of the scanned product. The segments between lasers may look like that above. The issue is that the calibration process may be both incorrect, and slightly not accurate enough thus I am where I am (and possibly containing individual elevation tilt errors so the segments are not shape-wise similar--close but not exact) and trying to make the best of the situation.
Visually, the segments have a slight bias in both directions as well as a rotational error compared to each other.
The difficulty is that the segments only partially overlap, contain a low but noticeable noise which maybe coherent, and the sampled point distribution is both low and uneven in the overlapping region, since the camera placement are apart (approximately 90 degrees).
My solution so far is to ignore the rotational bias, estimate the mean bias of selected corresponce points within point cloud segments in the overlapping region, and translate each segment by that estimate until I get to the opposite corner. It works somewhat OK, but it is a problem in the last set of sensors since all the errors appear to add up there. Additionally, it fails when there is little or no overlapping region.
I am not a specialist, so complicated solutions maybe useful for others. A relatively robust, iterative approach that can be simply coded is the best! I am thankfully grateful for any advice to solve this simple but quite challenging problem.
I'm currently working towards a 3D model of this, but I thought I would start with 2D. Basically, I have a grid of longitude and latitude with NO2 concentrations across it. What I want to produce, at least for now, is a total amount of Nitrogen Dioxide between two points. Like so:
2DGrid
Basically, These two points are at different lats and lons and as I stated I want to find the amount of something between them. The tricky thing to me is that the model data I'm working with is gridded so I need to be able to account for the amount of something along a line at the lat and lons at which that line cuts through said grid.
Another approach, and maybe a better one for my purposes, could be visualized like this:3DGrid
Ultimately, I'd like to be able to create a program (within any language honestly) that could find the amount of "something" between two points in a 3D grid. If you would like specfics, the bottom altitude is the surface, the top grid is the top of the atmosphere. The bottom point is a measurement device looking at the sun during a certain time of day (and therefore having a certain zenith and azimuth angle). I want to find the NO2 between that measurement device and the "top of the atmosphere" which in my grid is just the top altitude level (of which there are 25).
I'm rather new to coding, stack exchange, and even the subject matter I'm working with so the sparse code I've made might end up creating more clutter than purely asking the question and seeing what methods/code you might suggest?
Hopefully my question is beneficial!
Best,
Taylor
To traverse all touched cells, you can use Amanatides-Woo algorithm. It is suitable both for 2D and for 3D case.
Implementation clues
To account for quantity used from every cell, you can apply some model. For example, calculate path length inside cell (as difference of enter and exit coordinates) and divide by normalizing factor to get cell weight (for example, byCellSize*Sqrt(3) for 3D case as diagonal length).
I am trying to solve a programming interview question that requires one to find the maximum number of points that lie on the same straight straight line in a 2D plane. I have looked up the solutions on the web. All of them discuss a O(N^2) solution using hashing such as the one at this link: Here
I understand the part where common gradient is used to check for co-linear points since that is a common mathematical technique. However, the solution points out that one must beware of vertical lines and overlapping points. I am not sure how these points can cause problems? Can't I just store the gradient of vertical lines as infinity (a large number)?
Hint:
Three distinct points are collinear if
x_1*(y_2-y_3)+x_2*(y_3-y_1)+x_3*(y_1-y_2) = 0
No need to check for slopes or anything else. You need need to eliminate duplicate points from the set before the search begins though.
So pick a pair of points, and find all other points that are collinear and store them in a list of lines. For the remainder points do the same and then compare which lines have the most points.
The first time around you have n-2 tests. The second time around you have n-4 tests because there is no point on revisiting the first two points. Next time n-6 etc, for a total of n/2 tests. At worst case this results in (n/2)*(n/2-1) operations which is O(n^2) complexity.
PS. Who ever decided the canonical answer is using slopes knows very little about planar geometry. People invented homogeneous coordinates for points and lines in a plane for the exact reason of having to represent vertical lines and other degenerate situations.
So I am using Kinect with Unity.
With the Kinect, we detect a hand gesture and when it is active we draw a line on the screen that follows where ever the hand is going. Every update the location is stored as the newest (and last) point in a line. However the lines can often look very choppy.
Here is a general picture that shows what I want to achieve:
With the red being the original line, and the purple being the new smoothed line. If the user suddenly stops and turns direction, we think we want it to not exactly do that but instead have a rapid turn or a loop.
My current solution is using Cubic Bezier, and only using points that are X distance away from each other (with Y points being placed between the two points using Cubic Bezier). However there are two problems with this, amongst others:
1) It often doesn't preserve the curves to the distance outwards the user drew them, for example if the user suddenly stop a line and reverse the direction there is a pretty good chance the line won't extend to point where the user reversed the direction.
2) There is also a chance that the selected "good" point is actually a "bad" random jump point.
So I've thought about other solutions. One including limiting the max angle between points (with 0 degrees being a straight line). However if the point has an angle beyond the limit the math behind lowering the angle while still following the drawn line as best possible seems complicated. But maybe it's not. Either way I'm not sure what to do and looking for help.
Keep in mind this needs to be done in real time as the user is drawing the line.
You can try the Ramer-Douglas-Peucker algorithm to simplify your curve:
https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm
It's a simple algorithm, and parameterization is reasonably intuitive. You may use it as a preprocessing step or maybe after one or more other algorithms. In any case it's a good algorithm to have in your toolbox.
Using angles to reject "jump" points may be tricky, as you've seen. One option is to compare the total length of N line segments to the straight-line distance between the extreme end points of that chain of N line segments. You can threshold the ratio of (totalLength/straightLineLength) to identify line segments to be rejected. This would be a quick calculation, and it's easy to understand.
If you want to take line segment lengths and segment-to-segment angles into consideration, you could treat the line segments as vectors and compute the cross product. If you imagine the two vectors as defining a parallelogram, and if knowing the area of the parallegram would be a method to accept/reject a point, then the cross product is another simple and quick calculation.
https://www.math.ucdavis.edu/~daddel/linear_algebra_appl/Applications/Determinant/Determinant/node4.html
If you only have a few dozen points, you could randomly eliminate one point at a time, generate your spline fits, and then calculate the point-to-spline distances for all the original points. Given all those point-to-spline distances you can generate a metric (e.g. mean distance) that you'd like to minimize: the best fit would result from eliminating points (Pn, Pn+k, ...) resulting in a spline fit quality S. This technique wouldn't scale well with more points, but it might be worth a try if you break each chain of line segments into groups of maybe half a dozen segments each.
Although it's overkill for this problem, I'll mention that Euler curves can be good fits to "natural" curves. What's nice about Euler curves is that you can generate an Euler curve fit by two points in space and the tangents at those two points in space. The code gets hairy, but Euler curves (a.k.a. aesthetic curves, if I remember correctly) can generate better and/or more useful fits to natural curves than Bezier nth degree splines.
https://en.wikipedia.org/wiki/Euler_spiral
I was wondering what kind of model / method / technique Trendly might use to achieve this model:
[It tries to find the moments where significant changes set in and ignores random movements]
Any pointers very welcome! :)
I've never seen 'Trendly', and don't know anything about it, but if I wanted to produce that red line from that blue line, in an algorithmic fashion, I would try:
Fourier the whole data set
Choose a block size longer than the period of the dominant frequency
Divide the data up into blocks of the chosen size
Compare adjacent ones with a statistical test of some sort.
Where the test says two blocks belong to the same underlying distribution, merge them.
If any were merged, go back to 4.
Red trend line is the mean of each block.
A simple "median" function could produce smoother curves over a mostly un-smooth curve.
Otherwise, a brute-force or genetic algorithm could be used; attempting to find the way to split the data into sections, so that more sections = worse solution, and less accuracy of the lines = worse solution.
Another way would be like this: Start at the beginning. As soon as the line moves outside of some radius (3 above or 3 below the first, for instance) set the new height to an average of the current line's height and the previous marker.
If you keep doing that, it would ignore small fluctuations. However, if the fluctuation was large enough, it would still effect it.