I want to know when a time series stabilizes after its peak.
This time series shows a peak, and then it goes down (see images below (1); (2)).
I would like to calculate the moment at which this time-series stabilizes (becomes flatter) after its peak.
In an ideal world, the data should go down to 0 and stay there. But as you see, it does not reach 0, nor stay 100% stable.
I thought of various different ways/possibilities:
-Calculate a tangent point to see the slope change. But the data has many small ups & downs even after smoothing it.
-Calculate the average at the tail (end of the series /e.g. if time = 8000; calculate last 2000 values mean average), then calculate an interval (margin +- this value), then calculate the time at which the first value appears in this interval.
-Calculate pronounced changes in the trend or slope.
*Maybe you have a better idea I did not consider. Feel free to share it if you've already dealt with this in the past.
I need to know the time at which this stabilization happened.
Ideally, you could mute/ignore all values before the peak value, but without deleting these rows (time should stay). Calculating the peak is easy (max value).
I also standardized the data so it starts at y=0 (I have various time series, I make them all start at y=0 to compare them later*).
I do not know how to provide the data because it is about 8k values.
I would really appreciate your help.
Thank you very much.
I have lat/lng data of multirotor UAV flights. There are alot of datapoints (~13k per flight) and I wish to find line segments from the data. They give me flight speed and direction. I know that most of the flights are guided missons meaning a point is given to fly to. However the exact points are unknown to me.
Here is a graph of a single flight lat/lng shifted to near (0,0) so they are visible on the same time-series graph.
I attempted to generate similar data, but there are several constraints and it may take more time to solve than working on the segmenting.
The graphs start and end nearly always at the same point.
Horisontal lines mean the UAV is stationary. These segments are expected.
Beginning and and end are always stationary for takeoff and landing.
There is some level of noise in the lines for the gps accuracy tho seemingly not that much.
Alot of data points.
The number of segments is unknown.
The noise I could calculate given the segments and least squares method to the line. Currently I'm thinking of sampling the data (to decimate it a little) and constructing lines. Merging the lines with smaller angle than x (dependant on the noise) and finding the intersection points of the lines left.
Another thought is to try and look at this problem in the frequency domain. The corners should be quite high frequency. Maybe I could make a custom filter kernel that would enable me to use a window function and win in efficency.
EDIT: Rewrote the question for more clarity and less rambling.
I am trying to extract the length and width from a set lidar sensor points (pink) as shown in the image below. The points circled in blue and white are actually noise which I wish to eliminated. [The orange box is the length and width I currently have calculated from the points. As seen, the calculated width is much 1/3 wider than it is supposed to be, due to the noisy points i blue and white]
I've read some approaches to do corner/rectangle fitting, then discarding x% of the poorest fitting points. This would help me to get rid of the circled points. But so far, after many searches I still cannot find any concrete implementations on how to do this fitting.
Can any provide any suggestions how I can go about doing this?
I'm building an app (in Qt) that includes a few graphs in it which are dynamic (meaning refreshes to new values rapidly), and gets there values from a background thread.
I want the first graph, whose details are important refreshing at one speed (100 Hz) and 4 other graphs refreshing in lower speed (10Hz).
The problem is, that when I'm refreshing them all at the same rate (100 Hz) the app can't handle it and the computer stucks, but when the refresh rate is different the first signal gets artifacts on it (comparing to for example running them all an 10Hz).
The artifacts are in the form of waves (instead of straight line for example I get a "snake").
Any suggestions regarding why it has artifacts (rendering limits I guess) and what can be done about it?
I'm writing this as an answer even if this doesn't quite answer your question, because this is too long for a comment.
When the goal is to draw smooth moving graphics, the basic unit of time is frame. At 60 Hz drawing rate, the frame is 16.67 ms. The drawing rate needs to match the monitor drawing rate. Drawing faster than the monitor is totally unnecessary.
When drawing graphs, the movement speed of graph must be kept constant. If you wonder why, walk 1 second fast, then 1 seconds slow, 1 second fast and so on. That doesn't look smooth.
Lets say the data sample rate is 60 Hz and each sample is represented as a one pixel. In each frame all new samples (in this case 1 sample) is drawn and the graph moves one pixel. The movement speed is one pixel per frame, in each frame. The speed is constant, and the graph looks very smooth.
But if the data sample rate is 100 Hz, during one second in 40 frames 2 pixels are drawn and in 20 frames 1 pixel is drawn. Now the graph movement speed is not constant anymore, it varies like this: 2,2,1,2,2,1,... pixels per frame. That looks bad. You might think that frame time is so small (16.67 ms) that you can't see this kind of small variation. But it is very clearly seen. Even single varying speed frames can be seen.
So how is this data of 100 Hz sample rate is drawn smoothly? By keeping the speed constant, in this case it would be 1.67 (100/60) pixels per frame. That of course will require subpixel drawing. So in every frame the graph moves by 1.67 pixels. If some samples are missing at the time of drawing, they are simply not drawn. In practice, that will happen quite often, for example USB data acquisition cards can give the data samples in bursts.
What if the graph drawing is so slow that it cannot be done at 60 Hz? Then the next best option is to draw at 30 Hz. Then you are drawing one frame for every 2 images the monitor draws. The 3rd best option is 20 Hz (one frame for every 3 images the monitor draws), then 15 Hz (one frame for every 4 images) and so on. Drawing at 30 Hz does not look as smooth as drawing at 60 Hz, but the speed can still be kept constant and it looks better than drawing faster with varying speed.
In your case, the drawing rate of 20 Hz would probably be quite good. In each frame there would be 5 new data samples (if you can get the samples at a constant 100 Hz).
I would like to create a digital (square) signal on my sound card. It works great if I generate high frequencies. But, since I can't output DC on a sound card, for lower frequencies the resulting digital bits will all slowly fade to 0.
This is what the soundcards high pass does to my square wave:
http://www.electronics-tutorials.ws/filter/fil39.gif
What's the mathematical function of a signal, that, when passed through a high pass will become square?
Ideally, the solution is demonstrated in gnuplot.
The sound card cuts out the low frequencies in the waveform, so you need to boost those by some amount in what you pass to it.
A square wave contains many frequencies (see the section on the Fourier series here). I suspect the easiest method of generating a corrected square wave is to sum a Fourier series, boosting the amplitudes of the low frequency components to compensate for the high-pass filter in the sound card.
In order to work out how much to boost each low frequency component, you will first need to measure the response of the high-pass filter in your soundcard, by outputting sine waves of various frequencies but constant amplitude, and measuring for each frequency the ratio r(f) of the amplitude of the output to the amplitude of the input. Then, an approximation to a square wave output can be generated by multiplying the amplitude of each frequency component f in the square wave fourier series by 1/r(f) (the 'inverse filter').
It's possible that the high-pass filter in the soundcard also adjusts the phase of the signal. In this case, one might be better off modelling the high pass as an RC filter, (which is probably how the soundcard is doing the filtering), and invert both the amplitude and phase response from that.
Some of the previous answers have correctly noted that it is the high-pass filter (AC coupling capacitor on the soundcard's output) is what is preventing the low frequency square waves from "staying on" so they decay quickly.
There is no way to completely beat this filter from software or it wouldn't be there, now would it? If you can live with lower amplitude square waves at the lower frequencies, you can approximate them by sending out something like a triangle wave. From a transient analysis perspective, the theory of operation here is that as the coupling capacitor is discharging (blocking DC) you are increasing its bias voltage to counteract that discharge thus maintaining the square wave's plateau for a while. Of course you eventually run out of PCM headroom (you can't keep increasing the voltage indefinitely), so a 24-bit card is better in this respect than a 16-bit one as it will give you more resolution. Another, more abstract way to think of this is that the RC filter works as a differentiator, so in order to get the flat peaks of the square wave you need to give it the flat slopes of the triangle wave at the input. But this is an idealized behavior.
As quick proof of concept, here's what a 60Hz ±1V triangle signal becomes when passing through a 1uF coupling cap on a 1Kohm load; it approximates a ±200mV square wave
Note that the impedance/resistance of the load matters quite a bit here; if you lower it to, say, 100ohm the output amplitude decrease dramatically. This is how the coupling caps block DC on speakers/headphone because these devices have much lower impedance than 1Kohm.
If I can find a bit more time later today, I'll add a better simulation, with a better shaped stimulus instead of the simple triangle wave, but I can't get that from your average web-based circuit simulator software...
Well, if you're lucky you can get one of those $0.99 USB sound cards where the manufacturer has cut corners so much that they didn't install coupling caps. https://www.youtube.com/watch?v=4GNRzwfP7RE
Unfourtunately, you cannot get a good approximation of a square wave. Sound hardware is intentionally slew rate limited and would not be able to produce a falling or rising edge beyond its intended frequency range.
You can approximate a badly deformed square wave by alternating a high and low PCM code (+max, -max) every N samples.
You can't actually produce a true square wave, because it has infinite bandwidth. You can produce a reasonable approximation of a square wave though, at frequencies between say 10 Hz and 1 kHz (below 10 Hz you may have problems with the analogue part of your sound card etc, and above around 1 kHz the approximation will become increasingly inaccurate, since you can only reproduce a relatively small number of harmonics).
Tp generate the waveform the sample values will just alternate between +/- some value, e.g. full scale, which would be -32767 and +32767 for a 16 bit PCM stream. The frequency will be determined by the period of these samples. E.g. for a 44.1 kHz sample rate, if you have say 100 samples of -32767 and then 100 samples of +32767, i.e. period = 200 samples, then the fundamental frequency of your square wave will be 44.1 kHz / 200 = 220 Hz.
I found an application that I build on it.
http://www.blogger.com/blogger.g?blogID=999906212197085612#editor/target=post;postID=7722571737880350755
you can generate the format you want and even the pattern you need.
The code uses SLIMDX.