I want to know when a time series stabilizes after its peak.
This time series shows a peak, and then it goes down (see images below (1); (2)).
I would like to calculate the moment at which this time-series stabilizes (becomes flatter) after its peak.
In an ideal world, the data should go down to 0 and stay there. But as you see, it does not reach 0, nor stay 100% stable.
I thought of various different ways/possibilities:
-Calculate a tangent point to see the slope change. But the data has many small ups & downs even after smoothing it.
-Calculate the average at the tail (end of the series /e.g. if time = 8000; calculate last 2000 values mean average), then calculate an interval (margin +- this value), then calculate the time at which the first value appears in this interval.
-Calculate pronounced changes in the trend or slope.
*Maybe you have a better idea I did not consider. Feel free to share it if you've already dealt with this in the past.
I need to know the time at which this stabilization happened.
Ideally, you could mute/ignore all values before the peak value, but without deleting these rows (time should stay). Calculating the peak is easy (max value).
I also standardized the data so it starts at y=0 (I have various time series, I make them all start at y=0 to compare them later*).
I do not know how to provide the data because it is about 8k values.
I would really appreciate your help.
Thank you very much.
Related
I have a time dependent heat conduction simulation and need to plot the average temperature of some area over time. However, the exported table data apparently uses only a few data points and interpolates in between.
More specifically, I have some block of material (aluminum) that is heated periodically at some surface. I am now interested in temperature peaks at exactly this surface over time. I have defined the heating function, the surface, and have calculated the average temperature of the surface under observation over time. However, when I plot the exported data
the temperature data is really, REALLY coarse. The heating data however is very fine. Comsol seems to interpolate between very few points. Calculating with a finer temporal resolution won't fix it.
How can I tell Comsol to evaluate the temperature at every step?
OK, I found the answer:
https://www.comsol.com/support/knowledgebase/1254
Turns out the timesteps the solver chooses are completely separate from the ones the user can define for the simulation. This honestly makes me question the usefulness of the initial definition of the timesteps. It really seems to be just an additional hoop for people to be dependent on support....
Solution:
Turn the maximum timestep in Solution/Time Dependent Solver to an acceptable minimal value.
I have lat/lng data of multirotor UAV flights. There are alot of datapoints (~13k per flight) and I wish to find line segments from the data. They give me flight speed and direction. I know that most of the flights are guided missons meaning a point is given to fly to. However the exact points are unknown to me.
Here is a graph of a single flight lat/lng shifted to near (0,0) so they are visible on the same time-series graph.
I attempted to generate similar data, but there are several constraints and it may take more time to solve than working on the segmenting.
The graphs start and end nearly always at the same point.
Horisontal lines mean the UAV is stationary. These segments are expected.
Beginning and and end are always stationary for takeoff and landing.
There is some level of noise in the lines for the gps accuracy tho seemingly not that much.
Alot of data points.
The number of segments is unknown.
The noise I could calculate given the segments and least squares method to the line. Currently I'm thinking of sampling the data (to decimate it a little) and constructing lines. Merging the lines with smaller angle than x (dependant on the noise) and finding the intersection points of the lines left.
Another thought is to try and look at this problem in the frequency domain. The corners should be quite high frequency. Maybe I could make a custom filter kernel that would enable me to use a window function and win in efficency.
EDIT: Rewrote the question for more clarity and less rambling.
I'm using Graphite and Grafana and I'm trying to plot a series against a time shifted version of itself for comparison.
(I.e. is the current value similar to this time last week?)
What I'd like to do is plot;
the 5 minute moving average of the series
a band consisting of the 5 minute moving average of the series timeshifted by 7 days, bounded above and below by the standard deviation of itself
That way I can see if the current moving average falls within a band limited by the standard deviation of the moving average from a week ago.
I have managed to produce a band based on the timeshifted moving average, but only by offsetting either side by a constant amount. I can't work out any way of offsetting by the standard deviation (or indeed by any dynamic value).
I've copied a screenshot of the sort of thing I'm trying to achieve. The yellow line is the current moving average, the green area is bounded by the historical moving average offset either side by the standard deviation.
Is this possible at all in Grafana using Graphite as the backend?
I'm not quite on the latest version, but can easily upgrade (and will do so shortly anyway).
Incidentally, I'm not a statistician, if what I'm doing actually makes no sense mathematically, I'd love to know! ;-) My overall goal is to explore better alternatives, instead of using static thresholds, for highlighting anomalous or problematic server performance metrics - e.g. CPU load, disk IOPS, etc.
I'm trying to get some information out of a couple of waveforms, which I currently have in the format of a CSV table with the columns: Time, Volume, Pressure, Flow. (The data is the flow/pressure/volume data obtained from a healthcare ventilator.)
I want to get excel / R / another-programme-that-I've-not-yet-thought-of to do the following with the waveform:
Break it up into individual breaths - starting with when the pressure starts to rise above a baseline (shortly followed by the flow rising above 0)
For each breath, extract:The highest pressure that occurs, the pressure just before the start of the next breath, the lowest pressure that occurs
Does anyone know how to do that, without doing it manually? (It'd mean trawling through a few hours-worth of ventilator data for the project I'm trying to do)
I've attached a copy of the shapes of the waves I'm dealing with, to try to help make more sense.Pressure & Volume against time
My intention is to work out the breath-to-breath variability in maximum, minimum, and baseline pressures.
Ultimately, I've come up with my own fix using excel.
For each pressure value, I've taken the mean of that value and the two either side of it. I've then compared this to the previous mean. For my data, a change of this mean of more than 1% in either direction identifies the beginning and end of the up- and down-stroke of the pressure curve, and I can identify where in the curve I am based on the absolute value for pressure (the peak should never be lower than the highest baseline for the data I've collected), and for the slopes, the first direction of change of the mean (this smooths out occasionally inconsistent slopes).
I'm then graphing the data, along with the transition points, the calculated phase of the breath, and the breath count, so I can sense-check it visually.
Ged
Related to this extremely helpful question regarding finding the azimuth & elevation of the sun for a given date, and coordinates. I wish to find the inverse: times & dates the sun will be in that position of the sky.
Therefore I am wondering could someone help with maybe an existing formula or modifying the one linked to.
My current idea was to take two ranges with a variation of a couple of degrees for both, one for the azimuth (120-123 degrees) and elevation (18-21 degrees). Then write an algorithm to iterate through all days / times, and check if the given ranges exist for a time on that day. Looping through these days and using the attached algorithm isn't exactly going to keep Big O small, and also won't be best for performance.
Any help or tips appreciated, please.
Thanks.
There is some useful stuff here (see the links - in particular [12]-[15])
https://en.wikipedia.org/wiki/Position_of_the_Sun
One problem is if you are using this to determine things like "which days would the sun be directly over the 'Heel Stone' at Stonehenge in Z-thousand BC", then there will be a lot of sources of errors beyond precession and/or nutation (earthquakes change the earths rotation period, when the Sun is close to the horizon you'll get some significant refraction). There is also http://www.stargazing.net/kepler/sun.html . However, as there are many days and times when the sun is in a particular position, the method of guessing a window of date and time and then doing a Newton-style approximation iteratively is probably best. Perhaps if you could give more information as to why you are trying to find the answer (i.e."when does the shadow of the oak tree fall on the buried treasure..."), we could be more helpful?
After some thinking you can get the date like this:
if (ang>=0.0) date = (21.March) +ang*(21.June -21.March )/(23.4 degrees);
else date = (21.September)-ang*(21.December-21.September)/(23.4 degrees);
dates are pretty straight forward
ang is the current angle between the ecliptic plane and Earths equator plane
must be measured during day !!!
if you measure the suns height (at your latitude) at astronomical noon then:
ang = height - (90 degrees - your latitude)
to convert height measured at any time you need to apply vector math
see computation of angle between two planes
see image for more clarity
To compute time during the day you will need to look for
conversions between standard (UTC) time and stellar time
also a good idea is to look for solar clock design which includes all computations in geometrical manner.
Do not forget that this approach do not include precession, nutation ...
if you account for that then this task become unsolvable because of sun sky-dome path crossing which leads to multiple solutions for any given suns position
luckily precession is too slow and we can skip it for few thousands years
and nutation has small radius (affect accuracy only)