I would calculate the first derivative (dpH/dtime) of time series using two variables, time and pH.
Are there any kind of functions to do this in R or should I compute an extra function to do this?
Assuming pH and time are plain vectors try this:
library(pspline)
predict(sm.spline(time, pH), time, 1)
You might want to start with stats::deriv or diff.ts as Matt L suggested. Just keep in mind what a professor of mine used to tell all his students: numeric differentiation is known as "error multiplier."
EDIT:
To clarify -- what he was warning about was that any noise in your data can throw the derivative estimate way off. It's been said that integration is a low-pass filter and differentiation is a high-pass filter.
So, the important thing is to do some smoothing on your data before calculating a derivative. Hence Gabor's excellent suggestion to use predict.spline . But keep in mind that modifying the spline parameters will smooth your data to different levels, so always look at the results to make sure you removed apparent noise but not desired features.
Here's a link to "Numerical Differentiation".
http://en.wikipedia.org/wiki/Numerical_differentiation
Here's a link describing a method based on Taylor Series Expansions:
http://ocw.usu.edu/civil_and_environmental_engineering/numerical_methods_in_civil_engineering/ODEsMatlab.pdf
Related
I am trying to interpolate a series of data points using 2nd lagrange polinomial.
having
point1:(5;100)
point2: (9;17)
point3: (12;17)
and the formula
y=(x-x2)*(x-x3)/(x1-x2)*(x1-x3)*y1+
(x-x1)*(x-x3)/(x2-x1)*(x2-x3)*y2+
(x-x1)*(x-x2)/(x3-x1)*(x3-x2)*y3
It is obvious that a quadratic function might not fit the data.. It is just an example.
But i wonder why the value is surprisingly high for x=7.
If i am not wrong its y=1500.
Is the above formula correct?
answer:
In summary:
For the same x, you can't have two different y values; this violates the definition of a function.
you are missing brackets in your formula! Not (x-x2)*(x-x3)/(x1-x2)*(x1-x3), but ((x-x2)*(x-x3)) / ((x1-x2)*(x1-x3)).
back to 1>, note that the interpolation formula has x3-x2 in the denominator. If you have tied values, you will be dividing 0.
How can you make interpolation on such small data set? Yet you are asking for a quadratic interpolation!
follow-up:
1) fixed it. Accidentally i switched all the x and y values. So the points were in format (y,x).
Ah, haha, no wonder.
2) Thank you! The brackets improved the approximation. Regarding the missing brackets: I got the formula from the accepted answer here: Best way to find Quadratic Regression Curve in Java, but I don't understand this rule.
This is the famous, yet fundamental interpolation: Lagrange interpolation. You can verify its formula at Wolfram mathworld: Lagrange Interpolating Polynomial. I don't give you wikipedia link because this one looks more beautiful.
The link you found must contain a typo. Since you have suggest an edit to fix that, hopefully it will soon get approved.
3) It is a (significant larger (which answers your 4th question) time series. So it is impossible to have tied values.
Yes, time series won't have tied values.
formula should be correct.but when x=17,you have two different y value,it's might the cause of the trouble.you can try change anthor x.
This time my question is more methodological than technical. I have weekly time series data which gets updated every week. Unfortunately the time series is quite volatile. I would thus like to apply a filter/a smoothing method. I tried Hodrick-Prescott and LOESS. Both results look fine, with the downturn that if a new datapoint follows which diverges strongly from the historic data points, the older values have to be revised/are changing. Does somebody know a method which is implemented in R, which could do what I want? A name of a method/a function would probably be completely sufficient. It should however be something more sophisticated than a left sided moving average, because I would not like to lose data at the beginning of the time series. Every helping comment is appreciated! Thank you very much!
Best regards,
Andreas
I think (?) that the term you may be looking for is causal filtering, i.e. filtering that doesn't depend on future values. Within this category probably the simplest/best known approach is exponential smoothing, which is implemented in the forecast and expsmooth packages (library("sos"); findFn("{exponential smoothing}")).
Does that help?
It seems you need a robust two-sided smoother. The problem is that an outlier at an end-point is indistinguishable from a sudden change in the trend. It only becomes clear that it is an outlier after several more observations are collected (and even then some strong assumptions of trend smoothness are required).
I think you will find it hard to do better than loess(), but other functions that aim to do robust smoothing include
smooth() for Tukey's smoothers;
supsmu() for Friedman's super smoother;
Hodrick-Prescott smoothing is not robust to outliers.
I am looking to calculate the indefinite integral of an equation.
I have data from an accelerometer feed into R through a visual C program, and from there it was simple enough to come up with an equation to represent the acceleration curve. That is all well in good, however i need to calculate the impact velocity as well. From my understanding from the good ol' highschool days, the indefinite integral of my acceleration curve will yield the the equation for the velocity.
I know it is easy enough to perform numerical integration with the integrate() function, is there anything which is comparable for an indefinite integral?
library(Ryacas)
x <- Sym("x")
Integrate(sin(x), x)
gives
expression(-cos(x))
An alternative way:
yacas("Integrate(x)Sin(x)")
You can find the function reference here
If the NA's you mention are informative in the sense of indicating no acceleration input then they should be replace by zeros. Let's assume you have the data in acc.vec and the device recorded at a rate of rec_per_sec:
acc.vec[is.na(ac.vec)] <- 0
vel.vec <- cumsum(acc.vec)/recs_per_sec
I do not think constructing a best fit curve is going to improve your accuracy in this instance. To plot velocity versus time:
plot(1:length(acc.vec)/recs_per_sec, vel.vec,
xlab="Seconds", ylab="Integrated Acceleration = Velocity")
As Ben said, try the Ryacas package for calculating the antiderivative of a function. But you probably should ask yourself whether you really want to generate a continuous function which only approximates your data in the first place (fitting errors). I'd stick with numerical integration of your actual data. Keep in mind the uncertainty in each data point, of course.
If I have a system of a springs, not one, but for example 3 degree of freedom system of the springs connected in some with each other. I can make a system of differential equations for but it is impossible to solve it in a general way. The question is, are there any papers or methods for filtering such a complex oscilliations, in order to get rid of the oscilliations and get a real signal as much as possible? For example if I connect 3 springs in some way, and push them to start the vibrations, or put some weight on them, and then take the vibrations from each spring, are there any filtering methods to make it easy to determine the weight (in case if some mass is put above) of each mass? I am interested in filtering complex spring like systems.
Three springs, six degrees of freedom? This is a trivial solution using finite element methods and numerical integration. It's a system of six coupled ODEs. You can apply any form of numerical integration, such as 5th order Runge-Kutta.
I'd recommend doing an eigenvalue analysis of the system first to find out something about its frequency characteristics and normal modes. I'd also do an FFT of the dynamic forces you apply to the system. You don't mention any damping, so if you happen to excite your system at a natural frequency that's close to a resonance you might have some interesting behavior.
If the dynamic equation has this general form (sorry, I don't have LaTeX here to make it look nice):
Ma + Kx = F
where M is the mass matrix (diagonal), a is the acceleration (2nd derivative of displacements w.r.t. time), K is the stiffness matrix, and F is the forcing function.
If you're saying you know the response, you'll have to pre-multiply by the transpose of the response function and try to solve for M. It's diagonal, so you have a shot at it.
Are you connecting the springs in such a way that the behavior of the system is approximately linear? (e.g. at least as close to linear as are musical instrument springs/strings?) Is this behavior consistant over time? (e.g. the springs don't melt or break.) If so, LTI (linear time invariant) systems theory might be applicable. Given enough measurements versus the numbers of degrees of freedom in the LTI system, one might be able to estimate a pole-zero plot of the system response, and go from there. Or something like a linear predictor might be useful.
Actually it is possible to solve the resulting system of differential equations as long as you know the masses, etc.
The standard approach is to use a Laplace Transform. In particular you start with a set of linear differential equations. Add variables until you have a set of first order linear differential equations. (So if you have y'' in your equation, you'd add the equation z = y' and replace y'' with z'.) Rewrite this in the form:
v' = Av + w
where v is a vector of variable, A is a matrix, and w is a scalar vector. (An example of something that winds up in w is gravity.)
Now apply a Laplace transform to get
s L(v) - v(0) = AL(v) + s w
Solve it to get
L(v) = inv(A - I s)(s w + v(0))
where inv inverts a matrix and I is the identity matrix. Apply the inverse Laplace transform (if you read up on Laplace transforms you can find tables of inverse of common types of functions - getting a complete list of the functions you actually encounter shouldn't be that hard), and you have your solution. (Be warned, these computations quickly get very complex.)
Now you have the ability to take a particular setup and solve for the future behavior. You also have the ability to (if you do things really carefully) figure out how the model responds to a small perturbation in parameters. But your problem is that you don't know the parameters to use. However you do have the ability to measure the positions in the system at repeated times.
If you put this together, what you can do is this. Measure your position at a number of points. First estimate all of the initial values of the parameters, and then all of the values a second later. You can adjust your parameters (using Newton's method) to come close enough to the values a second later. Take the measurements from 5 seconds later and use that initial estimate as your starting point to refine your calculations for what is happening 5 seconds later. Repeat with longer intervals to get all of your answers.
Writing and debugging this should take you some time. :-) I would strongly recommend investigating how much of this Mathematica knows how to do for you already...
I'm creating a game where players can make an alloy. To make it less predictable and more interesting, I thought that the durability and hardness of an alloy should not be calculated by a simple formula, because it will be extremely easy to find extrema, where alloy have best statistics.
So the questions is, is there any formula for a function where extrema can be found only by investigating all points? Input values will be in percents: 0.0%-100.0%. I think it should look like this: half sound wave
A very simple way would be a couple of sin function, just vary the constants and the sign for each new player. Here is one example (sin(1.1*x) + sin(x) + sin(0.9 *x))^2
If you use this between 10pi and 20pi you have an by average increasing function with local minima.
Modulating a simple linear or exponential function with trigonometric functions whose frequency and amplitude are dependent on the input should get you what you want.
You don't need a formula, I think — throw a bunch of random values around your domain, and then interpolate (linear interpolation will do) between them. Then you can even change the "formula" completely each time the game is run, or once in a while, or change it slowly with time, etc, etc.
If you want something that is very hard to predict then I would suggest involving a random number generator with the same seed every time. You can use it as an envelope for whatever function you come up with (trig functions or what not) to make it more jagged.
An interesting formula to use would be that of gamma of the Black-Scholes options pricing model. It goes as follows:
You can easily replace the variables, here's a graph of how the function looks:
alt text http://www.sqbimmer.com/aalex/gamma.png