i need to fit some data with sum of exponantial functions, but this shows all the time. Any suggestions how to do the fit?
It just needed to guess the initial value of variables really close to the result of the fit.
Thanks for your answer.
Related
Background: I'm creating a recipe to clean and transform time-series data that will be used by multiple models. One of the steps in the recipe is to remove correlated predictors using the step_corr() function.
However, due to the nature of the data set, some of the variables can have a constant value for the entire set of training data when doing cross-validation using a rolling window and thus cause the step_corr() function to throw a warning.
Problem Statement: In such cases, is it possible to exclude such variables from the correlation step? Or perhaps remove the variable entirely?
P.S. I know I can easily ignore the warning and proceed. But I'm looking for a cleaner approach / best practice advice.
There are two steps for you to consider:
step_zv() will remove variables that all have the same value (zero variance)
step_nzv() will remove variables that almost all have the same value (highly sparse and unbalanced)
I am trying to interpolate a series of data points using 2nd lagrange polinomial.
having
point1:(5;100)
point2: (9;17)
point3: (12;17)
and the formula
y=(x-x2)*(x-x3)/(x1-x2)*(x1-x3)*y1+
(x-x1)*(x-x3)/(x2-x1)*(x2-x3)*y2+
(x-x1)*(x-x2)/(x3-x1)*(x3-x2)*y3
It is obvious that a quadratic function might not fit the data.. It is just an example.
But i wonder why the value is surprisingly high for x=7.
If i am not wrong its y=1500.
Is the above formula correct?
answer:
In summary:
For the same x, you can't have two different y values; this violates the definition of a function.
you are missing brackets in your formula! Not (x-x2)*(x-x3)/(x1-x2)*(x1-x3), but ((x-x2)*(x-x3)) / ((x1-x2)*(x1-x3)).
back to 1>, note that the interpolation formula has x3-x2 in the denominator. If you have tied values, you will be dividing 0.
How can you make interpolation on such small data set? Yet you are asking for a quadratic interpolation!
follow-up:
1) fixed it. Accidentally i switched all the x and y values. So the points were in format (y,x).
Ah, haha, no wonder.
2) Thank you! The brackets improved the approximation. Regarding the missing brackets: I got the formula from the accepted answer here: Best way to find Quadratic Regression Curve in Java, but I don't understand this rule.
This is the famous, yet fundamental interpolation: Lagrange interpolation. You can verify its formula at Wolfram mathworld: Lagrange Interpolating Polynomial. I don't give you wikipedia link because this one looks more beautiful.
The link you found must contain a typo. Since you have suggest an edit to fix that, hopefully it will soon get approved.
3) It is a (significant larger (which answers your 4th question) time series. So it is impossible to have tied values.
Yes, time series won't have tied values.
formula should be correct.but when x=17,you have two different y value,it's might the cause of the trouble.you can try change anthor x.
I'm trying to calculate percentiles (0.3 & 0.7) of a z-score variable (BuildingZ) by factor level (DistrictName). My research so far has pointed me in the direction of the ave() function, allowing me to sort by factor level, but given my "newness" to working in R, I don't really know how to work around this issue. Here's what I have tried:
Create the data frame with the variables I need:
MathGap2=data.frame(MathGap$DistrictName, MathGap$BuildingName, MathGap$Grade, MathGap$BuildingZ)
Use the ave() function to calculate the desired quantile as a new column:
MathGap2$Thirty<-ave(MathGap2$BuildingZ, MathGap2$DistrictName, fun=quantile(MathGap2$BuildingZ, c(0.3)))
I'm not sure if invoking "quantile" is even valid, or whether I'd have to write a function for this (which is beyond my experience). I've seen similar attempts here, but can't get them to work.
P.S. If it's any help, some of the factors may only occur 1-3 times. I'm not sure if this affects the ability to calculate percentiles or not. While this seems silly, ignore the shady math for now; simply trying to replicate an existing study as close as possible.
I would calculate the first derivative (dpH/dtime) of time series using two variables, time and pH.
Are there any kind of functions to do this in R or should I compute an extra function to do this?
Assuming pH and time are plain vectors try this:
library(pspline)
predict(sm.spline(time, pH), time, 1)
You might want to start with stats::deriv or diff.ts as Matt L suggested. Just keep in mind what a professor of mine used to tell all his students: numeric differentiation is known as "error multiplier."
EDIT:
To clarify -- what he was warning about was that any noise in your data can throw the derivative estimate way off. It's been said that integration is a low-pass filter and differentiation is a high-pass filter.
So, the important thing is to do some smoothing on your data before calculating a derivative. Hence Gabor's excellent suggestion to use predict.spline . But keep in mind that modifying the spline parameters will smooth your data to different levels, so always look at the results to make sure you removed apparent noise but not desired features.
Here's a link to "Numerical Differentiation".
http://en.wikipedia.org/wiki/Numerical_differentiation
Here's a link describing a method based on Taylor Series Expansions:
http://ocw.usu.edu/civil_and_environmental_engineering/numerical_methods_in_civil_engineering/ODEsMatlab.pdf
I have data points that represent a logarithmic function.
Is there an approach where I can just estimate the function that describes this data using R?
Thanks.
I assume that you mean that you have vectors y and x and you try do fit a function y(x)=Alog(x).
First of all, fitting log is a bad idea, because it doesn't behave well. Luckily we have x(y)=exp(y/A), so we can fit an exponential function which is much more convenient. We can do it using nonlinear least squares:
nls(x~exp(y/A),start=list(A=1.),algorithm="port")
where start is an initial guess for A. This approach is a numerical optimization, so it may fail.
The more stable way is to transform it to a linear function, log(x(y))=y/A and fit a straight line using lm:
lm(log(x)~y)
If I understand right you want to estimate a function given some (x,y) values of it. If yes check the following links.
Read about this:
http://en.wikipedia.org/wiki/Spline_%28mathematics%29
http://en.wikipedia.org/wiki/Polynomial_interpolation
http://en.wikipedia.org/wiki/Newton_polynomial
http://en.wikipedia.org/wiki/Lagrange_polynomial
Googled it:
http://www.stat.wisc.edu/~xie/smooth_spline_tutorial.html
http://stat.ethz.ch/R-manual/R-patched/library/stats/html/smooth.spline.html
http://www.image.ucar.edu/GSP/Software/Fields/Help/splint.html
I never used R so I am not sure if that works or not, but if you have Matlab i can explain you more.