Related to a previous question: I am using Julia Symbolics package and would like to represent 5*pi symbolically. I tried the following to no avail:
using Symbolics
5 * Symbolics.pi # expanded to 15.707963267948966
Num(5) * Symbolics.pi # expanded to 15.707963267948966
#variables x
Symbolics.pi * x # Gives πx, so it works with variables
Desired result, a symbolic expression using Symbolics.pi and the constant that are not numerically calculated on the spot but are kept as a product of 5 and π and show up as 5π, using Symbolics.pi.
Try:
x = Symbolics.pi * Num(5)
print(x)
Here x should be evaluated, hence pi should be promoted.
Related
The well known formula for OLS is (X'X)^(-1)X'y where X is nxK and y is nx1.
One way to implement this in Julia is (X'*X)\X'*y.
But I found that X\y gives the almost same output up to the tiny computational error.
Do they always compute the same thing (as long as n>k)? If so, which one should I use?
When X is square, there is a unique solution and LU-factorization (with pivoting) is a numerically-stable way to calculate this. That is the algorithm that backslash uses in this case.
When X is not square, which is the case in most regression problems, then there is no unique solution but there is a unique least square solution. The QR factorization method for solving Xβ = y is a numerically stable method for generating the least square solution, and in this case X\y uses the QR-factorization and thus gives the OLS solution.
Notice the words numerically stable. While (X'*X)\X'*y will theoretically always give the same result as backslash, in practice backslash (with the correct factorization choice) will be more precise. This is because the factorization algorithms are implemented to be numerically stable. Because of the change for floating point errors to accumulate when doing (X'*X)\X'*y, it's not recommended that you use this form for any real numerical work.
Instead, (X'*X)\X'*y is somewhat equivalent to an SVD factorization which is the most nuemrically stable algorithm, but also the most expensive (in fact, it's basically writing out the Moore-Penrose pseudoinverse which is how an SVD factorization is used to solve a linear system). To directly do an SVD factorization using a pivoted SVD, do svdfact(X) \ y on v0.6 or svd(X) \ y on v0.7. Doing this directly is more stable than (X'*X)\X'*y. Note that qrfact(X) \ y or qr(X) \ y (v0.7) is for QR. See the factorizations portion of the documentation for more details on all of the choices.
Following the documentation the result of X\y is (there notation \(A, B) is used not X and y):
For rectangular A the result is the minimum-norm least squares solution
This is your case I guess as you assume n>k (so your matrix is not square). So you can safely use X\y. Actually it is better to use it than the standard formula as you will get a result even if rank of X is less than min(n,k), whereas standard formula (X'*X)^(-1)*X'*y will fail or produce numerically unstable result if X'*X is nearly singular.
If X would be square (this is not your case) then we have a bit different rule in the documentation:
For input matrices A and B, the result X is such that A*X == B when A is square
This means that the \ algorithm would produce an error if your matrix were singular or produce numerically unstable results if the matrix were nearly singular (in practice most often lu function that is called internally for general dense matrices may throw SingularException).
If you want a catch-all solution (for square and non square matrices) then qr(X, Val(true)) \ y can be used.
Short answer: No, use the first one (the well-known one).
Long answer:
The linear regression model is Xβ = y, and it's easily to derive β = X \ y, which is your second method. However, in most time (when X is not invertible), this is wrong, since you cannot simply left multiply X^-1. The correct way is to solve β = argmin{‖y - Xβ‖^2} instead, which leads to the first method.
To show they are not always the same, simple construct a case where X is not invertible:
julia> X = rand(10, 10)
10×10 Array{Float64,2}:
0.938995 0.32773 0.740556 0.300323 0.98479 0.48808 0.748006 0.798089 0.864154 0.869864
0.973832 0.99791 0.271083 0.841392 0.743448 0.0951434 0.0144092 0.785267 0.690008 0.494994
0.356408 0.312696 0.543927 0.951817 0.720187 0.434455 0.684884 0.72397 0.855516 0.120853
0.849494 0.989129 0.165215 0.76009 0.0206378 0.259737 0.967129 0.733793 0.798215 0.252723
0.364955 0.466796 0.227699 0.662857 0.259522 0.288773 0.691278 0.421251 0.593215 0.542583
0.126439 0.574307 0.577152 0.664301 0.60941 0.742335 0.459951 0.516649 0.732796 0.990509
0.430213 0.763126 0.737171 0.433884 0.85549 0.163837 0.997908 0.586575 0.257428 0.33239
0.28398 0.162054 0.481452 0.903363 0.780502 0.994575 0.131594 0.191499 0.702596 0.0967979
0.42463 0.142 0.705176 0.0481886 0.728082 0.709598 0.630134 0.139151 0.423227 0.942262
0.197805 0.526095 0.562136 0.648896 0.805806 0.168869 0.200355 0.557305 0.69514 0.227137
julia> y = rand(10, 1)
10×1 Array{Float64,2}:
0.7751785556478308
0.24185992335144801
0.5681904264574333
0.9134364924569847
0.20167825754443536
0.5776727022413637
0.05289808385359085
0.5841180308242171
0.2862768657856478
0.45152080383822746
julia> ((X' * X) ^ -1) * X' * y
10×1 Array{Float64,2}:
-0.3768345891121706
0.5900885565174501
-0.6326640292669291
-1.3922334538787071
0.06182039005215956
1.0342060710792016
0.045791973670925995
0.7237081408801955
1.4256831037950832
-0.6750765481219443
julia> X \ y
10×1 Array{Float64,2}:
-0.37683458911228906
0.5900885565176254
-0.6326640292676649
-1.3922334538790346
0.061820390052523294
1.0342060710793235
0.0457919736711274
0.7237081408802206
1.4256831037952566
-0.6750765481220102
julia> X[2, :] = X[1, :]
10-element Array{Float64,1}:
0.9389947787349187
0.3277301697101178
0.7405555185711721
0.30032257202572477
0.9847899425069042
0.48807977638742295
0.7480061513093117
0.79808859136911
0.8641540973071822
0.8698636291189576
julia> ((X' * X) ^ -1) * X' * y
10×1 Array{Float64,2}:
0.7456524759867015
0.06233042922132548
2.5600126098899256
0.3182206475232786
-2.003080524452619
0.272673133766017
-0.8550165639656011
0.40827327221785403
0.2994419115664999
-0.37876151249955264
julia> X \ y
10×1 Array{Float64,2}:
3.852193379477664e15
-2.097948470376586e15
9.077766998701864e15
5.112094484728637e15
-5.798433818338726e15
-2.0446050874148052e15
-3.300267174800096e15
2.990882423309131e14
-4.214829360472345e15
1.60123572911982e15
Question: Is there symbolic ODE solver in R ? (ODE = ordinary differential equation)
I am afraid there is NO... but let me confirm from experts ...
For example, solve:
> (5x-6)^2 y' = 5(5x-6) y - 2
Here: y - unknown function, y' - its derivative
(It is easy to solve by hands: y = 1/(5(5x-6)) + C* (5x-6) , but I want to get that answer from R).
What I know:
1) There are NUMERICAL (not symbolic) solvers:
I know there are numerical ODE solvers like library(deSolve),
see answer here:
Can R language find a generic solution of the first order differential equation?
2) There are symbolic packages : (but they do not seem to contain ODE solvers)
There are symbolic packages in R like
see Ryacas and rSymPy and also some basic symbolic calculation in base R, see:
https://stats.stackexchange.com/questions/4775/symbolic-computation-in-r/4778
3) Brief overview of various differential equations solvers for R:
https://cran.r-project.org/web/views/DifferentialEquations.html
However I was unable to find sumbolic ODE solvers (((
I've had a play around with Ryacas, and you can in fact get symbolic solutions to some simple ODEs without too much work. Unfortunately, YACAS fails to find a solution for your example ODE. However, depending on the ODEs you are exploring, this might still be of use. If not, I'm happy to remove this post.
As an initial simple example, let's consider the following ODE: y'' + y = 0:
Load the library
library(Ryacas);
Since Ryacas is just an interface to YACAS, we can use YACAS' OdeSolve to solve the ODE
yacas("OdeSolve( y\'\' + y == 0 )")
#expression(C70 * exp(x * complex_cartesian(0, -1)) + C74 * exp(x *
# complex_cartesian(0, 1)))
This gives the correct solution const * exp(- ix) + const * exp(+ ix).
Unfortunately when using your particular example, OdeSolve fails to find a solution:
yacas("OdeSolve( y\'\' == (5 * (5 * x - 6) * y - 2) / (5 * x - 6)^2 )")
#expression(y(2) - (5 * ((5 * x - 6) * y(0)) - 2)/(5 * x - 6)^2)
The same happens when we use the YACAS online demo.
I defined the following two functions:
function [z]=f(x,y)
z = x + y - 8
endfunction
function [z]=g(x,y)
z = 2*x + y - 8
endfunction
I then wanted to find the roots of the two functions (equations). That is, I want a pair of numbers (a,b)
such that f(a,b) = g(a,b) = 0. So, I found the function fsolve in the documentation of Scilab which I believe will do what I want. So, I ran the following command:
fsolve([0;0],f,g)
and it produced the following error:
Undefined variable: y
at line 2 of function f called by :
fsolve([0;0],f,g)
I do not understand this error and I am hoping that somebody can tell me what I am doing wrong.
Bob
function c=f(xy),x=xy(1);y=xy(2);c=[(x+y-8);(2*x+y-8)];endfunction
fsolve([0;0],f)
I new to OpenMDAO and I'm still learning how to formulate the problems.
For a simple example, let's say I have 3 input variables with the given bounds:
1 <= x <= 10
0 <= y <= 10
1 <= z <= 10
and I have 4 outputs, defined as:
f1 = x * y
f2 = 2 * z
g1 = x + y - 1
g2 = z
my goal is to minimize f1 * g1, but enforce the constraint f1 = f2 and g1 = g2. For example, one solution is x=3, y=4, z=6 (no idea if this is optimal).
For this simple problem, you can probably just feed the output equality constraints to the driver. However, for my actual problem it's hard to find an initial starting point that satisfy all the constraints, and as the result the optimizer failed to do anything. I figure maybe I could define y and z as states in an implicit component and have a nonlinear solver figure out the right values of y and z given x, then feed x to the optimization driver.
Is this a possible approach? If so, how will the implicit component look like in this case? I looked at the Sellar problem tutorial but I wasn't able to translate it to this case.
You could create an implicit component if you want. In that case, you would define an apply_linear method in your component. That is done with the sellar problem here.
In your case since you have a 2 equation set of residuals which are both dependent on the state variables, I suggest you create a single array state variable of length 2, call it foo (I used a new variable to avoid any confusion, but name it whatever you want!). Then you will define two residuals, one for each element of the residual array of the new state variable.
Something like:
resids['foo'][0] = params['x'] * unknowns['foo'][0] - 2 * unknowns['foo'][1]
resids['foo'][1] = params['x'] + unknowns['foo'][0] - 1 - unknowns['foo'][1]
If you wanted to keep the state variable names separate you could, and it will still work. You'll just have to arbitrarily assign one residual equation to one variable and one to the other.
Then the only thing left is to add a non linear solver to the group containing your implicit component and it should work. If you choose to use a newton solver, you'll either need to set fd_options['force_fd'] = True or define derivatives of your residuals wrt all params and state variables.
I am using D to get derivatives of a function. However, R does not simplify the expression when returning the derivative. I need to figure out if a function has a derivative that can be expressed generically. Is there some way in R to simplify the expression?
> D(expression(sqrt(1 - x^2)), 'x')
-(0.5 * (2 * x * (1 - x^2)^-0.5))
> D(D(expression(sqrt(1 - x^2)), 'x'), 'x')
-(0.5 * (2 * (1 - x^2)^-0.5 - 2 * x * (-0.5 * (2 * x * (1 - x^2)^-1.5))))
Secondly, is there a way in R to do numerical integration?
library(Ryacas)
x <- Sym("x")
Simplify(deriv(sqrt(1 - x^2),x,2)) # return the result simplified
gives
expression((x^2 - 1 - x^2)/root(1 - x^2, 2)^3)
You can also try
PrettyForm(Simplify(deriv(sqrt(1 - x^2),x,2)))
which gives
2 2
x - 1 - x
---------------
3
/ 2 \
Sqrt\ 1 - x /
As for numerical integration try giving this to see what is available
library(sos)
findFn('{numerical+integration}')
As far as I know, R will not simplify the result of D(). It sounds as though you want a proper computer algebra system, and R is definitely not a full CAS. Mathematica and Maple are the most well-known, but there are also a number of open-source alternatives (as discussed on this SO post).
R can do numerical integration - for this kind of question it is worth searching in the R help pages first (i.e. help.search('integrate')). You can use integrate() in the stats package. There is also area() in the MASS package, but that is much simpler (i.e. for demonstration purposes).