Derivation of the equation of backpropagation algorithm - math

Can someone provide a derivation of the equation he gets on 1:15 using quotient rule https://www.youtube.com/watch?v=aVId8KMsdUU&index=18&list=LL2gry7n2BsijUeah-oFnPSg
Pretty simple question, but I get another equation, so I'm stuck.
Thanks in advance.

Your question probably belongs on math.stackexchange.com though.

Related

ckmeans divide and conquer technique

I was going through the ckmeans algorithm https://journal.r-project.org/archive/2011-2/RJournal_2011-2_Wang+Song.pdf
I understood the Dynamic Programming approach to fill the matrices and get clusters. The time complexity is O(kn^2) as described in above PDF.
An improvement of this algorithm exists, which uses divide and conquer technique to fill the matrices. Time complexity reduces to O(knlog(n)) I am unable to understand it from the code that exists for it https://github.com/simple-statistics/simple-statistics/blob/master/src/ckmeans.js
I have understood most of it, but the part where actual log(n) part comes. i.e from line L95 to L123
Please help just understand the gist of what is happening here.

Lagrange polynomial: Unexpected interpolation results

I am trying to interpolate a series of data points using 2nd lagrange polinomial.
having
point1:(5;100)
point2: (9;17)
point3: (12;17)
and the formula
y=(x-x2)*(x-x3)/(x1-x2)*(x1-x3)*y1+
(x-x1)*(x-x3)/(x2-x1)*(x2-x3)*y2+
(x-x1)*(x-x2)/(x3-x1)*(x3-x2)*y3
It is obvious that a quadratic function might not fit the data.. It is just an example.
But i wonder why the value is surprisingly high for x=7.
If i am not wrong its y=1500.
Is the above formula correct?
answer:
In summary:
For the same x, you can't have two different y values; this violates the definition of a function.
you are missing brackets in your formula! Not (x-x2)*(x-x3)/(x1-x2)*(x1-x3), but ((x-x2)*(x-x3)) / ((x1-x2)*(x1-x3)).
back to 1>, note that the interpolation formula has x3-x2 in the denominator. If you have tied values, you will be dividing 0.
How can you make interpolation on such small data set? Yet you are asking for a quadratic interpolation!
follow-up:
1) fixed it. Accidentally i switched all the x and y values. So the points were in format (y,x).
Ah, haha, no wonder.
2) Thank you! The brackets improved the approximation. Regarding the missing brackets: I got the formula from the accepted answer here: Best way to find Quadratic Regression Curve in Java, but I don't understand this rule.
This is the famous, yet fundamental interpolation: Lagrange interpolation. You can verify its formula at Wolfram mathworld: Lagrange Interpolating Polynomial. I don't give you wikipedia link because this one looks more beautiful.
The link you found must contain a typo. Since you have suggest an edit to fix that, hopefully it will soon get approved.
3) It is a (significant larger (which answers your 4th question) time series. So it is impossible to have tied values.
Yes, time series won't have tied values.
formula should be correct.but when x=17,you have two different y value,it's might the cause of the trouble.you can try change anthor x.

Solve Constrained Quadratic Programming with R

I really love R but from time to time it really gives me a headache...
I have the following simple quadratic minimization problem which can be formulated and solved within no time in Excel (click on picture to enlarge):
and
The problem itself is pretty straightforward: I want to minimize (w1^2+w2^2)/2 by finding the best combination of w1,w2 and b under the constraints that for all Y*(w1*X1+w2*X2+b) >= 1
I know that there is the quadprog package for solving these kinds of problems but I find it so unintuitive that I am not able to specify the problem correctly :-( I hate to say it but Excel seems to be better suited for specifying optimization problems like these :-(((
My question
How to formulate the above problem correctly so that it can be solved with R (no matter which package) and the program arrives at the correct values for w1, w2 and b (as can be seen in the picture above). Please don't just post links but please give actual code that works. It would be great if you could comment your code so it becomes clear why you do the things you do. Thank you!
The necessary data is here:
data <- matrix(c(2.947814,6.626878, 1,
2.530388,7.785050, 1,
3.566991,5.651046, 1,
3.156983,5.467077, 1,
2.582346,4.457777,-1,
2.155826,6.222343,-1,
3.273418,3.520687,-1),ncol=3,byrow=T)
colnames(data) <- c("X1","X2","y")
Addendum
Some people took offense at my request to provide code (and not simply links). I apologized for that and gave my reasons that I did not find any good approaches in the answers so far on SO. The deeper reason for that is that the problem is unusual in the sense that b is only in the constraint and not in the objective function. So I still think that this question is a good fit for SO.
Actually, the problem is a little tricky because b is only present in the inequality constraint matrix but not in the objective function. Therefore the matrix in the quadratic programming problem is only positive semidefinite but not positive definite.
My approach is therefore to set the matrix entry corresponding to b to a very small value - in my case 1e-9. Someone else more familiar with such optimization problems might know how to solve the problem properly...
Calculate solve.QP input
c1=data[,"X1"]*data[,"y"]
c2=data[,"X2"]*data[,"y"]
#I use 1e-9 for the b entry
Dmat=matrix(`[<-`(numeric(9),c(1,5,9),c(1,1,1e-9)),3,3)
dvec=rep(0,3)
Amat=cbind(c1,c2,data[,"y"])
bvec=rep(1,nrow(Amat))
Solve with solve.QP
library(quadprog)
sol=solve.QP(Dmat=Dmat,dvec=dvec,Amat=t(Amat),bvec=bvec)$solution
sol
#[1] 2.903910 1.201258 -14.734964
Same as excel.

Logistic equation with or without brackets is correct?

I am coding a logistic function where the function reads:
Now to calculate the % chance of an event happening I have been using:
e^x/1+e^x
now my answer changes significantly depending on if I use:
e^x/1+e^x or e^x/(1+(e^x))
which of these two is correct ?
thanks
Hyflex
Precedence question. The correct, unambiguous answer is,
(e^x)/(1+(e^x))

Can it be proven no polynomial algorithm exists for an NP-Complete prob.?

I can't really seem to grasp what it really means to say a problem is NP-Complete. Could anyone help me with the following question?
An NP-complete problem is a problem for which one can prove that an algorithm for solving it in polynomial time does not exist. Is the statement true?
I would want to say this statement isn't true, because can anyone actually prove that such an algorithm doesn't exist for any NP-Complete problem? From looking around on various sources, I understand that no polynomial time algorithm is known for any NP-Complete problem; however, it can't be proven.
Any help would be greatly appreciated. Thanks.
It is possible in some situations to prove that no algorithm exists that is better than a certain limit.
For example the O(n log n) bound for a comparison sort has been proven. No matter how clever we become in the future, we can be sure that no-one will ever invent an O(n) comparison sort.
In this case though, no-one has found a proof. But that doesn't mean it can't be proven.
The statement is more fundamentally wrong: There are problems that cannot be solved in polynomial time which are much harder than NP problems. The point of NP completeness is a polynomial time solution existing is equivalent to P=NP (which means additionally that a solution not existing means P!=NP).

Resources