I am using swi-prolog to produce some examples for students in the context of a Prolog course. Regarding unification I want to draw their attention to the dangers of infinite recursion in the unification process. However, mature Prolog implementations like swi-prolog are smart enough to avoid infinite recursion of the unification process in most cases. Is this true in all cases, or can more intricate examples be constructed where unification would still recurse infinitely?
?- foo(bar(X)) = X.
X = foo(bar(X)).
?- foo(X) = X.
X = foo(X).
?- foo(X) = Y, X = Y.
X = Y, Y = foo(Y).
?- foo(X) = Y, X = foo(Y).
X = Y, Y = foo(foo(Y)).
As a related side-question, why does (again, I used swi-prolog) unification binds X to Y in the following example? I didn't expect that.
?- X = f(X), Y = f(Y).
X = Y, Y = f(Y).
It all depends on what you understand by unification algorithm and by Prolog.
Your examples suggest that you are interested in syntactic unification. Here, unification is well defined and terminating as long as it is NSTO (not subject to occurs-check). All Prologs agree here.
Many Prologs offer rational tree unification. SWI offers it since 5.3 of 2004. And also this algorithm is terminating. Given these assumptions, the answer to your question is no, it cannot recurse infinitely.
However, the usefulness of rational trees for programming is rather limited. Its major motivation were efficiency considerations. Namely, variable-term unifications with occurs-check cost up to the size of term, only sometimes they can be reduced to constant cost. But they are always constant for rational trees.
As your interest is rather focused towards teaching, consider to avoid the creation of infinite trees by changing the unification algorithm like so (SWI since 5.6.38, Scryer) :
?- set_prolog_flag(occurs_check, error).
true.
?- X = f(X).
ERROR: =/2: Cannot unify _G2368 with f(_G2368): would create an infinite tree
For the time of development of a program, this flag can be left enabled. And it will help students to locate errors. As long as there is no such error, the program will produce exactly the same result also with rational tree unification.
So much for syntactic unification. In general, in the presence of constraints or coroutines there is no guarantee for termination. Think of
inf :- inf.
?- freeze(X, inf), X = 1.
For your side-question, this is a particularity of SWI's top level which helps to spot identical terms in an answer.
?- X = 1, Y = 1.
X = Y, Y = 1.
Is this true in all cases, or can more intricate examples be constructed where unification would still recurse infinitely?
AFAIK no.
I can't recall seeing where =/2 did not work as expected. Granted my expectations are of how SWI-Prolog does =/2 as noted below.
As a related side-question, why does (again, I used swi-prolog) unification binds X to Y in the following example? I didn't expect that.
See these comments in the SWI-Prolog C code for cyclic terms.
/* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cyclic term unification. The algorithm has been described to me by Bart
Demoen. Here it is (translated from dutch):
I created my own variation. You only need it during general unification.
Here is a short description: suppose you unify 2 terms f(...) and
f(...), which are represented on the heap (=global stack) as:
+-----+ and +-----+
| f/3 | | f/3 |
+-----+ +-----+
args args'
Before working on args and args', change this into the structure below,
using a reference pointer pointing from functor of the one to the other
term.
+-----+ and +-----+
| ----+----------------->| f/3 |
+-----+ +-----+
args args'
If, during this unification you find a compound whose functor is a
reference to the term at the right hand you know you hit a cycle and the
terms are the same.
Of course functor_t must be different from ref. Overwritten functors are
collected in a stack and reset regardless of whether the unification
succeeded or failed.
Note that we need to dereference the functors both left and right.
References at the right are rare, but possible. The trick is to use both
sharing and cycles, where the cycles at the left are shorter:
t :-
X = s(X), Y = y(X,X),
A = s(s(s(A))), B = y(A,A),
Y = B.
While unifying the first argument of y/2, the left-walker crosses to the
right after the first cycle and creates references in A, which are
processed by the right-walker when entering the second argument of y/2.
Initial measurements show a performance degradation for deep unification
of approx. 30%. On the other hand, if subterms appear multiple times in
a term unification can be much faster. As only a small percentage of the
unifications of a realistic program are covered by unify() and involve
deep unification the overall impact of performance is small (< 3%).
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */
Related
Example
sumArray :: Array Int -> State Int Unit
sumArray = traverse_ \n -> modify \sum -> sum + n
t1 :: Int
t1 = execState (do
sumArray [1, 2, 3]
sumArray [4, 5]
sumArray [6]) 0
-- returns 21
module Main where
import Prelude
import Effect (Effect)
import Data.Foldable (fold, traverse_)
import Control.Monad.State (State, execState)
import Control.Monad.State.Class (modify)
import Data.Maybe (Maybe(..))
import Effect.Console (log)
main :: Effect Unit
main = log $ show t1
sumArray :: Array Int -> State Int Unit
sumArray = traverse_ \n -> modify \sum -> sum + n
t1 :: Int
t1 = execState (do
sumArray [1, 2, 3]
sumArray [4, 5]
sumArray [6]) 0
{-
execState :: forall s a. State s a -> s -> s
execState (StateT m) s = case m s of Identity (Tuple _ s') -> s'
type State s = StateT s Identity
-}
Description
How I understand the evaluation of expression t1:
Each sumArray invocation returns a state monad holding the sum of given Int array values.
All three state monads are (somehow) unified into a single one, while accumulating intermediate sums.
execState returns the overall Int sum, given State Int Unit and initial value as input.
Issue
I cannot quite understand step 2 in particular. According to do notation, an expression like sumArray [1, 2, 3] desugars to bind x \_ -> ..., so former input is ignored. If I write do with a different monad type like
t2 :: Maybe Int
t2 = do
Just 3
Just 4
, the compiler complains:
A result of type Int was implicitly discarded in a do notation block. You can use _ <- ... to explicitly discard the result.
, so rules seem to be a bit different with t1.
Question
How exactly are the three separate state monads combined into a single one? More specifically: Why does the runtime calculate the overall sum of all intermediate state monad sum results, and not something like (1+2+3) * (4+5) * 6? In other words: where is the implicit + accumulator?
I feel like I miss some concept from Chapter 11: Monadic Adventures.
#Bergi's answer already gives the essential explanation - but I thought it might be useful to expand on some parts of what he says, and to answer some of your questions more directly.
According to do notation, an expression like sumArray [1, 2, 3]
desugars to bind x \_ -> ..., so former input is ignored.
This is in one sense perfectly true, but also betrays some misunderstandings.
For one thing, I find that quote misleadingly phrased - although it's perfectly acceptable in the context of the original source. It's not talking about how an expression like sumArray [1, 2, 3] "desugars" in and of itself, but about how successive lines ("statements") of a do block are desugared into a single expression that "combines" them - which seems to be in essence what your whole question is about. So yes, it's true - and basically the definition of do notation - that an expression like
do a <- x
y
desugars to bind x \a -> y (where we imagine that y is some more complex expression which presumably involves a). And likewise that
do x
y
desugars to bind x \_ -> y. But the latter case isn't "ignoring input" - it's ignoring output. Let me explain this a little more.
It's common to think of a general monadic value, of type m a, to be some sort of "computation" that "produces" value(s) of type a. That's necessarily quite an abstract formulation - because a Monad is such a general concept, and some specific Monads fit this mental picture better than others. But it's a good way to understand the basics of monads, and do notation in particular - each line can be thought of as a "statement" in some imperative language, which may have some "side effects" (of a kind strictly constrained by the particular monad you're using) and also produces a value as a "result".
In this sense, a do block of the first type above - where we "bind" the result using the "left arrow" notation - is using that computed value (denoted by a) to decide what to do next. (Incidentally, this is what distinguishes monads from applicatives - if you just have a series of computations and just want to combine their "effects", without allowing "intermediate results" to affect what you're doing, you don't actually need monads or bind.) Whereas the second one doesn't use the result of the first computation (that computation being x) - which is exactly what I meant when I said this is "ignoring output". It's ignoring the result of x. That doesn't (necessarily) mean that x is useless though. It's still being used for its "side effects".
To make it more concrete, I'll look in a bit more detail at both of your examples, starting with the simple one in the Maybe monad (I'll make the change that the compiler suggests in order to keep it happy - note that I'm personally much more familiar with Haskell than with Purescript so I may get Purescript-specific things like this wrong, as Haskell would be perfectly OK with your original code):
t2 :: Maybe Int
t2 = do
_ <- Just 3
Just 4
In this case, t2 will simply be equal to Just 4, and it will seem - correctly - that the first line of the do block is redundant. But that's just a consequence of how the Maybe monad works, as well as of the specific value we've got there. I can easily prove to you that the first line does still matter though, by making this change
t2 :: Maybe Int
t2 = do
_ <- Nothing
Just 4
Now you will find that t2 is equal not to Just 4, not to Nothing!
That's because each "computation" in the Maybe monad - that is, each value of type Maybe a - either "succeeds" with a "result" of type a (represented by a Just value), or "fails" (represented by Nothing). And, importantly, the way the Maybe monad is defined - that is, the definition of bind - deliberately propagates failure. That is, any Nothing value that is encountered at any point immediately terminates the computation with a Nothing result.
So even here, the "side effect" of the first computation - the fact that it succeeds or fails - does make a major difference to what happens overall. We just ignore the "result" (the actual value if the computation was successful).
If we now move to the State monad - this is a somewhat more complicated monad than Maybe, but may actually for that reason make the above points easier to understand. Because this is a monad where it really does make immediate sense to talk about the "side effects" and the "result" of each monadic value - which perhaps felt a bit forced, or even silly, in the Maybe case.
A value of type State s a represents a computation that results in a value of type a, while "keeping some state" of type s. That is, the computation may use the current state in order to compute its result, and/or it may update the state as part of the computation. Concretely, this is the same as a function of type s -> (a, s) - it takes some state, and returns an updated state (possibly the same) as well as the computed value. And indeed, the State s a type is essentially a simple newtype wrapper for such a function type.
And the implementation of bind in its Monad instance does the most obvious and natural thing - much easier to explain in words than to "see" from the actual implementation details. Two such "stateful functions" are combined by feeding the original state to the first function, then taking the updated state from that and feeding that to the second function. (Actually, bind needs to do - and does - more than this, because as I mentioned earlier, it needs to be able to use the "result" - the a - from the first computation to decide what to do with the second. But we don't need to go into that now, because in this example we don't use the result value - and indeed couldn't, as it's always of the trivial Unit type. It isn't actually complicated, but I won't go into the detail as I don't want to make this answer even longer!)
So when we do
do
sumArray [1, 2, 3]
sumArray [4, 5]
sumArray [6]
we are building a stateful computation of type State Int Unit - that is, a function of type Int -> (Unit, Int). Since Unit is a non-interesting type, and basically used as a placeholder here for "we don't care about any result", we're basically building a function of type Int -> Int from three other such functions. That's easy enough to do - we can just compose the three functions! And that's what, in this simple case, the implementation of bind for the State monad ends up doing.
Hopefully this answers your main question:
where is the implicit + accumulator?
by showing that there's no "implicit accumulator" other than function composition. It's the fact that those individual functions happen to add (respectively, in this case) 6, 9 and 6 to the input that results in the final result being the sum of those 3 numbers (due to the fact that the composition of two sums is itself a sum, which ultimately comes from the associativity of addition).
But more importantly, I hope this has given you a fuller explanation of Monads, and do notation, which you can apply to many other situations.
Each sumArray invocation returns a state monad holding the sum of given Int array values.
No, it doesn't return "a state monad", and it doesn't hold the sum of the array.
It returns a State Int Unit value, which represents a "stateful computation of a Unit" (using an Int for the state). To get the sum, you actually have to run that computation:
t :: State Int Unit
t = sumArray [1, 2, 3]
x = runState t 0 // ((), 6)
y = runState t 5 // ((), 11)
Notice how for the value y, the sum of the array elements is never computed - it adds 1 to 5, then 2 to 6, then 3 to 8.
How exactly are the three separate state monads combined into a single one?
The key to understanding is that they are not state values, but stateful computations. They can be combined by simply sequencing those computations one after the other, passing the result and state on to the next.
Here is my definition of power for integer exponents following this mailing-list post:
definition
"ipow x n = (if n < 0 then (1 / x) ^ n else x ^ n)"
notation ipow (infixr "^⇩i" 80)
Is there a better way to define it?
Is there an existing theory in Isabelle that already includes it so that I can reuse its results?
Context
I am dealing with complex exponentials, for instance consider this theorem:
after I proved it I realized I need to work with integers n not just naturals and this involves using powers to take out the n from the exponential.
I don't think something like this exists in the library. However, you have a typo in your definition. I believe you want something like
definition
"ipow x n = (if n < 0 then (1 / x) ^ nat (-n) else x ^ nat n)"
Apart from that, it is fine. You could write inverse x ^ nat (-n), but it should make little difference in practice. I would suggest the name int_power since the corresponding operation with natural exponents is called power.
Personally, I would avoid introducting a new constant like this because in order to actually use it productively, you also need an extensive collection of theorems around it. This means quite a bit of (tedious) work. Do you really need to talk about integers here? I find that one can often get around it in practice (in particular, note that the exponentials in question are periodic anyway).
It may be useful to introduce such a power operator nevertheless; all I'm saying is you should be aware of the trade-off.
Side note: An often overlooked function in Isabelle that is useful when talking about exponentials like this is cis (as in ‘cosine + i · sine‘). cis x is equivalent to ‘exp(ix)’ where x is real.
Is there a convergence theory in Isabelle/HOL? I need to define ∥x(t)∥ ⟶ 0 as t ⟶ ∞.
Also, I'm looking for vectors theory, I found a matrix theory but I couldn't find the vectors one, Is there exist such theory in Isabelle/HOL?
Cheers.
Convergence etc. are expressed with filters in Isabelle. (See the corresponding documentation)
In your case, that would be something like
filterlim (λt. norm (x t)) (nhds 0) at_top
or, using the tendsto abbreviation,
((λt. norm (x t)) ⤏ 0) at_top
where ⤏ is the Isabelle symbol \<longlongrightarrow>, which can be input using the abbreviation --->.
As a side note, I am wondering why you are writing it that way in the first place, seeing as it is equivalent to
filterlim x (nhds 0) at_top
or, with the tendsto syntax:
(x ⤏ 0) at_top
Reasoning with these filters can be tricky at first, but it has the advantage of providing a unified framework for limits and other topological concepts, and once you get the hang of it, it is very elegant.
As for vectors, just import ~~/src/HOL/Analysis/Analysis. That should have everything you need. Ideally, build the HOL-Analysis session image by starting Isabelle/jEdit with isabelle jedit -l HOL-Analysis. Then you won't have to process all of Isabelle's analysis library every time you start the system.
I assume that by ‘vectors’ you mean concrete finite-dimensional real vector spaces like ℝn. This is provided by ~~/src/HOL/Analysis/Finite_Cartesian_Product.thy, which is part of HOL-Analysis. This provides the vec type, which takes two parameters: the component type (probably real in your case) and the index type, which specifies the dimension of the vector space.
There is also a pre-defined type n for every positive integer n, so that you can write e.g. (real, 3) vec for the vector space ℝ³. There is also type syntax so that you can write 'a ^ 'n for ('a, 'n) vec.
Pure functional programming languages do not allow mutable data, but some computations are more naturally/intuitively expressed in an imperative way -- or an imperative version of an algorithm may be more efficient. I am aware that most functional languages are not pure, and let you assign/reassign variables and do imperative things but generally discourage it.
My question is, why not allow local state to be manipulated in local variables, but require that functions can only access their own locals and global constants (or just constants defined in an outer scope)? That way, all functions maintain referential transparency (they always give the same return value given the same arguments), but within a function, a computation can be expressed in imperative terms (like, say, a while loop).
IO and such could still be accomplished in the normal functional ways - through monads or passing around a "world" or "universe" token.
My question is, why not allow local state to be manipulated in local variables, but require that functions can only access their own locals and global constants (or just constants defined in an outer scope)?
Good question. I think the answer is that mutable locals are of limited practical value but mutable heap-allocated data structures (primarily arrays) are enormously valuable and form the backbone of many important collections including efficient stacks, queues, sets and dictionaries. So restricting mutation to locals only would not give an otherwise purely functional language any of the important benefits of mutation.
On a related note, communicating sequential processes exchanging purely functional data structures offer many of the benefits of both worlds because the sequential processes can use mutation internally, e.g. mutable message queues are ~10x faster than any purely functional queues. For example, this is idiomatic in F# where the code in a MailboxProcessor uses mutable data structures but the messages communicated between them are immutable.
Sorting is a good case study in this context. Sedgewick's quicksort in C is short and simple and hundreds of times faster than the fastest purely functional sort in any language. The reason is that quicksort mutates the array in-place. Mutable locals would not help. Same story for most graph algorithms.
The short answer is: there are systems to allow what you want. For example, you can do it using the ST monad in Haskell (as referenced in the comments).
The ST monad approach is from Haskell's Control.Monad.ST. Code written in the ST monad can use references (STRef) where convenient. The nice part is that you can even use the results of the ST monad in pure code, as it is essentially self-contained (this is basically what you were wanting in the question).
The proof of this self-contained property is done through the type-system. The ST monad carries a state-thread parameter, usually denoted with a type-variable s. When you have such a computation you'll have monadic result, with a type like:
foo :: ST s Int
To actually turn this into a pure result, you have to use
runST :: (forall s . ST s a) -> a
You can read this type like: give me a computation where the s type parameter doesn't matter, and I can give you back the result of the computation, without the ST baggage. This basically keeps the mutable ST variables from escaping, as they would carry the s with them, which would be caught by the type system.
This can be used to good effect on pure structures that are implemented with underlying mutable structures (like the vector package). One can cast off the immutability for a limited time to do something that mutates the underlying array in place. For example, one could combine the immutable Vector with an impure algorithms package to keep the most of the performance characteristics of the in place sorting algorithms and still get purity.
In this case it would look something like:
pureSort :: Ord a => Vector a -> Vector a
pureSort vector = runST $ do
mutableVector <- thaw vector
sort mutableVector
freeze mutableVector
The thaw and freeze functions are linear-time copying, but this won't disrupt the overall O(n lg n) running time. You can even use unsafeFreeze to avoid another linear traversal, as the mutable vector isn't used again.
"Pure functional programming languages do not allow mutable data" ... actually it does, you just simply have to recognize where it lies hidden and see it for what it is.
Mutability is where two things have the same name and mutually exclusive times of existence so that they may be treated as "the same thing at different times". But as every Zen philosopher knows, there is no such thing as "same thing at different times". Everything ceases to exist in an instant and is inherited by its successor in possibly changed form, in a (possibly) uncountably-infinite succession of instants.
In the lambda calculus, mutability thus takes the form illustrated by the following example: (λx (λx f(x)) (x+1)) (x+1), which may also be rendered as "let x = x + 1 in let x = x + 1 in f(x)" or just "x = x + 1, x = x + 1, f(x)" in a more C-like notation.
In other words, "name clash" of the "lambda calculus" is actually "update" of imperative programming, in disguise. They are one and the same - in the eyes of the Zen (who is always right).
So, let's refer to each instant and state of the variable as the Zen Scope of an object. One ordinary scope with a mutable object equals many Zen Scopes with constant, unmutable objects that either get initialized if they are the first, or inherit from their predecessor if they are not.
When people say "mutability" they're misidentifying and confusing the issue. Mutability (as we've just seen here) is a complete red herring. What they actually mean (even unbeknonwst to themselves) is infinite mutability; i.e. the kind which occurs in cyclic control flow structures. In other words, what they're actually referring to - as being specifically "imperative" and not "functional" - is not mutability at all, but cyclic control flow structures along with the infinite nesting of Zen Scopes that this entails.
The key feature that lies absent in the lambda calculus is, thus, seen not as something that may be remedied by the inclusion of an overwrought and overthought "solution" like monads (though that doesn't exclude the possibility of it getting the job done) but as infinitary terms.
A control flow structure is the wrapping of an unwrapped (possibility infinite) decision tree structure. Branches may re-converge. In the corresponding unwrapped structure, they appear as replicated, but separate, branches or subtrees. Goto's are direct links to subtrees. A goto or branch that back-branches to an earlier part of a control flow structure (the very genesis of the "cycling" of a cyclic control flow structure) is a link to an identically-shaped copy of the entire structure being linked to. Corresponding to each structure is its Universally Unrolled decision tree.
More precisely, we may think of a control-flow structure as a statement that precedes an actual expression that conditions the value of that expression. The archetypical case in point is Landin's original case, itself (in his 1960's paper, where he tried to lambda-ize imperative languages): let x = 1 in f(x). The "x = 1" part is the statement, the "f(x)" is the value being conditioned by the statement. In C-like form, we could write this as x = 1, f(x).
More generally, corresponding to each statement S and expression Q is an expression S[Q] which represents the result Q after S is applied. Thus, (x = 1)[f(x)] is just λx f(x) (x + 1). The S wraps around the Q. If S contains cyclic control flow structures, the wrapping will be infinitary.
When Landin tried to work out this strategy, he hit a hard wall when he got to the while loop and went "Oops. Never mind." and fell back into what become an overwrought and overthought solution, while this simple (and in retrospect, obvious) answer eluded his notice.
A while loop "while (x < n) x = x + 1;" - which has the "infinite mutability" mentioned above, may itself be treated as an infinitary wrapper, "if (x < n) { x = x + 1; if (x < 1) { x = x + 1; if (x < 1) { x = x + 1; ... } } }". So, when it wraps around an expression Q, the result is (in C-like notation) "x < n? (x = x + 1, x < n? (x = x + 1, x < n? (x = x + 1, ...): Q): Q): Q", which may be directly rendered in lambda form as "x < n? (λx x < n (λx x < n? (λx·...) (x + 1): Q) (x + 1): Q) (x + 1): Q". This shows directly the connection between cyclicity and infinitariness.
This is an infinitary expression that, despite being infinite, has only a finite number of distinct subexpressions. Just as we can think of there being a Universally Unrolled form to this expression - which is similar to what's shown above (an infinite decision tree) - we can also think of there being a Maximally Rolled form, which could be obtained by labelling each of the distinct subexpressions and referring to the labels, instead. The key subexpressions would then be:
A: x < n? goto B: Q
B: x = x + 1, goto A
The subexpression labels, here, are "A:" and "B:", while the references to the subexpressions so labelled as "goto A" and "goto B", respectively. So, by magic, the very essence of Imperativitity emerges directly out of the infinitary lambda calculus, without any need to posit it separately or anew.
This way of viewing things applies even down to the level of binary files. Every interpretation of every byte (whether it be a part of an opcode of an instruction that starts 0, 1, 2 or more bytes back, or as part of a data structure) can be treated as being there in tandem, so that the binary file is a rolling up of a much larger universally unrolled structure whose physical byte code representation overlaps extensively with itself.
Thus, emerges the imperative programming language paradigm automatically out of the pure lambda calculus, itself, when the calculus is extended to include infinitary terms. The control flow structure is directly embodied in the very structure of the infinitary expression, itself; and thus requires no additional hacks (like Landin's or later descendants, like monads) - as it's already there.
This synthesis of the imperative and functional paradigms arose in the late 1980's via the USENET, but has not (yet) been published. Part of it was already implicit in the treatment (dating from around the same time) given to languages, like Prolog-II, and the much earlier treatment of cyclic recursive structures by infinitary expressions by Irene Guessarian LNCS 99 "Algebraic Semantics".
Now, earlier I said that the magma-based formulation might get you to the same place, or to an approximation thereof. I believe there is a kind of universal representation theorem of some sort, which asserts that the infinitary based formulation provides a purely syntactic representation, and that the semantics that arise from the monad-based representation factors through this as "monad-based semantics" = "infinitary lambda calculus" + "semantics of infinitary languages".
Likewise, we may think of the "Q" expressions above as being continuations; so there may also be a universal representation theorem for continuation semantics, which similarly rolls this formulation back into the infinitary lambda calculus.
At this point, I've said nothing about non-rational infinitary terms (i.e. infinitary terms which possess an infinite number of distinct subterms and no finite Minimal Rolling) - particularly in relation to interprocedural control flow semantics. Rational terms suffice to account for loops and branches, and so provide a platform for intraprocedural control flow semantics; but not as much so for the call-return semantics that are the essential core element of interprocedural control flow semantics, if you consider subprograms to be directly represented as embellished, glorified macros.
There may be something similar to the Chomsky hierarchy for infinitary term languages; so that type 3 corresponds to rational terms, type 2 to "algebraic terms" (those that can be rolled up into a finite set of "goto" references and "macro" definitions), and type 0 for "transcendental terms". That is, for me, an unresolved loose end, as well.
Is it impossible to know if two functions are equivalent? For example, a compiler writer wants to determine if two functions that the developer has written perform the same operation, what methods can he use to figure that one out? Or can what can we do to find out that two TMs are identical? Is there a way to normalize the machines?
Edit: If the general case is undecidable, how much information do you need to have before you can correctly say that two functions are equivalent?
Given an arbitrary function, f, we define a function f' which returns 1 on input n if f halts on input n. Now, for some number x we define a function g which, on input n, returns 1 if n = x, and otherwise calls f'(n).
If functional equivalence were decidable, then deciding whether g is identical to f' decides whether f halts on input x. That would solve the Halting problem. Related to this discussion is Rice's theorem.
Conclusion: functional equivalence is undecidable.
There is some discussion going on below about the validity of this proof. So let me elaborate on what the proof does, and give some example code in Python.
The proof creates a function f' which on input n starts to compute f(n). When this computation finishes, f' returns 1. Thus, f'(n) = 1 iff f halts on input n, and f' doesn't halt on n iff f doesn't. Python:
def create_f_prime(f):
def f_prime(n):
f(n)
return 1
return f_prime
Then we create a function g which takes n as input, and compares it to some value x. If n = x, then g(n) = g(x) = 1, else g(n) = f'(n). Python:
def create_g(f_prime, x):
def g(n):
return 1 if n == x else f_prime(n)
return g
Now the trick is, that for all n != x we have that g(n) = f'(n). Furthermore, we know that g(x) = 1. So, if g = f', then f'(x) = 1 and hence f(x) halts. Likewise, if g != f' then necessarily f'(x) != 1, which means that f(x) does not halt. So, deciding whether g = f' is equivalent to deciding whether f halts on input x. Using a slightly different notation for the above two functions, we can summarise all this as follows:
def halts(f, x):
def f_prime(n): f(n); return 1
def g(n): return 1 if n == x else f_prime(n)
return equiv(f_prime, g) # If only equiv would actually exist...
I'll also toss in an illustration of the proof in Haskell (GHC performs some loop detection, and I'm not really sure whether the use of seq is fool proof in this case, but anyway):
-- Tells whether two functions f and g are equivalent.
equiv :: (Integer -> Integer) -> (Integer -> Integer) -> Bool
equiv f g = undefined -- If only this could be implemented :)
-- Tells whether f halts on input x
halts :: (Integer -> Integer) -> Integer -> Bool
halts f x = equiv f' g
where
f' n = f n `seq` 1
g n = if n == x then 1 else f' n
Yes, it is undecidable. This is a form of the halting problem.
Note that I mean that it's undecidable for the general case. Just as you can determine halting for sufficiently simple programs, you can determine equivalency for sufficiently simple functions, and it's not inconceivable that this could be of some use for an application. But you cannot make a general method for determining equivalency of any two possible functions.
The general case is undecidable by Rice's Theorem, as others have already said (Rice's Theorem essentially says that any nontrivial property of a Turing-complete formalism is undecidable).
There are special cases where equivalence is decidable, the best-known example is probably equivalence of finite state automata. If I remember correctly equivalence of pushdown automata is already undecidable by reduction to Post's Correspondence Problem.
To prove that two given functions are equivalent you would require as input a proof of the equivalence in some formalism, which you can then check for correctness. The essential parts of this proof are the loop invariants, as these cannot be derived automatically.
In the general case it's undecidable whether two turing machines have always the same output for the identical input. Since you can't even decide whether a tm will halt on the input, I don't see how it should be possible to decide whether both halt AND output the same result...
It depends on what you mean by "function."
If the functions you are talking about are guaranteed to terminate -- for example, because they are written in a language in which all functions terminate -- and operate over finite domains, it's "easy" (although it might still take a very, very long time): two functions are equivalent if and only if they have the same value at every point in their shared domain.
This is called "extensional" equivalence to distinguish it from syntactic or "intensional" equivalence. Two functions are extensionally equivalent if they are intensionally equivalent, but the converse does not hold.
(All the other people above noting that it is undecidable in the general case are quite correct, of course, this is a fairly uncommon -- and usually uninteresting in practice -- special case.)
Note that the halting problem is decidable for linear bounded automata. Real computers are always bounded, and programs for them will always loop back to a previous configuration after sufficiently many steps. If you are using an unbounded (imaginary) computer to keep track of the configurations, you can detect that looping and take it into account.
You could check in your compiler to see if they are "exactly" identical, sure, but determining if they return identical values would be difficult and time consuming. You would have to basically call that method and perform its routine over an infinite number of possible calls and compare the value with that from the other routine.
Even if you could do the above, you would have to account for what global values change within the function, what objects are destroyed / changed in the function that do not affect the outcome.
You can really only compare the compiled code. So compile the compiled code to refactor?
Imagine the run time on trying to compile the code with "that" compiler. You could spend a LOT of time on here answering questions saying: "busy compiling..." :)
I think if you allow side effects, you can show that the problem can be morphed into the Post correspondence problem so you can't, in general, show if two functions are even capable of having the same side effects.
Is it impossible to know if two functions are equivalent?
No. It is possible to know that two functions are equivalent. If you have f(x), you know f(x) is equivalent to f(x).
If the question is "it is possible to determine if f(x) and g(x) are equivalent with f and g being any function and for all functions g and f", then the answer is no.
However, if the question is "can a compiler determine that if f(x) and g(x) are equivalent that they are equivalent?", then the answer is yes if they are equivalent in both output and side effects and order of side effects. In other words, if one is a transformation of the other that preserves behavior, then a compiler of sufficient complexity should be able to detect it. It also means that the compiler can transform a function f into a more optimal and equivalent function g given a particular definition of equivalent. It gets even more fun if f includes undefined behavior, because then g can also include undefined (but different) behavior!