I'm trying to wrap my head around lambda calculus, and how it relates to language, compiler and binary code. What it actually means that lambda calculus is equivalent to turing machine, and where it actually manifest itself?
I don't understand how lambda calculus could supersede turing machine as a theoretical model of computation. Turing machine is about sequential instructions to mutate the state, lambda calculus is about expressions evaluating to something. It is more abstract, like a programming language of it's own, not the model of how to practically compute something, make things happen. Or let's put it this way: lambda calculus is like the road map, and turing machine like a model of car. How are these two considered equivalent? Would it be possible to run software on hardware without implementing turing machine?
How does for example, a lisp compiler and language relates to lambda calculus? In which layer is the lambda calculus implemented? Is the implementation pure in the terms of definition of lambda calculus? Where and how the theory behind lambda calculus transforms syntax into a running binary? For example, in lambda calculus numbers are encoded as special functions applied to some other function n times. Yet in syntax we use number literals. Where all these axioms are used?
All these foundational languages have been introduced in a era preceding the advent of computers. The whole point of the research consisted in charcterizing a class of (numerical) functions that looked "intuitively" computable, in algorithmic sense, and not necessarily by an automatic device. Now, it turns out that lambda calculus and Turing machines, as well as many others computational models like combinatory logic, Post systems, generalized recursive functions, and so on, precisely express the same class of computable functions. This motivated Church's thesis.
I agree with you that Turing machines (like random access machines) have a more
architectural flavour with respect to other models.
In fact, this is what convinced Goedel, who was at first a bit
skeptical, of the validity of Church's thesis.
I also agree with you that lambda calculus cannot supersede Turing machines as a theoretical model of computation: there is nothing evident to gain in such an operation.
At the same time, lambda calculus is fun, while turing machines are deadly
boring. It is fun, precisely because it is at the extreme opposite of Turing
Machines. I think one could reasonably argue that it is the highest level
model of computation that has been ever conceived (and probably that will ever
be). This is why it is a challenging and instructive language for every
programmer.
In computability theory, equivalence between two theoretical models of computation means that they can solve the same set of problems. Anything you can compute in a Turing Machine, you can compute using Lambda Calculus, and vice versa.
How do we prove this? It is sufficient to say that if we can model a Turing Machine in lambda calculus, than clearly lambda calculus can compute everything a Turing Machine can. And if we can solve lambda calculus on a Turing Machine, then the reverse holds true as well.
Both of these are possible, and so the models are said to be equivalent.
In practice of course, one model may easily be more practical for certain use cases. Computers today are based on the RAM state model, which in turn lends the basis of it's theory from the idea of a Turing Machine. Lambda calculus is indeed quite abstract, and it doesn't lend it's self as easily to implementation in physical hardware. The two models however both exist in the same computation class, and they can solve the same problems, and are therefor referred to as equivalent.
Related
I recently started learning about functional programming and getting hands on with Haskell. With the fundamental difference between functional paradigm and others is, we don't maintain states and there are no computations as in imperative paradigm. The basic theory behind imperative programming is Automata theory and automata theory is the basis for computers.
So having said that, I cannot see any relation between automata theory and functional programming. So has functional programming changed the fundamentals of programming inside-out?
About the same time as Turing Machines where developed (I'm just guessing that you are referring to TMs, when you say Automata theory is the basis for computers) also the lambda-calculus was suggested as a model of computation (okay not quite computers, but close). Both notions turned out to be equivalent in the sense that whatever can be "implemented" on a TM can also be "implemented" by a lambda term and vice versa. Moreover functional programming is more or less an implementation of the (typed) lambda calculus. So I guess it is just two different ways of looking at the same thing.
In object-oriented programming, we might say the core concepts are:
encapsulation
inheritance,
polymorphism
What would that be in functional programming?
There's no community consensus on what are the essential concepts in functional programming. In
Why Functional Programming Matters (PDF), John Hughes argues that they are higher-order functions and lazy evaluation. In Wearing the Hair Shirt: A Retrospective on Haskell, Simon Peyton Jones says the real essential is not laziness but purity. Richard Bird would agree. But there's a whole crowd of Scheme and ML programmers who are perfectly happy to write programs with side effects.
As someone who has practiced and taught functional programming for twenty years, I can give you a few ideas that are widely believed to be at the core of functional programming:
Nested, first-class functions with proper lexical scoping are at the core. This means you can create an anonymous function at run time, whose free variables may be parameters or local variables of an enclosing function, and you get a value you can return, put into data structures, and so on. (This is the most important form of higher-order functions, but some higher-order functions (like qsort!) can be written in C, which is not a functional language.)
Means of composing functions with other functions to solve problems. Nobody does this better than John Hughes.
Many functional programmers believe purity (freedom from effects, including mutation, I/O, and exceptions) is at the core of functional programming. Many functional programmers do not.
Polymorphism, whether it is enforced by the compiler or not, is a core value of functional programmers. Confusingly, C++ programmers call this concept "generic programming." When polymorphism is enforced by the compiler it is generally a variant of Hindley-Milner, but the more powerful System F is also a powerful basis for functional languages. And with languages like Scheme, Erlang, and Lua, you can do functional programming without a static type system.
Finally, a large majority of functional programmers believe in the value of inductively defined data types, sometimes called "recursive types". In languages with static type systems these are generally known as "algebraic data types", but you will find inductively defined data types even in material written for beginning Scheme programmers. Inductively defined types usually ship with a language feature called pattern matching, which supports a very general form of case analysis. Often the compiler can tell you if you have forgotten a case. I wouldn't want to program without this language feature (a luxury once sampled becomes a necessity).
In computer science, functional programming is a programming paradigm that treats computation as the evaluation of mathematical functions and avoids state and mutable data. It emphasizes the application of functions, in contrast to the imperative programming style, which emphasizes changes in state. Functional programming has its roots in the lambda calculus, a formal system developed in the 1930s to investigate function definition, function application, and recursion. Many functional programming languages can be viewed as embellishments to the lambda calculus. - Wikipedia
In a nutshell,
Lambda Calculus
Higher Order Functions
Immutability
No side-effects
Not directly an answer to your question, but I'd like to point out that "object-oriented" and functional programming aren't necessarily at odds. The "core concepts" you cite have more general counterparts which apply just as well to functional programming.
Encapsulation, more generally, is modularisation. All purely functional languages that I know of support modular programming. You might say that those languages implement encapsulation better than the typical "OO" variety, since side-effects break encapsulation, and pure functions have no side-effects.
Inheritance, more generally, is logical implication, which is what a function represents. The canonical subclass -> superclass relation is a kind of implicit function. In functional languages, this is expressed with type classes or implicits (I consider implicits to be the more general of these two).
Polymorphism in the "OO" school is achieved by means of subtyping (inheritance). There is a more general kind of polymorphism known as parametric polymorphism (a.k.a. generics), which you will find to be supported by pure-functional programming languages. Additionally, some support "higher kinds", or higher-order generics (a.k.a. type constructor polymorphism).
What I'm trying to say is that your "core concepts of OO" aren't specific to OO in any way. I, for one, would argue that there aren't any core concepts of OO, in fact.
Let me repeat the answer I gave at one discussion in the Bangalore Functional Programming group:
A functional program consists only of functions. Functions compute
values from their inputs. We can contrast this with imperative
programming, where as the program executes, values of mutable
locations change. In other words, in C or Java, a variable called X
refers to a location whose value change. But in functional
programming X is the name of a value (not a location). Any where that
X is in scope, it has the same value (i.e, it is referentially
transparent). In FP, functions are also values. They can be passed as
arguments to other functions. This is known as higher-order functional
programming. Higher-order functions let us model an amazing variety of
patterns. For instance, look at the map function in Lisp. It
represents a pattern where the programmer needs to do 'something' to
every element of a list. That 'something' is encoded as a function and
passed as an argument to map.
As we saw, the most notable feature of FP is it's side-effect
freeness. If a function does something more than computing a value
from it's input, then it is causing a side-effect. Such functions are
not allowed in pure FP. It is easy to test side-effect free functions.
There is no global state to set-up before running the test and there
is no global state to check after running the test. Each function can
be tested independently just by providing it's input and examining the
return value. This makes it easy to write automated tests. Another
advantage of side-effect freeness is that it gives you better control
on parallelism.
Many FP languages treat recursion and iteration correctly. They does this by
supporting something called tail-recursion. What tail-recursion is -
if a function calls itself, and it is the last thing it does, it
removes the current stack frame right away. In other words, if a
function calls itself tail-recursively a 1000 times, it does not grow
the stack a 1000 deep. This makes special looping constructs
unnecessary in these languages.
Lambda Calculus is the most boiled down version of an FP language.
Higher level FP languages like Haskell get compiled to Lambda
Calculus. It has only three syntactic constructs but still it is
expressive enough to represent any abstraction or algorithm.
My opinion is that FP should be viewed as a meta-paradigm. We can
write programs in any style, including OOP, using the simple
functional abstractions provided by the Lambda Calculus.
Thanks,
-- Vijay
Original discussion link: http://groups.google.co.in/group/bangalore-fp/browse_thread/thread/4c2cfa7985d7eab3
Abstraction, the process of making a function by parameterizing over some part of an expression.
Application, the process of evaluating a function by replacing its parameters with specific values.
At some level, that's all there is to it.
Though the question is older, thought of sharing my view as reference.
Core Concept in FP is "FUNCTION"
FP gives KISS(Keep It Simple Sxxxxx) programming paradigm (once you get the FP ideas, you will literally start hating the OO paradigm)
Here is my simple FP comparison with OO Design Patterns. Its my perspective of seeing FP and pls correct me if there is any discrepancy from actual.
There are many methods for representing structure of a program (like UML class diagrams etc.). I am interested if there is a convention which describes programs in a strict, mathematical way. I am especially interested in the use of mathematical notation for this purpose.
An example: Classes are represented as sets (fields, properties) and functions (operating on the elements of sets). A parent class' fields are a subset of child class'. Functions are described in pseudocode which has to look like this and that...
I know that Z Notation has been used to some extent in the formal verification of software, such as the Tokeneer project.
Z Notation
Z Reference Manual
http://www.amazon.com/Concrete-Mathematics-Foundation-Computer-Science/dp/0201558025
Yes, there is, Floyd-Hoare Logic.
There are a lot of way, but i think most of them are inconvenient for expressing the structure since the structure is often not expressable in default mathematical concepts. The main exception is of course functional programing languages. Think about folds (catamorphisme), groups, algebra's etc.
For imperative programming I know of the existence of Z, which uses (pure and extended) lambda calculus set theory and (first order) predicate logic. However, i dont think it's very convenient. The only upside of using mathematics to express structure is the fact that you can prove stuff about it. But if you want to do that, take a look at JML, Spec# or Eiffel.
Depends on what you're trying to accomplish, but going down this road with specific languages can get you into trouble.
For example, see the circle-ellipse discussion on C++ FAQ Lite.
This book applies the deductive method
to programming by affiliating programs
with the abstract mathematical
theories that enable them work. [...]
I believe that Elements of Programming by Alexander Stepanov and Paul McJones, is pretty close to what you are looking for.
Concepts
A concept is a description of
requirements on one or more types
stated in terms of the existence and
properties of procedures, type
attributes, and type functions defined
on the types.
Z, which has already been mentioned, is pretty much what you describe. There are some variants of it for object-oriented modelling, but I think you can get quite far with "standard Z's" schemas if you wish to model classes.
There's also Alloy, which is newer and inspired by Z. Its notation is perhaps a bit closer to object-orientation. It is also analysable, i.e. you can check the models you create whether they fulfill certain conditions, but it cannot prove that properties hold, just attempt to refute within a finite scope.
The article Dependable Software by Design is a nice introduction to Alloy and its ilk, along with a table of available similar tools.
You are looking for functional programming. There are several functional programming languages, and they are all based on a fundamental mathematical theory called the Lambda calculus. Programs written in a functional programming language such as LISP are a mathematical representation of themselves. ;-)
There is a mathematical language which actually describes a program or rather it's operations. You take the initial state and then transform this state until you reach the desired target state. The transformations yield the program code which must be executed.
See the Wikipedia article about Hoare logic.
The basic idea is that for every function (no matter if you put that into a class or into an old style function), you have a pre- and a post-condition. For example, the precondition can be that you have an array which has >= 0 elements. the post-condition is that every element[i] must by <= element[j] for every i <= j.
The usual description would be "the function sorts the array". But the mathematical terms allow you to transform the input (which must match the precondition) into the output (which must match the postcondition).
It's a bit unwieldy to use, especially for more complex programs but some of the examples are pretty impressive. Often, you get really compact code as the result which looks quite complex but works at first try.
I'd like to suggest Algebra of Programming. It's a calculational approach to programs, using Relational Algebra, and Galois Connections.
If you have further interest on this topic, you can find an amazing paper, here, by Shin-Cheng Mu, and José Nuno Oliveira (slides).
Using Relational Algebra and First-Order Logic, also has a nice synergy with Alloy, Functional Programming, and Design by Contract (easily applied to Object-Oriented Programming).
By Logic Programming I mean the a sub-paradigm of declarative programming languages. Don't confuse this question with "What problems can you solve with if-then-else?"
A language like Prolog is very fascinating, and it's worth learning for the sake of learning, but I have to wonder what class of real-world problems is best expressed and solved by such a language. Are there better languages? Does logic programming exist by another name in more trendy programming languages? Is the cynical version of the answer a variant of the Python Paradox?
Prototyping.
Prolog is dynamic and has been for 50 years. The compiler is liberal, the syntax minimalist, and "doing stuff" is easy, fun and efficient. SWI-Prolog has a built-in tracer (debugger!), and even a graphical tracer. You can change the code on the fly, using make/0, you can dynamically load modules, add a few lines of code without leaving the interpreter, or edit the file you're currently running on the fly with edit(1). Do you think you've found a problem with the foobar/2 predicate?
?- edit(foobar).
And as soon as you leave the editor, that thing is going to be re-compiled. Sure, Eclipse does the same thing for Java, but Java isn't exactly a prototyping language.
Apart from the pure prototyping stuff, Prolog is incredibly well suited for translating a piece of logic into code. So, automatic provers and that type of stuff can easily be written in Prolog.
The first Erlang interpreter was written in Prolog - and for a reason, since Prolog is very well suited for parsing, and encoding the logic you find in parse trees. In fact, Prolog comes with a built-in parser! No, not a library, it's in the syntax, namely DCGs.
Prolog is used a lot in NLP, particularly in syntax and computational semantics.
But, Prolog is underused and underappreciated. Unfortunately, it seems to bear an academic or "unusable for any real purpose" stigma. But it can be put to very good use in many real-world applications involving facts and the computation of relations between facts. It is not very well suited for number crunching, but CS is not only about number crunching.
Since Prolog = Syntactic Unification + Backward chaining + REPL,
most places where syntactic unification is used is also a good use for Prolog.
Syntactic unification uses
AST transformations
Type Inference
Term rewriting
Theorem proving
Natural language processing
Pattern matching
Combinatorial test case generation
Extract sub structures from structured data such as an XML document
Symbolic computation i.e. calculus
Deductive databases
Expert systems
Artificial Intelligence
Parsing
Query languages
Constraint Logic Programming (CLP)
Many very good and well-suited use cases of logic programming have already been mentioned. I would like to complement the existing list with several tasks from an extremely important application area of logic programming:
Logic programming blends seamlessly, more seamlessly than other paradigms, with constraints, resulting in a framework called Constraint Logic Programming.
This leads to dedicated constraint solvers for different domains, such as:
CLP(FD) for integers
CLP(B) for Booleans
CLP(Q) for rational numbers
CLP(R) for floating point numbers.
These dedicated constraint solvers lead to several important use cases of logic programming that have not yeen been mentioned, some of which I show below.
When choosing a Prolog system, the power and performance of its constraint solvers are often among the deciding factors, especially for commercial users.
CLP(FD) — Reasoning over integers
In practice, CLP(FD) is one of the most imporant applications of logic programming, and is used to solve tasks from the following areas, among others:
scheduling
resource allocation
planning
combinatorial optimization
See clpfd for more information and several examples.
CLP(B) — Boolean constraints
CLP(B) is often used in connection with:
SAT solving
circuit verification
combinatorial counting
See clpb.
CLP(Q) — Rational numbers
CLP(Q) is used to solve important classes of problems arising in Operations Research:
linear programming
integer linear programming
mixed integer linear programming
See clpq.
One of the things Prolog gives you for free is a backtracking search algorithm -- you could implement it yourself, but if your problem is best solved by having that algorithm available, then it's nice to use it.
The two things I've seen it be good at is mathematical proofs and natural language understanding.
Prolog is ideal for non-numeric problems. This article gives a few examples of some applications of Prolog and it might help you understand the type of problems that it might solve.
Prolog is great at solving puzzles and the like. That said, in the domain of puzzle-solving it makes easy/medium puzzle-solving easier and complicated puzzle solving harder. Still, writing solvers for grid puzzles and the like such as Hexiom, Sudoku, or Nurikabe is not especially tough.
One simple answer is "build systems". The language used to build Makefiles (at least, the part to describe dependencies) is essentially a logic programming language, although not really a "pure" logic programming language.
Yes, Prolog has been around since 1972. It was invented by Alain Colmerauer with Philippe Roussel, based on Robert Kowalski's procedural interpretation of Horn clauses. Alain was a French computer scientist and professor at Aix-Marseille University from 1970 to 1995.
And Alain invented it to analyse Natural Language. Several successful prototypes were created by him and his "followers".
His own system Orbis to understand questions in English and French about the solar system. See his personal site.
Warren and Pereira's system Chat80 QA on world geography.
Today, IBM Watson is a contempory QA based on logic with a huge dose of statistics about real world phrases.
So you can imagine that's where it's strength is.
Retired in 2006, he remained active until he died in 2017. He was named Chevalier de la Legion d’Honneur by the French government in 1986.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
To all the people who know lambda calculus: What benefit has it bought you, regarding programming? Would you recommend that people learn it?
The benefit of lambda calculus is that it's an extremely simple model of computation that is equivalent to a Turing machine. But while a Turing machine is more like assembly language, lambda calculus is more a like a high-level language. And if you learn Church encodings that will help you learn the programming technique called continuation-passing style, which is quite useful for implementing backtracking search and other neat tricks.
The main use of lambda calculus in practice is that it is a great laboratory tool for studying new programming-language ideas. If you have an idea for a new language feature, you can add the new feature to the lambda calculus and you get something that is expressive enough to program while being simple enough to study very thoroughly. This use is really more for language designers and theorists than for programmers.
Lambda calculus is also just very cool in its own right: just like knowing assembly language, it will deepen your understanding of computation. It's especially fun to program a universal turing machine in the lambda calculus. But this is foundational mathematics, not practical programming.
If you want to program in any functional programming language, it's essential. I mean, how useful is it to know about Turing machines? Well, if you write C, the language paradigm is quite close to Turing machines -- you have an instruction pointer and a current instruction, and the machine takes some action in the current state, and then ambles along to the next instruction.
In a functional language, you simply can't think like that -- that's not the language paradigm. You have to think back to lambda calculus, and how terms are evaluated there. It will be much harder for you to be effective in a functional language if you don't know lambda calculus.
To be honest, learning lambda calculus before functional programming has made me realize that the two are as unrelated as C is to any imperative programming.
Lambda calculus is a functional programming language, an esoteric one, a Turing tarpit if you like; accidentally it's also the first.
The majority of functional programming languages at all do not require you to 'learn' lambda calculus, whatever that would mean, lambda calculus is insanely minimal, you can 'learn' its axioms in an under an hour. To know the results from it, like the fixedpoint theorem, the Church-Rosser Theorem et cetera is just irrelevant to functional programming.
Also, lambda-abstractions are often held to be 'functions', I disagree with that, they are algorithms, not functions, a minor difference, most 'functional languages' treat their functions more in the way classical mathematics does.
However, to for instance effectively use Haskell you do need to understand certain type systems, that's irrespective of lambda calculus, the System F type system can be applied to all 'functions' and requires no lambda abstractions at all. Commonly in maths we say f : R^2 -> R : f (x) = x^2. We could've said: f (x) = x^2 :: R -> R -> R. In fact, Haskell comes pretty close to this notation.
Lambda calculus is a theoretical formalism, Haskell's functions are really no more 'lambda abstractions' than f : f(x) = x^2 really, what makes lambda abstractions interesting is that it enables us to define what are normally seen as 'constants' as 'functions', no functional language does that because of the huge computational overhead. Haskell and alike is just a restricted form of System F's type system applied to functions as used in everyday classical maths. Functions in Haskell are certainly not the anonymous formally symbolic reduction-applicants as they are in lambda-calculus. Most functional programming languages are not symbolic reduction-based re-writing systems. Lisps are to some degree but that's a paradigm on its own and its 'lambda keyword' really doesn't satisfy calling it lambda calculus.
I think the use of lambda calculus with respect to programming in practice is that it is a quite minimal system that captures the essence of abstraction (or "anonymous functions" or closures, if you will). Other than that I don't think it is generally essential except when you need to implement abstraction yourself (as Tetha (114646) mentioned).
I also completely disagree with Denis Bueno (114701) who says that it is essential for functional programming. It is perfectly well possible to define, use or understand a functional language without any lambda calculus at all. In order to understand the evaluation of terms in functional languages (which, in my opinion, somewhat contradicts the use of a functional language) you will most likely be better of learning about term rewrite systems.
I agree with those that say it is theoretically possible to learn functional programming without learning the lambda calculus—but what's the advantage of not learning the lambda calculus? It's not as if it takes a big investment of time.
Most likely, it will help you understand functional programming better. But even if it doesn't, it's still a cool thing worth learning. The Y-combinator is a thing of beauty.
If you only want to be a technician and write programs to do things, then you don't really need to know lambda-calculus, finite-state machines, pushdown automata, regular expressions, context-free grammar, discrete mathematics, etc.
But if you have curiosity about the deeper mysteries underlying this stuff, you can start to wonder how these questions might be answered. The concepts are beautiful and will expand your imagination. I also think they, incidentally, make one a better practicioner.
What got me hooked was Minsky's book Computation: Finite and Infinite Machines.
The lambda calculus is a computational model, just like the turing machine. Thus, it is useful if you need to implement a certain evaluator for a language based on this model, however, in practice, you just need the basic idea (uh. place argument semantically correct in the body of a function?) and that's about it.
One posible way to learn lambda calculus is
http://en.wikipedia.org/wiki/Lambda_Calculus
Or, if you want more, here is my blog dedicated to lambda calculus and stuff like that
http://weblogs.manas.com.ar/lziliani/
As every abstraction of computations, with lambda calculus you can model stuff used in most programming languages, like subtyping. For more about this, one of the best books with practical uses of lambda calculus in this sense is
http://www.amazon.com/Types-Programming-Languages-Benjamin-Pierce/dp/0262162091/ref=sr_1_1?ie=UTF8&s=books&qid=1222088714&sr=8-1
I found that the Lambda calculus was useful for understand how functional programming worked on a deeper level. Especially how to implement functional languages.
It has made it easier for me to understand advanced concepts like type-systems and evaluations strategies (e.g. call by name versus call by value).
I don't think one needs to know anything about the Lambda calculus to use basic functional programming techniques. However understanding the lambda calculus makes it easier to learn advanced programming theory.
I'd also like to mention that if you're doing anything in the area of NLP, lambda calculus is at the foundation of a massive body of work in compositional semantics.
The benefits for me is a more compact synergistic programming. Stuff tends to flow horizontally more than vertically. Plus it is very useful for prototyping simple algorithms. Don't know if I am using it to its full potential but I find it very useful.