Related
As I got a lot of spare time to spend ATM I read a few threads/comments on code-comments and documentation here.
As most people here I too think that you should write your code so that it's easy to read and self-commenting as far as it's possible.
On the other hand I am a huge FP-fanboy - and yes if you write your code the right way it will be very readable in FP - or so I thought.
Problem is that tiny things make a awful lot of difference in FP-world. If your colleague doesn't fully understand FP he might be able to "read" the indentation of the code but won't be able to modify or fully understand it. That stands for languagues like Haskell, where a '.' or '$' makes a big difference and also for languages like F# or even C# of VB.NET with lots of LINQ statements.
At first glance the problem might be, that your peer just doesn't get the language and it's not the codes fault - on the other hand: who truly gets all of FP? Look at some papers concerning Haskell - the code is beautifully crafted and self-commenting but just as in math you may have to chew on a line for several minutes before you get it.
Of course in those papers there will be a text-block trying to clarify just after the code ....
So IMHO you have to comment your FP-code as long as you work in a shop where not every colleague has a PhD in CS ;)
What do you think?
PS: first post here - really looked for answers concerning this questions but didn't find any - please be gentle if I just didn't look hard enough :)
Functional languages greatly favor the development of self-documenting code, because you can freely rearrange the order of functions, and easily abstract out any given part of the code, assigning it an explanatory name.
Abstract, abstract, abstract, is the keyword to master code complexity, and that's where the functional style shines. But there will be always things that cannot be expressed within the code itself.
One clear example is code for algorithms. It is unlikely that one can easily understand a complex algorithm just by looking at the implementation. Yes, functional languages make understanding simpler, becasue many gory details (trivial example: memory management) do not have to be coded explicitly, thus exposing the underlying logic more clearly.
However this is no substitute for an explanation in natural language, which conveys in an intuitive way how it works (and sometimes a picture is worth a thousand words). This is becasue our brain needs to observe difficult concepts from different point of views in order to understand them fully.
What to comment also depends on your audience. Beginners, average programmers or wizards? There is no one-fits-all solution.
E.g. you should explain the meaning of a "." (function composition) in Haskell if you are writing tutorial code, but certainly that would be a redundant explanation for anyone who has gone past chapter one/two of any Haskell book.
On the other hand some specific algorithm, like say red-black trees, could be a given for some programmers, and something very mysterious for others. In the second case you should add many comments to the code, or point to a document with further explanations.
Finally, one should notice that there is no consensus even among the masters. E.g. Dennis Ritchie is famous for being extremely parsimonious with comments, instead Don Knuth is an advocate of "Literate programming", where comments are as important as code itself. A set of rules will never be a substitute for personal taste.
Most people say that even functional programming is less likely to land you a job, you can become a better imperative/OO programmer by learning it.
For me, it's mostly about writing "non member non friend" functions that have no side effects. But I couldn't come up with more examples where functional programming can be effectively applied in imperative languages, because working around languages' lack of features is often too cumbersome.
So what are some more (specific) examples/techniques that you actually applied in non-functional languages that were inspired by functional programming?
Another of my own experience
This one is quite abstract, but due to the lack of "objects" in most FP languages, the culture there tends favor rigorous data structure design. Usually, in OOP languages, because stuffing an extra variable in a class is too easy, things tend to go mess up rather quickly. Though the same could be done using OCaml's and Haskell's record syntax, that kind of approach somehow feels out of place in FP.
Data Transformation
In my experience thinking on how to solve a problem functionally makes you think more about what data gets transformed to what - and not what state needs to be changed in order to keep the damn thing running...
Thinking of problems as transformations makes them appear different all by itself - which leads to different and most likely more elegant solutions.
Update: In c++ there is the <functional> header, and std::transform in <algorithm>.
Most Ruby Enumerable methods are inspired by Higher Order Functions from Functional programming
The new-ish JavaScript array functions, filter, map, every, some, reduce, and reduceRight, are functional-inspired.
Functional Java was already mentioned in the comments, but there is also some functional-ish stuff in Apache Commons Collections. See the org.apache.commons.collections.functors package.
I'm looking for non-trivial resources on concepts of asychronous programming, preferably books but also substantial articles or papers. This is not about the simple examples like passing a callback to an event listener in GUI programming, or having producer-consumer decoupled over a queue, or writing an onload handler for your HTML (although all those are valid). It's about the kind of problems the lighttpd developers might be concerned with, or someone doing substantial business logic in JavaScript that runs in a browser or on node.js. It's about situations where you need to pass a callback to a callback to a callback ... about complex asynchronous control-flows, and staying sane at the same time. I'm looking for concepts that allow you to do this systematically, to reason about this kind of control-flows, to seriously manage a significant amount of logic distributed in deeply nested callbacks, with all its ensuing issues of timing, synchronization, binding of values, passing of contexts, etc.
I wouldn't shrink away from some abstract explorations like continuation-passing-style, linear logic or temporal reasoning. Posts like this seem to go into the right direction, but discuss specific issues rather than a complete theory (E.g. the post mentions the "reactor" pattern, which seems relevant, without describing it).
Thanks.
EDIT:
To give more details about the aspects I'm interested in. I'm interested in a disciplined approach to asynchronous programming, a theory if you will, maybe just a set of specific patterns that I can pass to fellow programmers and say "This is the way we do asynchronous programming" in non-trivial scenarios. I need a theory to disentangle layers of callbacks that randomly fail to work, or produce spurious results. I want an approach which allows me to say "If we do it this way, we can be sure that ...". - Does this make things clearer?
EDIT 2:
As feedback indicates a dependency on the programming language: This will be JavaScript, but maybe it's enough to assume a language that allows higher-order functions.
EDIT 3:
Changed the title to be more specific (although I think design patterns are only one way to look at it; but at least it gives a better direction).
When doing layered callbacks currying is a useful technique.
For more on this you can look at http://en.wikibooks.org/wiki/Haskell/Higher-order_functions_and_Currying and for javascript you can look at http://www.svendtofte.com/code/curried_javascript/.
Basically, if you have multiple layers of callbacks, rather than having one massive parameter list, you can build it up incrementally, so that when you are in a loop calling your function, the various callback functions have already been defined, and passed.
This isn't meant as a complete answer to the question, but I was asked to put this part into an answer, so I did.
After a quick search here is a blog where he shows using currying with callbacks:
http://bjouhier.wordpress.com/2011/04/04/currying-the-callback-or-the-essence-of-futures/
UPDATE:
After reading the edit to the original question, to see design patterns for asynchronous programming, this may be a good diagram:
http://www1.cse.wustl.edu/~schmidt/patterns-ace.html, but there is much more to good asynchronous design, as first-order functions will enable this to be simplified, but, if you are using the MPI library and Fortran then you will have different implementations.
How you approach the design is affected heavily by the language and the technologies involved, that any answer will fall short of being complete.
What should be the measures that should be used to identify that code is over abstracted and very hard to understand and what should be done to reduce over abstraction?
"Simplicity over complexity, complexity over complicatedness"
So - there's a benefit to abstract something only if You are "de-leveling" complicatedness to complexity. Reasons to do that can vary: better modularity, better encapsulation etc.
Identifying over abstraction is a chicken and egg problem. In order to reduce over abstraction You need to understand actual reason behind code lines. That includes understanding idea of particular abstraction itself (in contrast to calling it over abstracted cause of lack of understanding). And that's not enough - You need to know a better, simpler solution to prove that it's over abstracted.
If You are looking for tool that could do it in Your place - look no more, only mind can reliably judge that.
I will give an answer that will get a LOT of down votes!
If the code is written in an OO language .. it is necessarily heavily over-abstracted. The purer the language the worse the problem.
Abstraction should be used with great caution. If in doubt always use concrete data structures. (You can always abstract later, this is easier than de-abstraction :)
You must be very certain you have the right abstraction in your current context, and you must be very sure that concept will stand the test of change. Abstraction has a high price in performance of both the code and the coder.
Some weak tests for over-abstraction: if the data structure is a product type (struct in C) and the programmer has written get and set method for each field, they have utterly failed to provide any real abstraction, disabled operators like C increment, for no purpose, and simply not understood that the struct field names are already the abstract representation of a product. Duplicating and laming up the interface is not a good idea.
A good test for the product case is whether there exist any data invariants to maintain. For example a pair of integers representing a rational number is almost sufficient, there's little need for any abstraction because all pairs are valid except when the denominator is zero. However for performance reasons one may choose to maintain an invariant, typically the denominator is required to be greater than zero, and the numerator and denominator are relatively prime. To ensure the invariant, the product representation is encapsulated: the initial value protected by a constructor and methods constrained to maintain the invariant.
To fix code I recommend these steps:
Document the representation invariants the abstraction is maintaining
Remove the abstraction (methods) if you can't find strong invariants
Rewrite code using the method to access the data directly.
This procedure only works for low level abstraction, i.e. abstraction of small values by classes.
Over abstraction at a higher level is much harder to deal with. Ideally you'd refactor the code repeatedly, checking to see after each step it continues to work. However this will be hard, and sometimes a major rewrite is required, rather than a refinement. It's probably not worth it unless the abstraction is so far off base it is not tenable to continue to maintain it.
Download Magento and have a look at the code, read some documents on it and have a look at their ERD: http://www.magentocommerce.com/wiki/_media/doc/magento---sample_database_diagram.png?cache=cache
I'm not joking, this is over-abstraction.. trying to please everyone and cover every base is a terrible idea and makes life extremely difficult for everyone.
Personally I would say that "What is the ideal level of abstraction?" is a subjective question.
I don't like code that uses a new line for every atomic operation, but I also don't like 10 nested operations within one line.
I like the use of recursive functions, but I don't appreciate recursion for the sole sake of recursion.
I like generics, but I don't like (nested) generic functions that e.g. use different code for each specific type that's expected...
It is a matter of personal opinion as well as common sense. Does this answer your question?
I completely agree with what #ArnisLapsa wrote:
"Simplicity over complexity, complexity over complicatedness"
And that
an abstraction is used to "de-level" those, from complicated to complex
(and from complex to simpler)
Also, as stated by #MartinHemmings a good abstraction is quite subjective because we don't all think the same way. And actually our way of thinking change with time. So Something that someone find simple might looks complex to others, and even become simpler with more experiences. Eg. A monadic operation is something trivial for functional programmer, but can be seriously confusing for others. Similarly, a design with mutable object communicating with each other can be natural for some and feel un-trackable for others.
That being said, I would like to add a couple of indicators. Note that this applies to abstractions used in code-base, not "paradigm abstraction" such as everything-is-a-function, or everything-is-designed-as-objects. So:
To the people it concerns, the abstraction should be conceptually simpler than other alternatives, without looking at the implementation. If you find that thinking of all possible cases is simpler that reasoning using the abstraction, then this abstraction is not suitable (for you)
Its implementation should reason only about the abstraction, not the specific cases that it will be used for. As soon as the abstraction implementation has parts made for specific cases, it indicates an "unfit" abstraction. And increasing generalization to cope with each new case, is going the wrong way (and tends to fall to the next issue).
A very common indicator of over-abstraction I have found (and actually fell for) are abstractions that represent more than what is needed, now. As much as possible, they should allow to do exactly what is required, but nothing more. For example, say you're thinking of, or already have, a "2d point" abstraction for which you can define many operators you need. Then you have another need that could really be a "4d point" similar to the 2d. Don't start to use a "Ndimensionnal point" abstraction, especially thinking that you might later need it. Maybe you'll never have anything else than 2 and 4d (because it stays as "a good idea" in the backlog forever) but instead some requirements pops to convert 4d points into pairs of 2d points. That's going to be hard to generalize to n-dimensions. So, each abstraction can be checked to cover and only cover the actual needs. In my point example, the complexity "n-dimensional" is actually only used to cope with the 2 and 4d cases (and the 4d might not even be used that much).
Finally, in a more global point of view, a code-base that has many not related abstractions, is an indicator that the dev team tends to abstract every little issues. So probably many of them are or became over-abstracted.
My professor told us that we could choose a programming language for our next programming assignment. I've been meaning to try out a functional language, so I figured I'd try out clojure. The problem is that I understand the syntax and understand the basic concepts, but I'm having problems getting everything to "click" in my head. Does anyone have any advice? Or am I maybe picking the wrong language to start functional programming with?
It's a little like riding a bike, it just takes practice. Try solving some problems with it, maybe ProjectEuler and eventually it'll click.
Someone mentioned the book "The Little Schemer" and this is a pretty good read. Although it targets Scheme the actual problems will be worth working through.
Good luck!
Well, for me, I encountered the same problem as you do in the beginning when I started doing OCAML, but the trick is that you must start thinking about what you want from the code and not how to do it!!!
For example, to calculate the square of list's elements, forget about the length of the list and such tricks, just think mathematically like that:
if the list is empty -> I am done
if not, then the list must have a head and tail -> you calculate the square of the head, then ask your function to do the same with the tail.
Just think about the general case and the base one, and that you are emitting data and not modifying it (unless you want to modify it ;) ).
Good luck!
You could check out The Little Schemer.
How about this: http://www.defmacro.org/ramblings/lisp.html
It is a very simple, step-by-step introduction to thinking in lisp from the point of view of a regular imperative programmer (Java, C#, etc.).
For educational purposes I would recommend PLT Scheme. It is a portable and powerful environment with very good examples and an even better documentation. It will help you to discover the thoughts behind functional programming step by step and in a very clean way. Choosing a little application to implement will help you learning the new language.
http://www.plt-scheme.org/
Additionally "Structure and Interpretation of Computer Programs" of H. Abelssn, G. Sussman, and J. Sussman is a very good book regarding Scheme (and programming).
Regards
mue
Take a look at 99 Lispy problems
Some thoughts on Lisps, not specific to Clojure (I'm not a Lisp expert, so I hope they're mostly correct and useful):
Coding in AST
I know little about compiler or interpreter theory, but every time I code in Lisp, it amazes me that it feels like directly building an AST.
That's part of what "code = data" means, coding in Lisp is a lot like filling data structures (nested lists) with AST nodes. Amazing, and it's easy to read too (with the right text editor).
A Programmable Programming Language
So code chunks are just nested lists, and list operations are part of the language. So you can very easily write Lisp code that generates Lisp code (see Lisp macros). This makes Lisp a programmable (in itself!) programming language.
This makes building a DSL or an interpreter in Lisp is very easy (see also meta-circular evaluation).
Never reboot anything
And in most Lisp systems, code (including documentation) can be introspected and hot swapped at run time.
Advanced OOP
Then, most Lisp Systems have some sort of Object System derived from CLOS, which is an advanced (compared to many OOP implementations) and configurable Object System (see The Art of the Metaobject Protocol).
All these features where invented long ago, but I'm not sure they are available in many other programming languages (although most are catching up, e.g. with closures), so you have to "rediscover" and get used to these by practicing (see the books in other answers).
Just remember: it's all data!
Write some simple classic functions that Lisp is good at, like
reverse a list
tell if an atom is somewhere in an s-expression
write EQUAL to tell if 2 s-expressions are equal
write FRINGE to get the list of atoms at the fringe of an s-expression
write SUBST, then write SUBLIS
Symbolic differentiation
Algebraic simplification
write a simple EVAL and/or APPLY
Understand that Lisp is good for these kinds of no-side-effect functional programs.
It is also useful for stateful side-effect (non-functional) programs, but those are more like "programs" than "functions".
Which is better for a given application depends on the application. In general, it should contain no less, and no more, state information than necessary.
Easy!
M-x lisp-mode
OK, OK, so you might not have Emacs for a brain. In all seriousness, what you need to do is to get really good at recursion. This can be quite a brain warp initially when trying to extend the concept of recursion beyond the canonical examples, but ultimately it will result in more fluid, lispy code.
Also, a lot of people get hung up on the parenthesis, and I don't really know why - the syntax is very simple and consistent and can be mastered in minutes. For me, I came to Scheme after having learned C++ and Java, and I always thought that the difference between "functions" and "operators" was a false dichotomy, and it was refreshing to see that distinction eliminated.
As far as functional programming goes, as long as you can wrap your head around the fact that a function is a first-class value and can be passed both into and out of other functions you should be fine. The usefulness of this will become clear over time, but it's enough that you can write function-taking and function-returning functions.
Finally, I'm not sure what support Clojure has for macros, but they're considered an essential part of lisp. However, I wouldn't worry about learning them until you're deeply familiar with the above items - though macros are incredibly useful and versatile, they're also used less often than the other techniques I mentioned.
I'd start with a language that can be interpreted. I found Moscow ML to be fairly easy. It is a lightweight implementation of Standard ML.
My personal practice is to find a small project (something that might take 3-5 nights hacking away) and implement it. How about a blog filter tool? Maybe just a Towers of Hanoi or Linked List implementation (those are usually 1-night projects).
The way it usually works out is I implement it poorly the first time, throw away what I had, and it finally clicks a few hours in.
A HUGE help is taking a course in something like... um... LISP! :) The homework will force you to confront a lot of the concepts and it clicked for me long before the semester ended.
Good luck!!
Good luck. It took me until about halfway through the "Programming Languages" course in college before Scheme "clicked". Once that happened, though, everything just made sense, and I fell in love with functional programming.
Write a Lisp interpreter in Lisp.
If you haven't alrady, read up on what makes lisp a unique language. If you don't do this first, you'll be trying to do the same things you could do in some other programming languages.
Then try to implement some small thing (try to make it useful to you or you might not have the motivation).
Lisp in a box is a great way to get your feet wet.
For me the important thing is to make sure you do everything in a 'lisp-y' way. Don't be tempted to think 'In Java I'd use a for loop here, how do I do for loops in Lisp?' but to go through enough examples and tutorials (as someone pointed out, SICP is perfect for this) that you can start to spot when code looks 'Lisp-y' and recognise common language paradigms.
I certainly know the feeling of looking at some code I've just written and intuitively knowing that it's correctly idiomatic for that language and platform/framework - that, I think, is when it 'clicks'.
Edit: And kudos for choosing a functional language, lesser students would have just done it in Java :)
Who said it is going to click? I'm always confused
But if you think about how much abstraction it is possible to hide away, behind lisp macros. Then your brain will explode.
:)
I'd check out Programming Clojure. It's a great book for non-lispers.
In addition to what other SO'ers have already suggested, here are my 2 cents:
Start learning the language and try out a few simple numerical/hobby problems in the language
IMPORTANT: Post the solution/code to StackOverflow, asking for people's opinion if that is really the LISPy way to do it.
Best of luck!