I was reflecting on Rich Hickey's talk, Simple Made Easy, when I got to the "What's in your Toolkit?" slide. There is a list of contrasts between complexity and simplicity, this one piqued my interest:
Complexity: Conditionals, Simplicity: Rules
Does anyone have any insight into what sorts of things Rich was proposing here?
I guess he is reffering on Conditionals as boolean expressions in programming languages. If you have many conditional Statements like loops and switches it will get complex very fast. The simpler way is to define Rules. Rules are in a more natural language, you can handle them in a more abstract way to describe your conditions.
Look at the Windows access Management where you can assign rules to users about the rights they have. Like changing the desktop wallpaper. The rules are easy to understand and it's an simple boolean decisions if an rule is applied or not. But if your extract all the conditionals behind the rules it will get very complex. Because you have to care about each and every thing that could affect the wallpaper in this case.
In the end both are methods to describe a Condition, but the one is simpler that the other.
Related
I am just beginning to learn about rule-based systems, and am having trouble understanding how they can handle recursion.
Some quick background. I have studied a little programming language theory and so am pretty familiar with Context-Free Grammars and stuff like that. I have recently come across rule-based systems and am wondering how they may be related to automata and recursion.
My question is: how do rule-based systems handle recursion? In particular, could a rule-based system be used for parsing nested parentheses? (Or parsing a CFG)?
I don't have a deep understanding of rule-based systems yet, but from what it seems, rules are never organized into a hierarchy or a network. Is this true, or is there a way to represent them as a hierarchy or network?
Without the ability to create a hierarchy of rules, it seems like the rule-system memory would have to keep track of the context manually, and that context would be used to test conditions. So like:
if parentConditionX and parentParentConditionY {
perform some action
}
But then the problem is, how do you return to a "parent" rule (like a parent function context)? If rules can't be organized/executed hierarchically, it seems you can't do recursion? That's where I'm confused.
Any ideas?
I have a system where 'dynamic logic' is implemented as Drools Rules, using Rules engine.
For each client implementation, custom pricing and tax calculation logic is implementted using the drl files for that specific implementation.
rule 'abc'
when
name = 'X'
then
price= '12'
end
And one rule's condition is dependent on what's set in the previous rules, so there is basically rules transition.
This is really painful as the drools rules are not sequential programming and is not developer friendly. There get introduced lots of bugs due to mis-interpretation of how drools evaluates.
Are there a better 'java/groovy' alternative that could easily replace it?
I think the answer is going to depend on what you ultimately want the end solution to be. If you are wanting to pull your business rules out of a rules engine and put them into java/groovy, that is very different from wanting to pull them out of one rules engine and into another.
Your questions seems to lean towards the prior, so I'll address that. Be very careful with this approach. The previous individuals who implemented this appear to have done this the proper way with respect to using the Rete algorithm, as it sounds like the firing of one rule can execute other rules, this is good business rules - they aren't sequential they are declarative. Remember that imperative software is written for engineers mostly, it doesn't map back to the real world 100% of the time :)
If you put want to move this into java/groovy you are moving into an imperative language, which could put you into an if/then/else hell. I would suggest the following:
Isolate this code away from the rest of your codebase - you will be doing a lot of maintenance on this code in the future when the business changes their rules. Good interface design and encapsulation here will payoff big time down the road.
Develop some type of DSL with your business customer so when they say something like "Credit Policy", you know exactly what they are referring to and can change the related rules appropriately.
Unit Test, Unit Test, Unit Test. This applies to your current configuration too. If you are finding bugs now, why aren't your tests? It doesn't take long to setup junit to create an object and invoke your Drools engine and test the response. If you add some loops to test ranges of variables that expect the same response you can be in the hundreds of thousands of tests in no time.
As an aside: If you don't want to go down this route, then I highly suggest getting some training on Drools so that you understand the engine and Rete if you don't already. There are some big wins you can make with your customers if you are able to quickly translate their rules into implementable software.
Everyone is aware of Dijkstra's Letters to the editor: go to statement considered harmful (also here .html transcript and here .pdf). I was wondering is anyone attempted to find a way to make code using goto's re-usable and maintainable and not-harmful by adding any other language extensions or developing a language which allows for gotos.
The reason I ask the question is that it occurs to me that code written in Assembly language often used goto's and global variables to make the program work well within a limited space. The Atari 2600 which had 128 bytes of ram and the program was loaded from ROM cartridge. In this case, it was better to use unstructured programming and to make the most of the freedoms this allows to make the most of a very limited space for the program.
When you compare this with a game programmed today without the use of gotos, the game takes up much more space.
Then it occurs to me that perhaps its possible to program with the use of gotos if some rules or other language changes are made to support this, then the negative effects of gotos could be reduced or eliminated. Has anyone tried to find a way to make goto's NOT considered harmful by creating a language or some rules to follow which allow gotos to be not harmful.
If no-one looked for a way to use gotos in a non-harmful way then perhaps we adopted structured programming un-necessarily based solely on this paper? Perhaps there is another solution which allows for the use of gotos without the down-side.
Comparing gotos to structured programming is comparing a situation where the programmer has to remember what every labels in the code actually mean and do, and where there are, to a situation where the conditional branches are explicitly described.
As of the advantages of the goto statement regarding the place a program might take, I think that games today are big because of the graphic and sound resources they use. That is, show 1,000,000 polygons. The cost of a goto compared to that is totally neglectable.
Moreover, the structural statements are ultimately compiled into goto ("jmp") statements by the compiler when outputting assembly.
To answer the question, it might be possible to make goto less harmful by creating naming and syntax conventions. Enforcing these conventions into rules is however pretty much what structural programming does.
Linus Torvald argued once that goto can make source code clearer, but goto is useful in so very special cases that I would not dare use it as a programmer.
This question is somehow related to yours, since I think this one of the most common situations where a goto is needed.
What is a combinator??
Is it "a function or definition with no free variables" (as defined on SO)?
Or how about this: according to John Hughes in his well-known paper on Arrows, "a combinator is a function which builds program fragments from program fragments", which is advantageous because "... the programmer using combinators constructs much of the desired program automatically, rather than writing every detail by hand". He goes on to say that map and filter are two common examples of such combinators.
Some combinators which match the first definition:
S
K
Y
others from To Mock a Mockingbird (I may be wrong -- I haven't read this book)
Some combinators which match the second definition:
map
filter
fold/reduce (presumably)
any of >>=, compose, fmap ?????
I'm not interested in the first definition -- those would not help me to write a real program (+1 if you convince me I'm wrong). Please help me understand the second definition. I think map, filter, and reduce are useful: they allow me to program at a higher level -- fewer mistakes, shorter and clearer code. Here are some of my specific questions about combinators:
What are more examples of combinators such as map, filter?
What combinators do programming languages often implement?
How can combinators help me design a better API?
How do I design effective combinators?
What are combinators similar to in a non-functional language (say, Java), or what do these languages use in place of combinators?
Update
Thanks to #C. A. McCann, I now have a somewhat better understanding of combinators. But one question is still a sticking point for me:
What is the difference between a functional program written with, and one written without, heavy use of combinators?
I suspect the answer is that the combinator-heavy version is shorter, clearer, more general, but I would appreciate a more in-depth discussion, if possible.
I'm also looking for more examples and explanations of complex combinators (i.e. more complex than fold) in common programming languages.
I'm not interested in the first definition -- those would not help me to write a real program (+1 if you convince me I'm wrong). Please help me understand the second definition. I think map, filter, and reduce are useful: they allow me to program at a higher level -- fewer mistakes, shorter and clearer code.
The two definitions are basically the same thing. The first is based on the formal definition and the examples you give are primitive combinators--the smallest building blocks possible. They can help you to write a real program insofar as, with them, you can build more sophisticated combinators. Think of combinators like S and K as the machine language of a hypothetical "combinatory computer". Actual computers don't work that way, of course, so in practice you'll usually have higher-level operations implemented behind the scenes in other ways, but the conceptual foundation is still a useful tool for understanding the meaning of those higher-level operations.
The second definition you give is more informal and about using more sophisticated combinators, in the form of higher-order functions that combine other functions in various ways. Note that if the basic building blocks are the primitive combinators above, everything built from them is a higher-order function and a combinator as well. In a language where other primitives exist, however, you have a distinction between things that are or are not functions, in which case a combinator is typically defined as a function that manipulates other functions in a general way, rather than operating on any non-function things directly.
What are more examples of combinators such as map, filter?
Far too many to list! Both of those transform a function that describes behavior on a single value into a function that describes behavior on an entire collection. You can also have functions that transform only other functions, such as composing them end-to-end, or splitting and recombining arguments. You can have combinators that turn single-step operations into recursive operations that produce or consume collections. Or all kinds of other things, really.
What combinators do programming languages often implement?
That's going to vary quite a bit. There're relatively few completely generic combinators--mostly the primitive ones mentioned above--so in most cases combinators will have some awareness of any data structures being used (even if those data structures are built out of other combinators anyway), in which case there are typically a handful of "fully generic" combinators and then whatever various specialized forms someone decided to provide. There are a ridiculous number of cases where (suitably generalized versions of) map, fold, and unfold are enough to do almost everything you might want.
How can combinators help me design a better API?
Exactly as you said, by thinking in terms of high-level operations, and the way those interact, instead of low-level details.
Think about the popularity of "for each"-style loops over collections, which let you abstract over the details of enumerating a collection. These are just map/fold operations in most cases, and by making that a combinator (rather than built-in syntax) you can do things such as take two existing loops and directly combine them in multiple ways--nest one inside the other, do one after the other, and so on--by just applying a combinator, rather than juggling a whole bunch of code around.
How do I design effective combinators?
First, think about what operations make sense on whatever data your program uses. Then think about how those operations can be meaningfully combined in generic ways, as well as how operations can be broken down into smaller pieces that are connected back together. The main thing is to work with transformations and operations, not direct actions. When you have a function that just does some complicated bit of functionality in an opaque way and only spits out some sort of pre-digested result, there's not much you can do with that. Leave the final results to the code that uses the combinators--you want things that take you from point A to point B, not things that expect to be the beginning or end of a process.
What are combinators similar to in a non-functional language (say, Java), or what do these languages use in place of combinators?
Ahahahaha. Funny you should ask, because objects are really higher-order thingies in the first place--they have some data, but they also carry around a bunch of operations, and quite a lot of what constitutes good OOP design boils down to "objects should usually act like combinators, not data structures".
So probably the best answer here is that instead of combinator-like things, they use classes with lots of getter and setter methods or public fields, and logic that mostly consists of doing some opaque, predefined action.
What should be the measures that should be used to identify that code is over abstracted and very hard to understand and what should be done to reduce over abstraction?
"Simplicity over complexity, complexity over complicatedness"
So - there's a benefit to abstract something only if You are "de-leveling" complicatedness to complexity. Reasons to do that can vary: better modularity, better encapsulation etc.
Identifying over abstraction is a chicken and egg problem. In order to reduce over abstraction You need to understand actual reason behind code lines. That includes understanding idea of particular abstraction itself (in contrast to calling it over abstracted cause of lack of understanding). And that's not enough - You need to know a better, simpler solution to prove that it's over abstracted.
If You are looking for tool that could do it in Your place - look no more, only mind can reliably judge that.
I will give an answer that will get a LOT of down votes!
If the code is written in an OO language .. it is necessarily heavily over-abstracted. The purer the language the worse the problem.
Abstraction should be used with great caution. If in doubt always use concrete data structures. (You can always abstract later, this is easier than de-abstraction :)
You must be very certain you have the right abstraction in your current context, and you must be very sure that concept will stand the test of change. Abstraction has a high price in performance of both the code and the coder.
Some weak tests for over-abstraction: if the data structure is a product type (struct in C) and the programmer has written get and set method for each field, they have utterly failed to provide any real abstraction, disabled operators like C increment, for no purpose, and simply not understood that the struct field names are already the abstract representation of a product. Duplicating and laming up the interface is not a good idea.
A good test for the product case is whether there exist any data invariants to maintain. For example a pair of integers representing a rational number is almost sufficient, there's little need for any abstraction because all pairs are valid except when the denominator is zero. However for performance reasons one may choose to maintain an invariant, typically the denominator is required to be greater than zero, and the numerator and denominator are relatively prime. To ensure the invariant, the product representation is encapsulated: the initial value protected by a constructor and methods constrained to maintain the invariant.
To fix code I recommend these steps:
Document the representation invariants the abstraction is maintaining
Remove the abstraction (methods) if you can't find strong invariants
Rewrite code using the method to access the data directly.
This procedure only works for low level abstraction, i.e. abstraction of small values by classes.
Over abstraction at a higher level is much harder to deal with. Ideally you'd refactor the code repeatedly, checking to see after each step it continues to work. However this will be hard, and sometimes a major rewrite is required, rather than a refinement. It's probably not worth it unless the abstraction is so far off base it is not tenable to continue to maintain it.
Download Magento and have a look at the code, read some documents on it and have a look at their ERD: http://www.magentocommerce.com/wiki/_media/doc/magento---sample_database_diagram.png?cache=cache
I'm not joking, this is over-abstraction.. trying to please everyone and cover every base is a terrible idea and makes life extremely difficult for everyone.
Personally I would say that "What is the ideal level of abstraction?" is a subjective question.
I don't like code that uses a new line for every atomic operation, but I also don't like 10 nested operations within one line.
I like the use of recursive functions, but I don't appreciate recursion for the sole sake of recursion.
I like generics, but I don't like (nested) generic functions that e.g. use different code for each specific type that's expected...
It is a matter of personal opinion as well as common sense. Does this answer your question?
I completely agree with what #ArnisLapsa wrote:
"Simplicity over complexity, complexity over complicatedness"
And that
an abstraction is used to "de-level" those, from complicated to complex
(and from complex to simpler)
Also, as stated by #MartinHemmings a good abstraction is quite subjective because we don't all think the same way. And actually our way of thinking change with time. So Something that someone find simple might looks complex to others, and even become simpler with more experiences. Eg. A monadic operation is something trivial for functional programmer, but can be seriously confusing for others. Similarly, a design with mutable object communicating with each other can be natural for some and feel un-trackable for others.
That being said, I would like to add a couple of indicators. Note that this applies to abstractions used in code-base, not "paradigm abstraction" such as everything-is-a-function, or everything-is-designed-as-objects. So:
To the people it concerns, the abstraction should be conceptually simpler than other alternatives, without looking at the implementation. If you find that thinking of all possible cases is simpler that reasoning using the abstraction, then this abstraction is not suitable (for you)
Its implementation should reason only about the abstraction, not the specific cases that it will be used for. As soon as the abstraction implementation has parts made for specific cases, it indicates an "unfit" abstraction. And increasing generalization to cope with each new case, is going the wrong way (and tends to fall to the next issue).
A very common indicator of over-abstraction I have found (and actually fell for) are abstractions that represent more than what is needed, now. As much as possible, they should allow to do exactly what is required, but nothing more. For example, say you're thinking of, or already have, a "2d point" abstraction for which you can define many operators you need. Then you have another need that could really be a "4d point" similar to the 2d. Don't start to use a "Ndimensionnal point" abstraction, especially thinking that you might later need it. Maybe you'll never have anything else than 2 and 4d (because it stays as "a good idea" in the backlog forever) but instead some requirements pops to convert 4d points into pairs of 2d points. That's going to be hard to generalize to n-dimensions. So, each abstraction can be checked to cover and only cover the actual needs. In my point example, the complexity "n-dimensional" is actually only used to cope with the 2 and 4d cases (and the 4d might not even be used that much).
Finally, in a more global point of view, a code-base that has many not related abstractions, is an indicator that the dev team tends to abstract every little issues. So probably many of them are or became over-abstracted.