Some time back I was working on an algorithm that processed code, and required a reflections API. We were interested in its implementation for multiple languages, but the reflections API for a language would not work for any other language. So is there any thing like a "universal reflections API" that would work for all languages, or maybe for a few mainstream languages (.NET,Java,Ruby,Python)
If there isnt any, is it possible to build such a thing that can process classes from different languages.
How would you go about having a unified way to process OO code from multiple languages
I don't believe there is universal Reflection API. Any Reflection API depends on the metadata that the compiler generates for the language constructs and these can vary quite a lot from language to language, even though there is a common subset across multiple languages.
In .NET there is CodeDOM, which provides a way to generate a universal syntax tree and then serialize it as (C#, VB .NET etc...) code and/or compile it. Of course that's the mirror image of Reflection, but if anyone ever writes a tool to generate the AST directly from IL the functionality could start to overlap.
In any case its the closest thing I can think of.
A reflection API depends on the metadata generated for the code, so you can have a universal API for all languages on the JVM, or all languages on the CLR...but it wouldn't really be possible to make one that does Python, Java, and VB etc...
If you want a universal API, you need to step outside the language. See our DMS meta-tool for processing arbitrary languages, and answering arbitrary questions, including those you think of as reflection.
(Op asked for support for various languages: DMS has full parsers for C#, VB.net, Java, and Python. Ruby not yet in the list; we're working on it).
Related
I wonder if there is a way to introspect XQuery modules and dynamically access functions. This would help me to implement the GOF strategy pattern as XQuery does not have support for multiple implementations for the same interface.
The problem, of course, is that there seems to be no support for introspection in XQuery except for checking for types of variables.
Any ideas on how this pattern can be implemented in XQuery? (I use MarkLogic 9)
thanks a lot,
K.
PS. Unfortunately, most XQuery resources I could find focus on small details of this or that functionality but I could not find one such resource that takes XQuery as a serious programming language on its own right and addresses such software design issues
XQuery 3.0+ and MarkLogic 9 support first-class functions. In many cases, you can rework common OO design patterns into functional programming equivalents using first-class functions and dependency injection.
Also, you can access in-scope functions via xdmp:functions().
There is a way to get reflection-like functionality in ML, but it's kind of a hack. Take a look at this library, which throws an exception, catches it, and uses the exception payload to inspect the stack:
https://github.com/marklogic/cq/blob/master/lib-debug.xqy
For more XQuery programming patterns and techniques, you might want to review the proceedings from various XML-oriented conferences like Balisage, XML Prague, and XML London. Also, searching GitHub repos for XQuery projects and reading code can be helpful.
There are plenty of options for powerful server side languages, but I can't think of any strongly typed, truly powerful client side languages. Javascript does a lot with forms, basic math, and interacting with the server with AJAX and such, but it has it's limits when compared to other languages.
I'm looking for a strongly typed client side language capable of image processing or data crunching, raw things like that. Or if there are javascript libraries that help enforce data types of variables, that might work too.
In the browser as far as strong typing goes you have Java with GWT (static), Python (dynamic) with Pyjamas, Dart, Opa (static) and a bunch of languages that can cross compile to javascript from LLVM (C, C++, Objective C, Java, Ada, and many many others) thanks to emscripten.
In terms of performance you can take a look at Google Native Client and Javascript typed arrays.
Personally I think Opa is one of the most interesting web languages today.
Haxe is a statically typed language that compiles to various client-side environments, e.g. JavaScript or Flash.
You might want to have a look at ST-JS. This tool allows you to write javascript, but borrowing the syntax and static typing of java. The tools provides a bunch of Java interfaces and classes that acurately reflect the standard javascript library, plus some other common useful libraries (DOM, jquery, etc...). You can then write your source code in java, and the ST-JS translater will transform that static-typed code into plain vanilla (and most importantly readable) javascript code. ST-JS is also compatible with any other javascript library that you would want to use, as long as you write the corresponding java interfaces.
Another advantage of ST-JS, is that since you are writing java code, you can use all the really powerful tools from the java ecosystem. Refactoring tools in all IDEs will work out of the box. Static analysis tools such as Findbugs, PWD, checkstyle also work great and are very accurate on ST-JS enabled code.
You can think of ST-JS as being somewhere in the middle between GWT (completely hides the javascript standard libraries from you, and prevents you from using external libraries) and some javascript static analysis tools such as JSLint.
TypeScript, Flow and Dart have optional static typing. TypeScript is a superset of JavaScript (you can annotate existing libraries with type information); Dart is a separate language that comes with a growing standard library.
Scala.js is another option. Its JavaScript interoperability is important to read.
I search for a programming language for which a compiler exists and that supports self modifying code. I’ve heared that Lisp supports these features, but I was wondering if there is a more C/C++/D-Like language with these features.
To clarify what I mean:
I want to be able to have in some way access to the programms code at runtime and apply any kind of changes to it, that is, removing commands, adding commands, changing them.
As if i had the AstTree of my programm. Of course i can’t have that tree in a compiled language, so it must be done different. The compile would need to translate the self-modifying commands into their binary equivalent modifications so they would work in runtime with the compiled code.
I don’t want to be dependent on an VM, thats what i meant with compiled :)
Probably there is a reason Lisp is like it is? Lisp was designed to program other languages and to compute with symbolic representations of code and data. The boundary between code and data is no longer there. This influences the design AND the implementation of a programming language.
Lisp has got its syntactical features to generate new code, translate that code and execute it. Thus pre-parsed code is also using the same data structures (symbols, lists, numbers, characters, ...) that are used for other programs, too.
Lisp knows its data at runtime - you can query everything for its type or class. Classes are objects themselves, as are functions. So these elements of the programming language and the programs also are first-class objects, they can be manipulated as such. Dynamic language has nothing to do with 'dynamic typing'.
'Dynamic language' means that the elements of the programming language (for example via meta classes and the meta-object protocol) and the program (its classes, functions, methods, slots, inheritance, ...) can be looked at runtime and can be modified at runtime.
Probably the more of these features you add to a language, the more it will look like Lisp. Since Lisp is pretty much the local maximum of a simple, dynamic, programmable programming language. If you want some of these features, then you might want to think which features of your other program language you have to give up or are willing to give up. For example for a simple code-as-data language, the whole C syntax model might not be practical.
So C-like and 'dynamic language' might not really be a good fit - the syntax is one part of the whole picture. But even the C syntax model limits us how easy we can work with a dynamic language.
C# has always allowed for self-modifying code.
C# 1 allowed you to essentially create and compile code on the fly.
C# 3 added "expression trees", which offered a limited way to dynamically generate code using an object model and abstract syntax trees.
C# 4 builds on that by incorporating support for the "Dynamic Language Runtime". This is probably as close as you are going to get to LISP-like capabilities on the .NET platform in a compiled language.
You might want to consider using C++ with LLVM for (mostly) portable code generation. You can even pull in clang as well to work in C parse trees (note that clang has incomplete support for C++ currently, but is written in C++ itself)
For example, you could write a self-modification core in C++ to interface with clang and LLVM, and the rest of the program in C. Store the parse tree for the main program alongside the self-modification code, then manipulate it with clang at runtime. Clang will let you directly manipulate the AST tree in any way, then compile it all the way down to machine code.
Keep in mind that manipulating your AST in a compiled language will always mean including a compiler (or interpreter) with your program. LLVM is just an easy option for this.
JavaScirpt + V8 (the Chrome JavaScript compiler)
JavaScript is
dynamic
self-modifying (self-evaluating) (well, sort of, depending on your definition)
has a C-like syntax (again, sort of, that's the best you will get for dynamic)
And you now can compile it with V8: http://code.google.com/p/v8/
"Dynamic language" is a broad term that covers a wide variety of concepts. Dynamic typing is supported by C# 4.0 which is a compiled language. Objective-C also supports some features of dynamic languages. However, none of them are even close to Lisp in terms of supporting self modifying code.
To support such a degree of dynamism and self-modifying code, you should have a full-featured compiler to call at run time; this is pretty much what an interpreter really is.
Try groovy. It's a dynamic Java-JVM based language that is compiled at runtime. It should be able to execute its own code.
http://groovy.codehaus.org/
Otherwise, you've always got Perl, PHP, etc... but those are not, as you suggest, C/C++/D- like languages.
I don’t want to be dependent on an VM, thats what i meant with compiled :)
If that's all you're looking for, I'd recommend Python or Ruby. They can both run on their own virtual machines and the JVM and the .Net CLR. Thus, you can choose any runtime you want. Of the two, Ruby seems to have more meta-programming facilities, but Python seems to have more mature implementations on other platforms.
Somebody that I work with and respect once remarked to me that there shouldn't be any need for the use of reflection in application code and that it should only be used in frameworks. He was speaking from a J2EE background and my professional experience of that platform does generally bear that out; although I have written reflective application code using Java once or twice.
My experience of Ruby on Rails is radically different, because Ruby pretty much encourages you to write dynamic code. Much of what Rails gives you simply wouldn't be possible without reflection and metaprogramming and many of the same techniques are equally as applicable and useful to your application code.
Do you agree with the viewpoint that reflection is for frameworks only? I'd be interested to hear your opinions and experiences.
There's the old joke that any sufficiently sophisticated system written in a statically-typed language contains an incomplete, inferior implementation of Lisp.
Since your requirements tend to become more complicated as a project evolves, you often eventually find that the common idioms in statically-typed object systems eventually hit a wall. Sometimes reaching for reflection is the best solution.
I'm happy in dynamically-typed languages like Ruby, and statically-typed languages like C#, but the implicit reflection in Ruby often makes for simpler, easier-to-read code. (Depending on the metaprogramming magic required, sometimes harder to write).
In C#, I've found problems that couldn't be solved without reflection, because of information I didn't have until runtime. One example: When trying to manipulate some third-party code that generated proxies to Silverlight objects running in another process, I had to use reflection to invoke a specific strongly-typed "Generic" version of a method, because the marshalling required the caller to make an assumption about the type of the object in the other process was in order to extract the data we needed from it, and C# doesn't allow the "type" of the generic method invocation to be specified at run time (except with reflection techniques). I guess you could argue our tool was kind of a framework, but I could easily imagine a case in an ordinary application facing a similar problem.
Reflection makes DRY a lot easier. It's certainly possible to write DRY code without reflection, but it's often much more verbose.
If some piece of information is encoded in my program in one way, why wouldn't I use reflection to get at it, if that's the easiest way?
It sounds like he's talking about Java specifically. And in that case, he's just citing a special case of this: in Java, reflection is so wonky it's almost never the easiest way to do something. :-) In other languages like Ruby, as you've seen, it often is.
Reflection is definitely heavily used in frameworks, but when used correctly can help simplify code in applications.
One example I've seen before is using a JDK Proxy of a large interface (20+ methods) to wrap (i.e. delegate to) a specific implementation. Only a couple of methods were overridden using a InvocationHandler, the rest of the methods were invoked via reflection.
Reflection can be useful, but it is slower that doing a regular method call. See this reflection comparison.
Reflection in Java is generally not necessary. It may be the quickest way to solve a certain problem, but I would rather work out the underlying problem that causes you to think it's necessary in app code. I believe this because it frequently pushes errors from compile time to run time, which is always a Bad Thing for large enough software that testing is non-trivial.
I disagree, my application uses reflection to dynamically create providers. I might also use reflection to control logic flow, if the logic is simple and doesn't warrant a more complicated pattern.
In C# I use reflection to grab attributes off Enumeration which help me determine how to display an enumeration to an end user.
I disagree, reflection is very useful in application code and I find myself using it quite often. Most recently, I had to use reflection to load an assembly (in order to investigate its public types) from just the path of the assembly.
Several opinions on this subject are expressed here...
What is reflection and why is it useful?
Use reflection when there is no other way! This is a matter of performance!
If you have looked into .NET performance pitfalls before, it might not surprise you how slow the normal reflection is: a simple test with repeated access to an int property proved to be ~1000 times slower using reflection compared to the direct access to the property (comparing the average of the median 80% of the measured times).
See this: .NET reflection - performance
MSDN has a pretty nice article about When Should You Use Reflection?
If your problem is best solved by using reflection, you should use it.
(Note that the definition of 'best' is something learnt by experience :)
The definition of framework vs. application isn't all that black & white either. Sometimes your app needs a bit of framework to do its job well.
I think the observation that there shouldn't be any need for the use of reflection in application code and that it should only be used in frameworks is more or less true.
On the spectrum of how coupled some piece of code are, code joined by reflection are as loosely coupled as they come.
As such, the code which is doing it's job via reflection can quite happily fulfil it's role in life knowing not-a-thing about the code which is using it.
I work in a Microsoft environment, so I can use my C# hammer on any nails I come across. That being said, what languages (compiled, interpreted, scripting, functional, any types!) complement knowing C#, and for what purposes? For example, I've moved a lot of script functionality away from compiled console apps and into Powershell scripts. If you're an MS developer, have you found a niche in your world for other languages like F#, IronRuby, IronPython, or something similar, and what niche do they fill?
Note: this question is directed at the Microsoft dev people since I can't run off and start installing LAMP stacks around my company, and therefore having to support it forever. :) However, feel free to mention any other languages that you found interesting to fulfill a certain task/role in your world apart from your main language.
Python/Perl/Ruby/PowerShell are great supplements to C#/VB.NET. If your boss hands you a text file and says insert it into the database once or twice, then any of Perl/Python/Ruby (I'm not sure about powershell but I imagine it is not that much more difficult) should be fine to parse it. Either way, for your main applications you will probably be stuck in C#. You can use one of the more dynamic languages to do code generation in C#.
Since you are in a Microsoft Environment, probably your best chance at getting your solution accepted is PowerShell. Next to that I'd say IronPython or something else that integrates with the CLR. But main issue is that for someone else to maintain what you do, they would have to know whatever language you are using. MS in the future has plans to use PowerShell a lot more, so it is probably easier to justify PowerShell then say Python/Perl/Ruby.
If you are just processing a text file for one time use. Or creating a one time code generator to generate all the code and then intend to maintain the generated code, then it doesn't matter. You are the one who will consume the results and if you save time using Perl then more power to you. But if you are doing something that will be used over and over again (like an active code generator where you change the templates and run the generator instead of maintaining the generated code) then other developers working on what you did will need to know the language you used. It is much harder to argue learning Perl/Ruby/Python in a Microsoft Shop. But PowerShell seems like the easier argument. I think the MS grand plan is that eventually applications will expose more functionality for power shell through commandlets. Assuming this happens then PowerShell is even more of a no-brainer because it will expose tons of scriptable functionality that you won't get any other way.
A nice scripting language is always a good tool to have on you belt. See Ruby, or Python.
I use python for prototyping, since there's almost no turn around time between edits and actually running the new version of the code. I may even end up using it for a real project - the more I use it, the more I like it.
It will take some getting used to as a C# programmer, though - the indentation-defines-structure system it uses is a little weird at first.
Since you are in a MS shop, I would suggest PowerShell as a decent scripting language to learn. It plays well with C#.
I'm a big fan of Ruby too.
I'd like to second or third python. Specifically, IronPython (ttp://www.codeplex.com/Wiki/View.aspx?ProjectName=IronPython) lets you learn python but also gives you access to the .net framework goodies.
It's quite nice for scripting-related tasks so it'll probably be useful for your day-to-day coding life, and also a nice way to muck around in an experimental coding/prototyping way.
While it's a bit of a fringe language, I'm compelled to mention Erlang. Erlang is an excellent language to have in your toolbox since it's unusual strengths tend to compliment other programming platforms. Erlang is very useful for building distributed, concurrent, fault-tolerant systems. It's used a lot in the instant-messaging and telephony world where there's a need for distributed, yet interconnected architectures.
Maybe play around with Boo and see what you think.
Boo at Codehaus.org
Boo at Wikipedia
If your using the .Net framework the language is really not important as the compiler and interpreter create the same IL code in any case.
If you step away from the .Net world, I am of the opinion that development tools and languages are a tool box. I strive to use the right tools for the job at hand, taking into consideration what the skill base of the other developers are and what direction a company is seeking of course (I'm a consultant).
I'm with jjnguy. Try one of the scripting languages. Plus as a bonus, when you learn Ruby/PythonPearl, etc...it's a gateway drug...err language to developing for other environments.
Trying out languages outside of your normal toolbox will give you new ways of approaching things in your current favorite language. Even if you don't use them for serious projects languages like Perl (for data mangling), Lisp (functional programming), and Javascript (prototype based programming) will teach you new ways to think about problems in your current language.
As a web developer by trade, you might look into the XSLT/XPath family as for certain types of XML processing they can be very powerful tools.
Granted, in C# 3.x Linq2Xml exposes some similar functionality inline.
XSLT, however, can be a powerful way to separate data from presentation in your apps.
I am very interested in F# and some of the other new languages in the CLR/DLR. The DLR languages might be a lot better for your UI, because they don't make you cast a lot of stupid things.
However, I think that it is important to keep in mind tha learning a new language, especially in a new area, like functional programming, is always a good way to re-train your mind so that you are exposed to new concepts and you can code better in your language of choice, even if you never use those new languages.
Check out Boo - it runs on top of the .NET stack, but its syntax is more like Python.
To learn a new language that complements C#, I'd go with C++. You can use it in a 'way better than p/invoke' style to get access to unmanaged code from your C# apps. You can then start using it to write the memory-constrained apps, and/or performance-critical bits in, if you find some of your .NET applications start hogging all the RAM and/or CPU or just generally aren't as fast as you'd like.