Which programming languages provide the best support for self-modifying code?
In particular, since the program will need to make extensive use of self-modifying code, I am looking forward at the ability to remove from memory some parts of code, after they are no longer needed, thus freeing that memory. Also, it would be a plus if there was the ability to identify and index the routines (procedures, functions, etc) with some sort of serial number, so that they could be easily managed in the memory (deleted, cloned etc) at runtime.
Operating systems need to have some more-or-less "self-modifying code" in order to load programs and dynamic link libraries from storage into RAM and later free up that RAM for other things, do relocation fix-ups, etc.
My understanding is that currently the C programming language is by far the most popular language for writing an operating systems.
The OSDev.org wiki has many tips of writing a new custom operating system, including a brief discussion of languages suitable for writing an operating system -- C, Assembly language, Lisp, Forth, C++, C#, PL/1, etc.
Just-in-time (JIT) compilers also need to have some more-or-less "self-modifying code" to compile source text into native instructions and run them, then later free up that memory for the next hot-spot.
Perhaps you could find some OS project or JIT project and use their code with relatively little modification.
A few people, when they say they want "self-modifying code", really want a language that supports homoiconicity such Scheme or some other dialect of Lisp, Prolog, TCL, Curl, etc.
Related
I was reading "The Art of Unix Programming" book and I found a quote saying :
In the Unix world, libraries which are delivered as libraries should come with exerciser programs.
So what exactly is a library exerciser?
A library exerciser is simply a program, or a collection of programs, that is testing that library (by calling some, or most, and ideally all public functions or methods of that library). Read also about unit testing.
BTW, this advice to make library exercisers is IMHO not specific to the Unix world (it should also hold for GNU Hurd, POSIX, VMS and even Windows systems), but generally useful for any software library. I guess it is related to modules and names in linkers. In some exotic, but interesting, programming environments (think of Lisp or Smalltalk machines, or persistent academic OSes like Grasshopper) the very notion of library does not exist, or is so far from Linux-like libraries (written in C or C++) that exercisers might not mean the same thing... ....
Notice that some languages (Ocaml, Go, D, ... but not C11 or C++14) might know some notion of modules and have a module-aware notion of libraries
I would want to compile existing software into presentation that can later be run on different architectures (and OS).
For that I need a (byte)code that can be easily run/emulated on another arch/OS (LLVM IR? Some RISC assemby?)
Some random ideas:
Compiling into JVM bytecode and running with java. Too restricting? C-compilers available?
MS CIL. C-Compilers available?
LLVM? Can Intermediate representation be run later?
Compiling into RISC arch such as MMIX. What about system calls?
Then there is the system call mapping thing, but e.g. BSD have system call translation layers.
Are there any already working systems that compile C/C++ into something that can later be run with an interpreter on another architecture?
Edit
Could I compile existing unix software into not-so-lowlevel binary, which could be "emulated" more easily than running full x86 emulator? Something more like JVM than XEN HVM.
There are several C to JVM compilers listed on Wikipedia's JVM page. I've never tried any of them, but they sound like an interesting exercise to build.
Because of its close association with the Java language, the JVM performs the strict runtime checks mandated by the Java specification. That requires C to bytecode compilers to provide their own "lax machine abstraction", for instance producing compiled code that uses a Java array to represent main memory (so pointers can be compiled to integers), and linking the C library to a centralized Java class that emulates system calls. Most or all of the compilers listed below use a similar approach.
C compiled to LLVM bit code is not platform independent. Have a look at Google portable native client, they are trying to address that.
Adobe has alchemy which will let you compile C to flash.
There are C to Java or even JavaScript compilers. However, due to differences in memory management, they aren't very usable.
Web Assembly is trying to address that now by creating a standard bytecode format for the web, but unlike the JVM bytecode, Web Assembly is more low level, working at the abstraction level of C/C++, and not Java, so it's more like what's typically called an "assembly language", which is what C/C++ code is normally compiled to.
LLVM is not a good solution for this problem. As beautiful as LLVM IR is, it is by no means machine independent, nor was it intended to be. It is very easy, and indeed necessary in some languages, to generate target dependent LLVM IR: sizeof(void*), for example, will be 4 or 8 or whatever when compiled into IR.
LLVM also does nothing to provide OS independence.
One interesting possibility might be QEMU. You could compile a program for a particular architecture and then use QEMU user space emulation to run it on different architectures. Unfortunately, this might solve the target machine problem, but doesn't solve the OS problem: QEMU Linux user mode emulation only works on Linux systems.
JVM is probably your best bet for both target and OS independence if you want to distribute binaries.
As Ankur mentions, C++/CLI may be a solution. You can use Mono to run it on Linux, as long as it has no native bits. But unless you already have a code base you are trying to port at minimal cost, maybe using it would be counter productive. If it makes sense in your situation, you should go with Java or C#.
Most people who go with C++ do it for performance reasons, but unless you play with very low level stuff, you'll be done coding earlier in a higher level language. This in turn gives you the time to optimize so that by the time you would have been done in C++, you'll have an even faster version in whatever higher level language you choose to use.
The real problem is that C and C++ are not architecture independent languages. You can write things that are reasonably portable in them, but the compiler also hardcodes aspects of the machine via your code. Think about, for example, sizeof(long). Also, as Richard mentions, there's no OS independence. So unless the libraries you use happen to have the same conventions and exist on multiple platforms then it you wouldn't be able to run the application.
Your best bet would be to write your code in a more portable language, or provide binaries for the platforms you care about.
I am looking for a JIT compiler or a small compiler library that can be embedded in my program. I indent to use it to compile dynamically generated code that perform complex number arithmetics. The generated code are very simple in structure: no loops, no conditionals, but they can be quite long (a few MB when compiled by GCC). The performance of the resulting machine code is important, while I don't really care about the speed of compilation itself. Which JIT compiler is best for my purpose? Thanks!
Detailed requirements
Support double precision complex number arithmetics
Support basic optimization
Support many CPUs (x86 and x86-64 at least)
Make use of SSE on supported CPUs
Support stack or a large set of registers for local variables
ANSI-C or C++ interface
Cross platform (mainly Linux, Unix)
You might want to take a look at LLVM.
Cint is a c++(ish) environment that offers the ability to mix compiled code and interpreted code. There is a set of optimization tools for the interpreter. ROOT extends this even further by supporting compile and link at run-time at run-time (see the last section of http://root.cern.ch/drupal/content/cint-prompt), though it appears to use the system compiler and thus may not help. All the code is open source.
I make regular use of all these features as part of my work.
I don't know if it makes active use of SIMD instructions, but it seems to meet all your other requirements.
As I see that you are currently using the compile to dynamic library at link on the fly methond, you might consider TCC, though I don't believe that it does much optimization and suspect that it does not support SIMD.
Sounds like you want to be able to compile on the fly and then dynamically load the compiled library (.DLL or .so). This would give you the best performance, with an ANSI-C or C++ interface. So, forget about JITing and consider spawning a C/C++ compiler to do the compilation.
This of course assumes that a compiler can be installed at the point where the dynamically generated code is actually generated.
I have seen a few blog entries on this and have had a discussion or two with my team mates but I would like to see what the stack overflow community thinks.
So why does the Adobe Alchemy Tool create so much faster running flash byte code than the flex compiler?
Also, when will the flex compiler be able to make similar performance gains?
Will it require programmer specific use of special Array's or something of that nature to get the same performance?
Alchemy is an implementation of LLVM in ActionScript. Simply put, it's an virtual machine that uses a ByteArray as it's memory store.
The C code compiled by Alchemy has direct access to "memory" (via some opcodes introduced in Flash 10), allowing it to chunk memory around at it's leisure (including pointers to objects). This results in some, but by no means all, code running faster. Some types of code will actually run slower in Alchemy due to it being a VM running on top of the AVM (another VM).
Additionally, Alchemy does not have native access to ActionScript classes and must access them through interop classes.
The alchemy tool creates code that uses instructions in the flash player that aren't available to the regular compiler (and the talk is that these instructions were exposed especially for alchemy).
Whether the regular compiler will eventually make similar gains, hopefully. It's been proven a few times that the compiler creates substandard code, and there are a couple of projects which optimise the generated code. These may shame Adobe into improving.
Chances are, no, there won't be anything special a programmer needs to do to get these performance gains (though check out the optimising blogs, writing loops in a particular way means they can be optimised better).
I'm toying with the idea of writing a command line interpreter and I suspect that a functional language such as Clojure is well suited to this task.
I am however 5 years out of a CS degree and my only experience with functional languages was a harrowing experience with Haskell in a third year languages course.
So, is a language such as Clojure ideal for this task? If not, what is an ideal language.
Loose requirements:
Has to run on a JVM
Provide an interactive shell where users enter commands with a CLI like syntax
User commands ultimately end up making calls to a remote service using SOAP.
Thanks!
You can approximately do that out-of-the-box with Clojure and Scala, and with Java if you add BeanShell. You might look at the REPL facilities they already have.
I imagine that's suited only for sophisticated users. But really, it's hard to imagine a language that wouldn't do a fine job on a CLI.
Deciding between platforms, the more of a modern system it is, the more it will have scripting language convenience.
I certainly know what I would use given your requirements: JRuby. (It has an out-of-the-box REPL, too.)
I don't think a CLI has any specific requirements language-wise; you could probably do just as well writing it in Java or Scala. Ultimately I think language choice is down to:
Which ones you are most comfortable working with.
Which ones have adequate library support for what you want to do (i.e. web services).