How to use Common Lisp sort of like a smalltalk image - common-lisp

Goal
I would like to have my Common Lisp (SBCL + GNU Emacs + Slime) environment be sort of like a Smalltalk image in that I want to have a big ball of mud of all my code organized in packages and preferably projects. In other words I have messed about a bit with save-lisp-and-die and setting Lisp in Emacs to bring up the saved image. Where I get lost is the appropriate way to make it work with Swank.
Problem
I believe it is required to put swank hooks inside my Lisp image before save-lisp-and-die. But it seems a bit fragile as on change to either my SBCL version or Slime version it seems to throw a version mismatch.
Question
Am I missing something? Do people work this way or tend to be more separate project as a loadable set of packages under ASDF?
I really miss the Smalltalk way and feel like per project ASDF is a bit clunkier and more rooted in the file system. In comparison it reminds me too much of every other language and their app/project orientation. OTOH it seem a bit more stable-ish re-versions of depended upon packages. Well, the entire versioning hell across languages is another matter.
Any hints how to do what I want or why it isn't such a good idea would be much appreciated.

Images
Common Lisp implementations like SBCL support images. The idea of saved memory appeared early in Lisp in the 60s.
Smalltalk took that idea from Lisp. In many Smalltalk implementations images might be portable (OS, runtime, ...) - especially when using machine independent byte code. SBCL OTOH compiles to native machine code.
Managed source code
Smalltalk added the idea of managed source code. Smalltalk often uses a simple database plus a change log to store source code. One Lisp doing something similar was Xerox Interlisp - but with slightly different approaches.
Other Lisp implementations / IDEs don't support managed source code that way - only the Xerox Interlisp variants - AFAIK.
DEFSYSTEM
In Common Lisp the use of defsystem facilities like ASDF and IDEs like GNU Emacs + SLIME is much more file system based. Code resides in multiple systems, which are files in a directory with a system description.
It's not even clear that it's meaningful to load a newer version of a system into a Lisp system where an older version is loaded. One might be able to arrange that, but there is nothing preventing me from messing that up.
Updating Lisp
Updating a Lisp like SBCL from one version to another might
make the saved image incompatible to the runtime
make the compiled code in FASL files incompatible with the runtime
You might save an image with the runtime included/bundled. That way you have the right combination of image and runtime.
But when you update the runtime, you usually/often need to regenerate a new compatible images with your code loaded.
Since SBCL brings releases once a month, there is a temptation to update regularly. Other implementations might use different strategies: LispWorks is an example. LispWorks is released much less often and publishes patches between releases, which are loaded into the released version.
Updating SLIME
I have no idea if it would be possible to update a loaded SLIME (a SLIME which has been already loaded in an earlier version into a Lisp system) by loading a new version on top. Probably a good idea to check with the SLIME maintainers.

Related

Is cl-http still usable and how to install it

How to install fantastic beast (CL-HTTP) on SBCL (or ccl) if it is still usable and why people are not using it anymore (extra question)?
I think that this is an old lisp project. You can find the sources and the binaries here. Since is written in Common Lisp it should work (maybe with some changes) in modern implementations (last release is from 2003), and probably modify the way that the system is build with ASDF
This is a good project for learning, but if you want to make web development with common lisp I recommend you to use first Quicklisp, for installing the libraries and take a look here for web frameworks or servers:
http://quickdocs.org/search?q=web
I also recommend that take a look to:
https://github.com/edicl/hunchentoot (as suggested in a comment by #Martin Buchmann and is an http server)
https://shirakumo.github.io/radiance/ (this is my favorite, have a nice tutorial)
https://github.com/fukamachi/caveman (all the things comming from Fukamachi are really great common lisp tools)
https://github.com/skypher/weblocks (old but still mantained)

Are there features of R that are system-dependent?

My co-workers would like to make sure that our work in R is platform-independent, specifically that code will run on Linux, Mac, and Windows, and that files created on one system will work on other systems.
Since the issue has come up before in my group, I would appreciate a general answer that will make it easier for me to confidently assure my collaborators that there will not be an issue. E.g., it would help to have a reference other than "because (subject matter expert) said so on SO".
Generally, is there a way to know if any features of R are platform-specific (can I assume that this would be stated in a function's help)?
Are there packages or functions that I can be confident will be platform-independent?
Are there types of packages or functions that I should be wary of?
I have previously asked two questions about the cross-platform readability of files created by R: What are the disadvantages of using .Rdata files compared to HDF5 or netCDF? and Are R objects dumped using `dump` readable cross-platform?
Besides Carl's answer, the obvious way to ensure that your work in platform-independent is to test on all platforms.
Which is precisely what CRAN does with its 3800+ packages, and you have access to logs here.
In short, R really tries hard to be platform-independent, and mostly succeeds. To do so with your code, it is up to you to avoid APIs or tools which introduce dependencies. Look at abstractions like system.file(package="boot") and the functions they use---you can easily abstract file-system "roots", and separators are already taken care of.
Check cran.r-project.org for package listings. Every package has a page which will tell you if it's passed testing for different operating systems. Further, as you suggested, the help files are pretty explicit about OS dependencies.
R is "smart" enough to translate "/" to "\" in pathnames for those poor folks working in Windows.
Generally speaking, graphics access is the area most likely to have platform dependencies. Obviously if you system lacks {X11, ImageMagick, ..} you're stuck anyway.
Besides Carl's and Dirk's comments, you should understand that any package that requires compilation from source (as do many (all?) packages that are on Omegahat, Rforge or r-forge) will need to be done on a machine that has the proper C and Fortran libraries. Some interesting packages depend on GTK+ and Tcl/Tk, and there may be a need to make sure you can get the right versions. The http://r.research.att.com/ page that Simon Urbanek maintains is a useful resource for keeping up with supporting resources for Macs.

JIT compilers for math

I am looking for a JIT compiler or a small compiler library that can be embedded in my program. I indent to use it to compile dynamically generated code that perform complex number arithmetics. The generated code are very simple in structure: no loops, no conditionals, but they can be quite long (a few MB when compiled by GCC). The performance of the resulting machine code is important, while I don't really care about the speed of compilation itself. Which JIT compiler is best for my purpose? Thanks!
Detailed requirements
Support double precision complex number arithmetics
Support basic optimization
Support many CPUs (x86 and x86-64 at least)
Make use of SSE on supported CPUs
Support stack or a large set of registers for local variables
ANSI-C or C++ interface
Cross platform (mainly Linux, Unix)
You might want to take a look at LLVM.
Cint is a c++(ish) environment that offers the ability to mix compiled code and interpreted code. There is a set of optimization tools for the interpreter. ROOT extends this even further by supporting compile and link at run-time at run-time (see the last section of http://root.cern.ch/drupal/content/cint-prompt), though it appears to use the system compiler and thus may not help. All the code is open source.
I make regular use of all these features as part of my work.
I don't know if it makes active use of SIMD instructions, but it seems to meet all your other requirements.
As I see that you are currently using the compile to dynamic library at link on the fly methond, you might consider TCC, though I don't believe that it does much optimization and suspect that it does not support SIMD.
Sounds like you want to be able to compile on the fly and then dynamically load the compiled library (.DLL or .so). This would give you the best performance, with an ANSI-C or C++ interface. So, forget about JITing and consider spawning a C/C++ compiler to do the compilation.
This of course assumes that a compiler can be installed at the point where the dynamically generated code is actually generated.

What is currently the best build system [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
A few years ago I looked into using some build system that isnt Make, and tools like CMake and SCons seemed pretty primitive. I'd like to find out if the situation has improved. So, under the following criteria, what is currently the best build tool:
platform agnostic: should work on windows, linux, mac
language agnostic: should have built-in support for common things like building C/C++ and other static langs. I guess it doesn't need to support the full autotools suite.
extensible: I need to be able to write rules to generate files, like from restructuredText, latex, custom formats, etc. I dont really care what language I have to write the rules in, but I would prefer a real language rather than a DSL.
I would prefer to avoid writing any XML by hand, which I think for example ant requires.
Freely available (preferably open source)
The term "best" is slightly subjective, but I think answers can be rated objectively by the criteria above.
I'd definitively put my vote up for premake. Although it is not as powerful as it's older brothers, it's main advantage is absurd simplicity and ease of use. Makes writing multi-compiler, multi-platform code a breeze, and natively generates Visual Studio solutions, XCode projects, Makefiles, and others, without any additional work needed.
So, judging purely by the criteria set forth in the question, the build system that seems like the best fit is probably waf - pure Python, provides support for C++ and other languages, general, powerful, not a DSL.
However, from my personal experience, I prefer CMake for C++ projects. (I tried CMake, SCons, and waf, and liked them in roughly that order). CMake is a general solution, but it has built-in support for C++ that makes it nicer than a more generic solution when you're actually doing C++.
CMake's build model for C++ is more declarative and less imperative, and thus, to me, easier to use. The CMake language syntax isn't great, but a declarative build with odd syntax beats an imperative build in Python. Of the three, CMake also seems to have the best support for "advanced" things like precompiled headers. Setting up precompiled headers reduced my rebuild time by about 70%.
Other pluses for CMake include decent documentation and a sizable community. Many open source libraries have CMake build files either in-tree or provided by the CMake community. There are major projects that already use CMake (OGRE comes to mind), and other major projects, like Boost and LLVM, are in the process of moving to CMake.
Part of the issue I found when experimenting with build systems is that I was trying to build a NPAPI plugin on OS X, and it turns out that very few build systems are set up to give XCode the exact combination of flags required to do so. CMake, recognizing that XCode is a complex and moving target, provides a hook for manually setting commands in generated XCode projects (and Visual Studio, I think). This is Very Smart as far as I'm concerned.
Whether you're building a library or an application may also determine which build system is best. Boost still uses a jam-based system, in part because it provides the most comprehensive support for managing build types that are more complex than "Debug" and "Release." Most boost libraries have five or six different versions, especially on Windows, anticipating people needing compatible libraries that link against different versions of the CRT.
I didn't have any problems with CMake on Windows, but of course your mileage may vary. There's a decent GUI for setting up build dependencies, though it's clunky to use for rebuilds. Luckily there's also a command-line client. What I've settled on so far is to have a thin wrapper Makefile that invokes CMake from an objdir; CMake then generates Makefiles in the objdir, and the original Makefile uses them to do the build. This ensures that people don't accidentally invoke CMake from the source directory and clutter up their repository. Combined with MinGW, this "CMake sandwich" provides a remarkably consistent cross-platform build experience!
Of course that depends on what your priorities are. If you are looking primarily for ease of use, there are at least two new build systems that hook into the filesystem to automatically track dependencies in a language agnostic fashion.
One is tup:
http://gittup.org/tup/
and the other is fabricate:
http://code.google.com/p/fabricate/
The one that seems to be the best performing, portable, and mature (and the one I have actually used) is tup. The guy who wrote it even maintains a toy linux distro where everything is a git submodule, and everything (including the kernel) is build with tup. From what I've read about the kernel's build system, this is quite an accomplishment.
Also, Tup cleans up old targets and other cruft, and can automatically maintain your .gitignore files. The result is that it becomes trivial to experiment with the layout and names of your targets, and you can confidently jump between git revisions without rebuilding everything. It's written in C.
If you know haskell and are looking for something for very advanced use cases, check out shake:
http://community.haskell.org/~ndm/shake/
Update: I haven't tried it, but this new "buildsome" tool also hooks into the filesystem, and was inspired by tup, so is relevant:
https://github.com/ElastiLotem/buildsome
CMake
CMake is an extensible, open-source
system that manages the build process
in an operating system and in a
compiler-independent manner.
Gradle seems to match all the criteria mentioned above.
It's a build system which took the best of Maven and Ant combined. To me, that's the best.
The Selenium project is moving over to Rake, not because its the best but because it handles multiple languages slightly better than all the other build tools and is cross platform (developed in Ruby).
All build tools have their issues and people learn to live with them. Something that runs on the JVM tends to be really good for building apps so Ant, Maven (i know its hideous), Ivy, Rake
Final Builder is well known in Windows world
smooth build matches most of your requirements.
platform agnostic: yes, it's written in java
language agnostic: it doesn't support c/c++t yet, only java but it is extensible via plugins written in java so adding more compilers support is not a problem
extensible: yes, you can implement smooth function via java plugin, you can also create smooth function via defining it as expression built of other smooth functions.
I would prefer to avoid writing any XML: you won't see a single line of it in smooth build
Freely available: yes, Apache 2 license
disclosure: I'm the author of smooth build.

How do small software patches correct big software?

One thing I've always wondered about is how software patches work. A lot of software seems to just release new versions on their binaries that need to be installed over older versions, but some software (operating systems like Windows in particular) seem to be able to release very small patches that correct bugs or add functionality to existing software.
Most of the time the patches I see can't possibly replace entire applications, or even small files that are used within applications. To me it seems like the actual binary is being modified.
How are these kinds of patches actually implemented? Could anyone point me to any resources that explain how this works, or is it just as simple as replacing small components such as linked libraries in an application?
I'll probably never need to do a deployment in this manner, but I am curious to find out how it works. If I'm correct in my understanding that patches can really modify only portions of binary files, is this possible to do in .NET? If it is I'd like to learn it since that's the framework I'm most familiar with and I'd like to understand how it works.
This is usually implemented using binary diff algorithms -- diff the most recently released version against the new code. If the user's running the most recent version, you only need to apply the diff. Works particularly well against software, because compiled code is usually pretty similar between versions. Of course, if the user's not running the most recent version you'll have to download the whole thing anyway.
There are a couple implementations of generic binary diff algorithms: bsdiff and xdelta are good open-source implementations. I can't find any implementations for .NET, but since the algorithms in question are pretty platform-agnostic it shouldn't be too difficult to port them if you feel like a project.
If you are talking about patching windows applications then what you want to look at are .MSP files. These are similar to an .MSI but just patch and application.
Take a look at Patching and Upgrading in the MSDN documents.
What an .MSP files does is load updated files to an application install. This typically is updated dll's and resource files, but could include any file.
In addition to patching the installed application, the repair files located in C:\WINDOWS\Installer are updated as well. Then if the user selects "Repair" from Add / Remove programs the updated patch files are used as well.
I'm thinking that the binary diff method discussed by John Millikin must be used in other operating systems. Although you could make it work in windows it would be somewhat alien.

Resources