Julia language compiles the script every time, can't we compile binaries with julia instead?
I tried a small helloworld script with println function it took like 2,3 seconds for julia to show the output! It would be better if we can make binaries instead of compiling every time
Update: There have been some changes in Julia, since I asked this question. Though I'm not following the updates for julia anymore, since I've asked this question and if you're looking for something similar, look into the below answers and comments by people who are following julia.
Also, its good to know that now it takes around 150ms to load a script.
Keno's answer is spot on, but maybe I can give a little more detail on what's going on and what we're planning to do about it.
Currently there is only an LLVM JIT mode:
There's a very trivial interpreter for some simple top-level statements.
All other code is jitted into machine code before execution. The code is aggressively specialized using the run-time types of the values that the code is being applied to, propagated through the program using dynamic type inference.
This is how Julia gets good performance even when code is written without type annotations: if you call f(1) you get code specialized for Int64 — the type of 1 on 64-bit systems; if you call f(1.0) you get a newly jitted version that is specialized for Float64 — the type of 1.0 on all systems. Since each compiled version of the function knows what types it will be getting, it can run at C-like speed. You can sabotage this by writing and using "type-unstable" functions whose return type depends on run-time data, rather than just types, but we've taken great care not to do that in designing the core language and standard library.
Most of Julia is written in itself, then parsed, type-inferred and jitted, so bootstrapping the entire system from scratch takes some 15-20 seconds. To make it faster, we have a staged system where we parse, type-infer, and then cache a serialized version of the type-inferred AST in the file sys.ji. This file is then loaded and used to run the system when you run julia. No LLVM code or machine code is cached in sys.ji, however, so all the LLVM jitting still needs to be done every time julia starts up, which therefore takes about 2 seconds.
This 2-second startup delay is quite annoying and we have a plan for fixing it. The basic plan is to be able to compile whole Julia programs to binaries: either executables that can be run or .so/.dylib shared libraries that can be called from other programs as though they were simply shared C libraries. The startup time for a binary will be like any other C program, so the 2-second startup delay will vanish.
Addendum 1: Since November 2013, the development version of Julia no longer has a 2-second startup delay since it precompiles the standard library as binary code. The startup time is still 10x slower than Python and Ruby, so there's room for improvement, but it's pretty fast. The next step will be to allow precompilation of packages and scripts so that those can startup just as fast as Julia itself already does.
Addendum 2: Since June 2015, the development version of Julia precompiles many packages automatically, allowing them to load quickly. The next step is static compilation of entire Julia programs.
At the moment Julia JIT compiles its entire standard library on startup. We are aware of the situation and are currently working on caching the LLVM JIT output to remedy the situation, but until then, there's no way around it (except for using the REPL).
Related
I am new to Common Lisp. This is how I develop programs in other languages, and also how I now develop programs in Common Lisp:
Open a text editor (e.g. vim or emacs) to create/edit a text file.
Write source code into the text file. (If unsure about the behavior of a snippet of code, and an REPL is available, then evaluate the snippet in the REPL, verify that the snippet evaluates as expected, and then go back to writing more code.)
Save the text file.
Ask the compiler/interpreter to load and run the source code in the text file. (e.g. sbcl --script myprog.lisp)
Go to step 1 if needed.
This is the conventional write-compile-run development cycle for most programming languages. However, in the lisp world, I hear things like "interactive development" and "image-based development", and I feel that I am missing out on an important feature of Common Lisp. How do I do "image-based development" instead of "write-compile-run development"?
Can someone provide a step-by-step example of "image-based development" similar to how I described "write-compile-run development" above?
(Note: I am using SBCL)
In typical Common Lisp implementations the runtime, the compiler, parts of the development environment and the program you are developing reside in the same program and share the same object space. The compiler is always available while you develop the program and the program can be incrementally developed. The development tools have access to all objects and can inspect their state. One can also undefine/remove, replace, enhance functionality from the running program.
Thus:
don't restart the program you are developing. Stay connected and update it. Even days, weeks, or months - if possible.
write code in such a way that the program can be replicated and built from scratch if necessary. Build it from time to time and fix any build problems.
once you use our program and there is an error -> fix the error within the program, while being able to inspect the full error state
creating a running program is either loading all code into a plain Lisp all the time or saving an executable image with the loaded code/data
Fixes to program bugs can also shipped to the user as compiled Lisp files, which gets loaded into the delivered program and update the code then.
Let's say that you are using SBCL with Emacs and SLIME (e. g. through Portacle).
Open Emacs
Start SLIME (M-x slime) — this starts a “plain” Lisp process in the background and connects the editor functions provided by slime to it; then gives you a REPL that is also connected into this process (image)
Open a text file (e. g. foo.lisp)
Type some code
Press C-c C-k to compile the file and load it into the running Lisp process
Switch to the REPL, try it out
Switch to the Lisp file (step 4).
This is just very basic usage. Further things to do/learn
You can also compile and load just a single toplevel form (C-c C-c)
Learn about packages
Learn about systems (ASDF)
Learn how to use Quicklisp to get the libraries you want
Learn how to access inline documentation from the REPL
Note that you never need to unload your program, you just modify it, even when downloading and loading new libraries. This makes the feedback cycle instantaneous in most cases. You also never need to switch away from the IDE (Emacs).
I have a large GPR based project that can take over 30 minutes to compile.
Having analyzed the build process I noticed many obvious inefficiencies (multiple calls to gprbuild rather than aggregates, excessive use of alternative files rather than configurations, etc). I am wondering if there is some means to 'profile' the build process to see what takes so long.
In particular it takes about 5 minutes to recompile when even a single file changes and there is an error in it. In theory it should be pretty quick to realize that that file has to be recompiled (its the only one that does) and start the compilation process, rapidly discovering the error.
From the verbose output it looks like it takes quite a while just parsing the massive web of gpr files used to define the build, but I would like to know where it spends most of its time.
Thus my question is: Is it possible to profile a build done by gprbuild? If so, how?
From low to high complexity:
Ask gprbuild to report more details about what it is doing with the flag -vh.
Run gprbuild through strace.
Rebuild gprbuild with the required flags to profile it using gprof (but be aware that gprof doesn't always tell the truth).
This is mostly a stupid question, since UPX (a tool that wrings extra bytes out of your executable files) saves a tiny amount of space over the built in compression in the buildapp tool.
A very small demo application creates a 42 megabyte file. Understandable, since the SBCL environment isn't tiny.
Passing the --compress-core option to buildapp shrinks that down to 9.2MB.
I thought I'd try throwing UPX at the resulting binary, and the savings only amounts to a few more bytes: 9994288 -> 9871360
However, the resulting file no longer runs anymore - it just jumps into the SBCL REPL (with no errors, it's as if I just ran sbcl by hand), and some poking around there reveals that the functions making up my test program no longer exist.
What did UPX do to the binary that resulted in this breakage?
This may not be the answer, but it may serve as a clue: I've found that if you add anything, even a single byte, to the end of an SBCL executable created with sb-ext:save-lisp-and-die, then all the definitions disappear, just as you described.
Perhaps SBCL creates executables by appending the core (which contains your definitions) to a copy of the SBCL ELF (or PE on Windows) binary, plus some metadata at the end so that SBCL can still find the beginning of the core even though it's appended to an executable.
If you hex-edit an executable created with save-lisp-and-die, you'll find that it ends with the string "LCBS" (SBCL backwards), which seems to support my theory. "LCBS" probably serves as a magic number, letting SBCL know that yes, this executable contains its own core.
UPX compresses executables, probably including that magic number at the end. When SBCL opens its UPX-compressed self on disk, it won't find "LCBS" at the end, so it assumes that there is no core appended to the executable.
I can't explain why the standard library seems to still be there if this is the case. It could be that SBCL loads /usr/lib/sbcl/sbcl.core (or its equivalent on Windows) in that case. This could be tested by moving the executable to a machine where SBCL is not installed and seeing if it works at all, and if so, whether you still have car, cdr, list, etc.
I use a third-party DLL (FTD2xx) to communicate with an external device. Using Qt4, in debug mode everything works fine, but the release crashes silently after successfully completing a called function. It seems to crash at return, but if I write something to the console (with qDebug) at the end of the function, sometimes it does not crash there, but a few, or few dozen lines later.
I suspect a not properly cleaned stack, what the debug build can survive, but the release chokes on it. Did someone encounter a similar problem? The DLL itself cannot be changed, as the source is not available.
It seems the reduction of the optimization level was the only way around. The DLL itself might have problems, as a program which does nothing but calls a single function from that DLL crashes the same way if optimization is turned on.
Fortunately, the size and speed lost by the change in optimization level is negligible.
Edit: for anyone with similar problems on Qt 5.0 or higher: If you change the optimization level (for example, to QMAKE_CXXFLAGS_RELEASE = -O0), it's usually not enough to just rebuild the application. A full "clean all" is required.
Be warned - the EPANET library is not thread safe, it contains a lot of global variables.
Are you calling two methods of that library from different threads?
I have developed a big library of functions in R.
For the moment I just load ("source") the functions at the beginning of all my scripts.
I have seen that I can create packages.
My question is: Will that improve the execution time of my functions? (by transforming interpreter code into machine language?)
What does the package creation does? Does it creates binaries?
Thanks
fred
There isn't an R compiler yet Packaging your R code won't improve its execution time massively. It also won't create binaries for you - you need to build those from the package tarball (or get CRAN or similar to build them for you). There is now a byte compiler for R and R's packages are now by default byte compiled. Speed improvements are in general modest - don't expect C-like speed.
Packaging R code just does exactly that; it packages the R code, code to be compiled (C Fortran etc), man pages, documentation, tests etc into a standard format that can be distributed to users and installed/built on multiple architectures.
Packages can take advantage of things like lazy loading such that R objects (your functions say) are only loaded when needed, whereas source loads them all into the global environment (by default).
If you don't intend to distribute your code then there are few benefits of packaging just for your own use, but if you do package and write documentation and examples/tests, you might be alerted to changes in the package code that break examples or cause tests to fail. That way you are better informed as to the reliability of your code, even if it is only you using it!