Planck Caching

November 26, 2015

Planck now has support for script compilation caching.

Planck executes scripts by compiling the ClojureScript to JavaScript for execution in JavaScriptCore. This is done dynamically and is usually very fast.

But, if you have scripts that don't change frequently, and they're expensive to compile, it may make sense to save the resulting JavaScript so that subsequent script execution can bypass compilation.

To enable compilation caching in Planck, you simply need to pass the -k option, specifying a directory into which Planck can write cache files. Here's an example: Let's say you have a foo.cljs script file that you run via

planck foo.cljs

First make an empty cache directory.

mkdir cache

Now you can tell Planck to use this directory for caching:

planck -k cache foo.cljs

The first time you run Planck this way, it will save the results of compilation into your cache directory. Then subsequent executions will use the cached results instead.

For some files, this can drastically speed up execution. Take this one, for example:

(defn solve [xs v]
  (for [ndx0 (range 0          
               (- (count xs) 3))
        ndx1 (range (inc ndx0) 
               (- (count xs) 2))
        ndx2 (range (inc ndx1) 
               (- (count xs) 1))
        ndx3 (range (inc ndx2) 
               (count xs))
        :when (= v (+ (xs ndx0) 
                      (xs ndx1) 
                      (xs ndx2) 
                      (xs ndx3)))]
    (list (xs ndx0) 
      (xs ndx1) 
      (xs ndx2) 
      (xs ndx3))))

(prn (solve [3 4 5 6 7 8 9 10] 30))

Interestingly, for the above code, compiling the definition of solve actually takes around 300 ms compared to 10 ms to call it.

Running this with Plank and timing it results a total time of around 590 ms for the first run, followed by 230 ms for subsequent runs. This is great considering that 230 ms is around the time needed to simply do planck -e 1 on this box.

In other words, the compilation time has been effectively eliminated and we are only seeing the time needed to load the standard library and execute the JavaScript.

In addition to caching compiled JavaScript, the associated analysis metadata is cached. This makes it possible for Planck to know the symbols in a namespace, their docstrings, etc., without having to consult the original source. For additional speed, this analysis metadata is written using Transit instead of, say, edn as is currently done today by the ClojureScript compiler.

This caching works for

  • top-level files like the example above (in which case it is assumed that the forms are in the cljs.user namespace, for caching purposes)
  • ClojureScript files in a source directory
  • code obtained from JARs (when passing the -c option to planck)

The caching mechanism works whether your are running planck to execute a script, or if you are invoking require in an interactive REPL session.

Planck uses a (naïve) file timestamp mechanism to know if cache files are stale, and it additionally looks at comments like the following

// Compiled by ClojureScript 1.7.170

in the compiled JavaScript to see if the files are applicable. If a file can’t be used, it is replaced with an updated copy.

I say it is naïve because Planck doesn’t attempt to do sophisticated dependency graph analysis. So, there may be corner cases where you have to manually blow away the contents of your cache directory, especially if the cached code involved macroexpansion and macro definitions have changed, for example.

If you are implementing a bootstrapped REPL and are interested in caching, this capability is directly supported by the cljs.js namespace. It has the needed hooks, providing cached information and allowing your code to return cached JavaScript when namespaces are loaded. All you need to do is arrange to have the cached info persisted to disk.

Tags: Planck ClojureScript Bootstrap