Testing with Planck

February 27, 2016

The cljs.test namespace has been ported for use in Planck. In short, this means you can require this namespace and make use of it, just like you can do in regular JVM ClojureScript.




What's the big deal? Why did it need to be ported?

A lot of cljs.test is macros, like deftest and is, and one key aspect is that these macros call other macros in the cljs.test namespace. While this is cool in Clojure, it is verboten in ClojureScript. This alone means one does not simply (require 'cljs.test) in a bootstrap REPL. This can be solved, though, by constructing a macro tower.

Another problem is that some of the cljs.test macros call upon Clojure code in the compiler. This is fortunately easily solved by porting this Clojure code over to ClojureScript and including it internally within Planck.

Also with the port, the cljs.test/assert-expr mechanism was made directly accessible to ClojureScript runtime code. This is covered here.

All of this is built into the Planck 1.10 binary. There is nothing special you need to do other than to require the cljs.test namespace and optionally :refer and :refer-macros as you see fit.

Fallout

Of course, the primary result of this is you can now write unit tests for any code you write for use with Planck. Things just work.

Another nice outcome is that Planck's own unit tests can now be written using cljs.test. When Planck was first being developed, this wasn't possible. In fact, the first tests for Planck took on a rudimentary form: A shell script that ran forms through the Planck binary, recording standard out and err, and comparing the results against expected output using diff. Painful, but workable. Then, later, it became possible to write unit tests using cljs.test, but because of the macros issue, these tests had to be AOT compiled using JVM ClojureScript and baked into the Planck binary. This meant that to fix a test, you'd have to rebuild everything. Ugh. Now, unit tests for Planck can just be simply required and executed by Planck. They can be changed and dynamically reloaded and re-run. Nice.

And yet another consequence of having ported cljs.test to Planck is that now, for the first time:

We can compile and run the ClojureScript compiler's cljs.core namespace unit tests in bootstrapped mode.

Read that again if it didn't sink in. This is absolutely huge. It means we can be much more certain that bootstrapped ClojureScript behaves like JVM ClojureScript.

While both variants of ClojureScript are based on the same codebase, there is a decent amount of conditional code catering to the differences between Java and JavaScript. Quite a few defects related to this have been discovered and fixed over the past half year or so, driving the bootstrapped ClojureScript known bug count to zero. But, new corner cases are still being discovered, as you might expect.

Here is what running the test suite in Planck looks like, elided for brevity:

Testing cljs.core-test

Testing cljs.reader-test

Testing clojure.string-test

...

Testing cljs.import-test

Ran 184 tests containing 17219 assertions.
0 failures, 0 errors.

Without this capability, the ClojureScript compiler does have a few unit tests specifically for the cljs.js namespace, and it is used to exercise some things at a high level within Node in bootstrap mode. But it is nowhere near as comprehensive as being able to compile and run the thousands of assertions that have been written over the years for JVM ClojureScript. Having this capability also means that, as new fixes or features are made in the ClojureScript compiler codebase, Planck can help automatically catch any bootstrap regressions that might creep in—Planck runs all of the tests above in CircleCI whenever changes are committed to Planck.

I'm excited about all of this as it helps ensure that bootstrapped ClojureScript can be relied upon and it also helps you easily write tests for any scripts you might write for use with Planck.

Tags: Planck ClojureScript Bootstrap