Live Programming a Live Programming Environment: An Experience Report
Abstract & Introduction
What is it like to build a live programming environment from within another live programming environment? Programmers use Clojure and Clerk1, a moldable live programming environment that works in complement with the Clojure REPL, to build reports. We have been exploring the process of live programming a UI-based report builder tool within Clerk.
In the paper and videos below, we share our idealized live programming experience of building a report, then abstracting that report into a report builder UI. We include various expandable asides on the problems we faced in understanding what we were working towards, and what this live programming experience should look and feel like.
Goal and Detours
Over the course of this project, our goal has shifted from the specifics of a report builder UI — which we found to have been well explored elsewhere2 — to exploring the boundaries of live programming in Clojure with Clerk.
In Clojure and Clerk, we have culture and tools around maintaining a live programming workflow; but while building the report builder UI, we encountered situations where we dropped out of our live workflow in order to make progress.
After building a report builder UI once, we looked into the issues that caused these losses of liveness. We set out to address all of these issues. If an issue was caused by a limitation of Clerk, we improved Clerk. After addressing these issues, we found a way to build the report builder, from beginning to end, while staying in a live programming workflow.
We extensively rewrote this paper to emphasize this idealized live programming workflow. However, we believe it’s important to include the many detours and moments of confusion we had while creating this experience report, as they are part of our journey too. You’ll find them included as expandable asides throughout the paper, offering additional context and insight into the issues we faced.
Remember: the point is to support programming interactively. You don’t want to have to kill your program and rebuild it from scratch just because you changed a definition. That’s silly; adding and changing definitions is most of what you do! If your development environment is going to support interactive development, then it had better know how to keep your program running when you change some definitions. [emphasis added] 3
— Mikel Evins
Define Live Programming
There is no agreed-upon definition of live programming, and ongoing debates exist about whether live programming and live coding are the same or how they differ. For the purposes of this experience report, each of the main authors will contribute their own definition of live programming, and we will try to incorporate each perspective as we work through building the report builder.
But what is “livecoding” exactly? “Live coders are basically performing by writing computer programs live on stage, while the programs are generating their art – whether that’s visuals or music,’ McLean says. “Their computer screens are projected, so that the audience can see the code being manipulated. But the focus is on the music, on people dancing and seriously enjoying themselves”4
— Alex McLean
Philippa’s definition of live programming: For me, live programming is about entering a flow state in the psychological sense, as described by Csikszentmihalyi5. The use of short feedback loops and structured iteration with the REPL makes code manipulation feel immediate and direct. I become deeply focused, and it feels as though action and awareness seamlessly merge into one.
Martin’s definition of live programming: For me, it’s a feeling of immediate connection between the code I write and its effects. Building complex expressions feels effortless because I’m making tiny, continuous changes with instant feedback. Mistakes are easy to catch — often without needing to read error messages — since I know exactly what I just changed and where to look for mistakes.
Elliot’s definition of live programming: I see Philippa and Martin practice live programming as a part of their day-to-day. Live programming can be a culture or a practice. For me though, live programming is a label I put on practices, tools, and ideas I come across. I reference things under this label to try to understand how people can interact with computation.
Bottom-up Design
That LISP users tend to prefer structured growth rather than stepwise refinement6 is not an effect of the programming system, since both methods are supported. I believe, however, that it is a natural consequence of the interactive development method, since programs in early stages of growth can be executed and programs in early stages of refinement cannot.
– Erik Sandewall7
The live programming workflow we present below can perhaps be characterized as structured growth, or as we will call it, bottom-up design. We define bottom-up design as a programming method where a small initial program is gradually expanded. Functionality is tested and enhanced incrementally, adding new features and increasing the complexity of existing ones, all while maintaining a continuously running and inspectable system. If an error occurs, the programmer knows that it is a result of their most recent small change.
Clear examples of applying bottom-up design to build up functions can be found in The first tiny report and Displaying table-shaped results as tables.
[...] a method of program development which may be characterized as structured growth: an initial program with a pure and simple structure is written, tested, and then allowed to grow by increasing the ambition of its modules. The process continues recursively as each module is rewritten. The principle applies not only to input/output routines, but also to the flexibility of the data handled by the program, the sophistication of deduction, the number and versatilty of the services provided by the system, etc. The growth can occur both “horizontally,” through the addition of more facilities, and “vertically” through a deepening of existing facilities and making them more powerful in some sense.
– Erik Sandewall
Real World Reports
We (the main authors from Nextjournal) have been successfully using Clojure and Clerk to build reports for the automotive logistics industry. Because it has worked well for us (and other users of Clerk), we know from experience that Clojure and Clerk are well-suited for creating actual reports. Now let's explore how the efficacy of building reports in Clojure and Clerk might be extended to a report builder UI.
Clojure & Clerk
Good support for live programming 8 was part of Clojure's motiviation. Clojure’s choice of defaults – particularly its focus on (mostly) functional programming with immutable data structures – makes it especially well-suited to live programming, as it prevents many of the state bugs common in imperative systems that support interactive code evaluation.
It is common to conflate any interactive language prompt with a REPL, but I think it is an important aspect of Lisp REPLs that they are a composition of read-eval-print. From a language perspective, one aspect of supporting REPL-driven development is that there are no language semantics in Clojure associated with files or modules. While it is possible to compile and load files, the effect of such loading is always as if executing each contained expression sequentially.
– Rich Hickey in A history of Clojure.
Clerk is a moldable9 development environment that complements the Clojure REPL10. It provides incremental computation over a Clojure source file and comes with an extensible viewers library including built-ins for Clojure data structures, tables and visualizations using e.g. Vega or Plotly. Clerk's sidecar display runs in the browser. (This writeup was also created using Clerk.)
Building a Report Builder
Our bottom-up design process goes like this:
- Live program concrete reports as targets
- Extract a DSL that can generate the insights in those target reports
- Build an interactive report builder that is able to generate those target reports
We know from experience that Clojure & Clerk are well suited for the first two steps and will provide us with a live programming experience throughout.
We chose this process because we believe that a good way to design an interactive programming system is to model it via a textual DSL. The DSL will be the foundation for our report builder. Design mistakes like inconsistencies or confusing semantics are easier to spot and cheaper to fix in a textual DSL than a UI. Clerk's support for literate programming supports this discovery process well.
The third step is the most complex of the three, and we've only explored it in tiny experiments until now. We plan to build it inside Clerk: viewers will show a visual representation of the report's code and interacting with it will change that code and trigger a recomputation.
Our plan is to maintain a live programming experience throughout (hence the title) and be mindful about when we're dropping out of it.
The First Tiny Report
We start by building a very simple report using bottom-up design and the Clojure REPL.
In the video below we write code to load the dispatchable vehicles dataset from our database11 and immediately evaluate the code to see the result in our editor. This is a small demonstration of standard REPL-driven development in Clojure.
load-builder namespace to query this data.Now that we have our data, let’s create a report document.
In the next video we import Clerk, add a comment with Markdown for a document header, and press a hotkey to show the Clerk document based on our Clojure file in the browser.
Then we
- use a viewer to display our data as a table,
- write a pipeline
->>12 of functions to process the data into a format that we can plot, - plot the formatted data in a bar graph,
- and re-structure our code so that the plotting operations also happens in the function pipeline
->>.
->> is an idiomatic choice to represent a sequence of transformations. With the threading construct we can incrementally add steps to transform our dataset and evaluate our code to see the results of transform as we build it.Voilà, that’s how we build a first simple report using live programming with Clerk. Now we have a concrete target for what we want our report builder to be able to produce.
Report DSL
To figure out what operations we want to support in the report builder, let’s consider the function sequence we just wrote in the first tiny report:
For this simple report we only need three functions: map, frequencies and plot. We will take these three functions, composed into a pipeline using ->>, to be our report builder Domain Specific Language (DSL).
Our report builder will start with a ->> expression that includes an initial element, such as a dataset or, later on, a query.
We want users to interactively add steps to the pipeline, such as (map :make/name), frequencies, and plot, to generate our initial report. The user will choose steps from a list of valid choices; a valid piece of clojure code at a step in a pipeline is called a choice.
We also want users to see the intermediate results at each step in pipeline, interleaved with the code.
Data Structure
Now that we've gathered an idea of what we want the user to be able to edit, let's set aside our first tiny report and plan how we will live program a report builder.
Our goal is to develop a report builder UI that a user can use to create the first tiny report and variations on it. The builder should display intermediate results and allow users to easily identify and select the next steps. Here's how we went about live programming the report builder in the following steps:
First, in UI Using Viewers: we create a data structure that represents the state of the report builder, including choices at each step in the pipeline and results at each step in the pipeline. Additionally, we build a Clerk viewer to visualize this data structure, which will later function as an editor.
Next, in Computing Choices: we discuss the choices function for computing a list of choices at each step in the pipeline.
Next, in Tracing Macro: we implement a tracing macro that generates this data structure.
Finally, in Saving and Evaluating: we develop a function that allows the editor UI to save user edits back to the source code and trigger Clerk to re-evaluate the file.
UI Using Viewers
We begin by designing a data structure to represent the state of our report builder. We require that the structure represent each step in a pipeline along with its corresponding editing UI. Since the first step in a report is the dataset, which cannot be edited, we decided to represent it solely by its Clojure form.
Starting with a concrete data structure helped give us the flexibility to manually and iteratively refine the shape of the data without depending on the complexity of a macro-based program generator (yet).
Since all LISP code is tree-shaped, and our report programs are Clojure forms, we will explore a tree-structured data model and UI for building reports.
Our tree structure will combine a code form, the results for each step, and applicable options (suggesting possible next steps based on the output).
Here’s a video demonstrating how we can grow an interactive UI from this sample data structure using Clerk viewers:
Although we're making a Clojure code editor, we start with live-coding based on a sample data structure that maps closely to the UI.
- We incrementally change what could be the function body inside the let binding, evaluating continuously after every tiny change.
- Once our function body does what we want, we pull out the example arguments and paste them after the let binding.
- We then refactor the let binding into a function definition and name the function as late as possible.
- Finally, we use the pulled out example data and use it to add an example function invocation in a rich comment.
What happened here?
We built and applied a custom Clerk viewer to our sample data structure. By default, our sample structure is displayed in the notebook using Clerk’s default data viewer. However, since we provide a custom viewer with a predicate that matches any item containing a :form key, we can override the default behavior.
Viewers are applied recursively: the custom viewer will be applied to any and all nested data that match the predicate.
Iterating on the viewer’s design
This viewer is still in its early stages — we definitely want to improve its appearance and refine the visual design! Typically, this means the HTML structure will become more complex, and we’ll need to refactor it into smaller functions to avoid having one large render function within the Clojure namespace.
- You can configure a viewer to use any ClojureScript function from an external ClojureScript namespace by setting
:require-cljsand referencing the viewer’srender-fnto a quoted symbol representing a function in that namespace. - You can connect your editor to a browser REPL (in addition to your Clojure REPL), allowing you to leverage structured growth and REPL-driven development for any render code in external CLJS files.
- Any render errors are captured and displayed, ensuring that what you see in the browser is never broken to the point where you lose REPL-driven development (eliminating the need for fallback
console.logdebugging)
Here’s what our first basic design iteration looks like using these features:
Refactoring into composable parts
This already looks better, but we want to make it more modular and easier to document. To achieve this, we can further break down the render code. For example, we can create separate functions: one responsible for rendering each step, another for handling both the step and its result, and a form editor that allows users to select options.
Clerk examples allow you to show function calls beside sample inputs and outputs. Additionally, we can leverage Clerk’s viewer metadata to apply custom viewers to all outputs generated within an example block.
We use Clerk's example function to document how each of these components looks with sample data and how they fit together to form the report builder.
Displaying table-shaped results as tables
We aim to display a report’s intermediate results in a rich, intuitive format. For example, if the result of an expression is a list of hashmaps, it would be ideal to present it as a table. Clerk includes a default table viewer capable of rendering various data shapes as tables.
Let's add a custom viewer that automatically applies the table viewer to any :result that matches a table-like shape based on a predicate.
Once again, we utilize the REPL, Clerk viewers, Clerk examples to live-program this process:
Let's recap what just happened in the video: our goal is to write a new function but we start with a let binding that binds example arguments instead.
- We incrementally change what could be the function body inside the let binding, evaluating continuously after every tiny change.
- Once our function body does what we want, we pull out the example arguments and paste them after the let binding.
- We then refactor the let binding into a function definition and name the function as late as possible.
- Finally, we use the pulled out example data and use it to add an example function invocation in a rich comment.
Computing Choices
In order to compute the choices for a next step in the pipeline, we associate each function in our dsl with another applicable-choices function. We stored the associated function in variable metadata. Our choices function takes a result and returns a list of applicable choices for the next step in the pipeline.13.
Tracing Macro
With a data structure and viewer in place to display an initial read-only report builder UI, our next step is to write a trace macro to generate it. This macro will operate on ->> expressions, capturing each form in the pipeline, storing intermediate results, and presenting potential choices for each step.
Next, we verify that the viewers function correctly with the generated data structure.
Saving and Evaluating
With the edit UI in place, the next step is to persist the changes. Clerk’s viewer API provides each :render-fn with an argument containing context data. Using this data, we created a function that writes edits back to the source code and re-renders the notebook. Modifying our notebook's source code to take advantage of Clerk’s incremental computation and update capabilities.
Completing the First Cycle
With the viewer's event handler in place, we implemented a set of functions that can re-write the Clojure source and finally trigger clerk/show! on the current source file — the same mechanism that is usually bound to an editor hotkey to re-compute what's changed in the source file and update what's visible in the browser.
What you see in the above video is a reporting pipeline that is apart from the initial dataset. You see a preview of the dataset and a dropdown menu that lets you select from the next steps based on the dataset. When you select a next step, the step function is written back the source file, the file is re-computed and the browser shows the step and the result along with the choices for the next step after it. Finally, the steps are built up to terminate in a bar plot.
Here is one more video showing the report builder after another iteration on the viewers' visual design:
Growing it from here
Nesting
In our first cycle, adding to the pipeline only worked on the top-level. A nested Clojure expression like (map :make/name) is represented as a single dropdown.
Alternatively to the single dropdown we could represent this using nesting instead. This could give users more compositional power.
Form-Specfic UI
The usage of nesting exposes other interesting possiblilties. For example, we can introduce a viewer for the Clojure set data literal and show that as a multi-select. This could be used to parametrize a select-columns function, for example.
Directly manipulating data to create transforms
We discussed ways that a user could add transforms to the data pipeline by selecting them explicitly, or by directly manipulating data. We discussed how different representations of data would require different interactions to do similar things, but we could design protocols for viewers that allow them to map user interactions to our set of transformation primitives. We’re inspired by the data direct manipulation interfaces of projects like Engraft’s extractor component and Persist14.
Related Work
If I have seen further it is by standing on the shoulders of Giants.
— Isaac Newton (who this quote is almost exclusively attributed to), written in 1675 and published in 1855
This work drew much inspiration from the following projects:
- The Gamma: Programmatic Data Exploration for Non-programmers by Tomas Petricek
- Glamorous Toolkit, see Towards Moldable Development Tools by Andrei Chiş, Oscar Nierstrasz and Tudor Gîrba
- Ampleforth: A Live Literate Editor by Gilad Bracha
- maria.cloud, a web-hosted interactive Clojure learning tool created by Matt Huebert, Dave Liepmann, and Jack Rusher.
- When I Sit Down At My Editor, I Feel Relaxed by Matt Huebert, presented at PX/16
- Histogram: You have to know the past to understand the present by Tomas Petricek
- The Common LISP presentation system which originated on the LISP machine and lives on in a reduced form as a feature of the emacs package Slime
- Augmenting Human Intellect: A Conceptual Framework by Douglas Engelbart
- Enhancing Liveness with Exemplars in the Newspeak IDE by Gilad Bracha.
- Babylonian-style Programming: Design and Implementation of an Integration of Live Examples into General-purpose Source Code by David Rauch, Patrick Rein, Stefan Ramson, Jens Lincke, Robert Hirschfeld.
- Filling Typed Holes with Live GUIs by Cyrus Omar, David Moon, Andrew Blinn, Ian Voysey, Nick Collins, and Ravi Chugh.
- Adding interactive visual syntax to textual code by Leif Andersen, Michael Ballantyne and Matthias Felleisen
- Nextjournal, the product
- Org mode for Emacs
- the Streamlit Python library which eshews a custom format and enables building a web UI on regular python scripts
Conclusion
Philippa’s conclusion: Going through the process of implementing a first version of the report builder using Clojure and Clerk, I’ve come to realize that live programming is as much about practice — the techniques one discovers and the levels of detail one addresses — as it is about the tools themselves.
There is a rich history of live programming with various LISPs, and this practice continues daily across many fields, including academia, industry, and the arts. Yet, much of the focus in live programming research today seems to center on other languages and primarily on the tools themselves. This makes me curious: what other techniques exist in the folk culture15 of live programming? How do people live program — not just with what tools?
Martin’s conclusion: Could we live-program our report builder? I think, for the most part, yes. I was delighted by how well the approach of starting from a concrete data structure and building viewers around it worked. This was a pleasantly live experience throughout, and the other pieces naturally fell into place. However, there’s inherent complexity in Clerk when dealing with the two runtimes (JVM and browser), which is hard to overcome. I always dread working across these boundaries. I believe the live experience would be significantly better without this issue, and I envy Glamorous Toolkit for avoiding it.
On the other hand, we gain a lot of reach by being able to work with both runtimes. It’s what enables using Clerk in anger. My main takeaways for improving Clerk — areas that often disrupt the live flow — are that the viewer API is not yet expressive enough and it’s difficult to see its inner workings. We’ve discussed fixing this by drawing more inspiration from presentation systems in older LISP environments. Additionally, the error messages from Clerk’s various layers need significant improvement.