Programming with Comics

Dedicated to Edward Ocampo-Gooding

Jason Brennan ✉️ / May 4 2016

Today’s programming environments are hostile to beginners. Popular languages like Javascript and Swift are based around brittle plain text files, which are hard to read and write, and even harder to understand. Blocks-based languages, like Hopscotch and Scratch, solve some of the issues of brittle plain text, but programs remain opaque to learners—reading a program involves playing computer to understand what’s happening.

This essay presents a prototype programming environment based on reading and understanding programs, without any baggage from plain text languages.

Understanding Programs

This environment borrows from the medium of comics to unify creating and understanding programs, and was inspired in large part by Scott McCloud’s Understanding Comics.

The language works like a simplified version of Logo.** Logo was chosen because it has well understood semantics and has a history of educational usage, not because it’s a perfect programming environment. However, it is extensively documented in Seymour Papert’s must-read book Mindstorms. In Logo programs, you control a small graphical drawing character called the turtle by issuing it commands, like forward 50 to move forward or left 90 to turn 90 degrees. You can also “teach” the turtle new words, which act like functions in other languages, and you can give those function parameters, too. Logo code might look like:

to square :size
repeat 4 [forward :size left 90]
end

In text-based programming languages you can’t see what code does just by looking at it. If you want to know what it outputs, you have to run it, but even then, all you see is the final output. You don’t get to see what happened step by step, and usually you must play computer, imagine in your head what the computer does at each line of code.

Instead of Logo-like lines of plain text code, this environment uses a comic of panels showing each command, where each panel shows the output of the program up until now. To see what the program does at each instruction, you read it left to right, like a comic.

Each panel shows the output, but it also displays the instruction in a human-readable sentence. The sentence tells what the panel does, by how much, and in what direction. Instead of using robotic sounding code, the sentence is as natural sounding as possible (e.g., things are properly pluralized and articles aren’t omitted).

The sentence can be read and fully understood at a glance, but it can also be interacted with, letting the reader change values in the program by dragging on any underlined text. When a panel is interacted with, all panels in the program update immediately so the reader always sees the current program.

Hover to play

The instruction sentence also describes how a panel may be repeated:

Hover to play

When a panel is repeated, the reader can skim through the iterations by panning the mouse cursor left-to-right** A technique popularized by Apple’s iPhoto, progressively playing through the panel to the end, while the sentence provides additional context.

Hover to play

A group of comic panels is called a comic strip. If individual panels are like lines of code in a text-based language, then a strip is like a function. Unlike a function, however, a strip can have a human readable name.

Hover to play

Panels are added to a strip by dragging and dropping them from the Toolbox:

Hover to play

The toolbox shows all panels and strips available to the programmer, with a small (but live updating) icon and a sentence representing each strip in the comic. When a strip is dragged from the Toolbox, a transparent “ghost panel” appears at the end of every strip, which shows what the strip would look like if that panel were included.

A strip can be used as a panel in another strip, which is akin to calling a function in a text-based language. And yes, you can recursively add a strip to itself.** Recursion cuts off at an arbitrary depth.

Finally, strips can have input, which is like a function parameter in text-based languages. Inputs have human-readable names, displayed with a sentence:

Hover to play

An input can then be selected for use in one or more of the strip’s panels:

Hover to play

Inputs are also shown with an example value for use within the strip. In plain text languages, function parameters are always abstract within the function—the parameter is an unknown and the programmer must imagine what value will be filled in at run time. In this environment, inputs have example values so the rest of the strip can work with a concrete value, instead. The actual input value may be different when the strip is used, but the example offers a good starting point.

Hover to play

When a strip with an input is used elsewhere, its example is used as a default value. This default can also be changed.

Hover to play

Towards New Programming Environments

While the environment shown above is incomplete and based on an old programming paradigm, I believe it shows great promise. It presents an alternative to both brittle plain text and blocks-based environments where the programmer is forced to play computer to have any glimpse as to what the computer is doing.** See also Bret Victor’s Learnable Programming and Substroke, and Matthias Graf’s Vogo.

It may not be perfect, but it’s a good first step.