Recreating John Maeda's "Design By Numbers" programming environment in JavaScript
Recreating John Maeda's "Design By Numbers" programming environment in JavaScript
This is so wonderful! I have both of my Design by Numbers books always near. —Casey Reas
OMG it's crazy to see things I made a quarter-century ago, and haven't seen since then. —Golan Levin
OMG —John Maeda
Being a student of media art for some years, I would, from time to time, hear the legendary "Design by Numbers" mentioned, and how it was before Processing and OpenFrameworks, and how DBN started it all—but somehow it never occurred to me to actually try it, until recently.
As a "prologue" to commence the first year of my PhD program, I'm taking on the ambitious task to try out each and every creative coding framework that exists or existed; to learn from their designs, and see where they succeeded and failed. And naturally, the first I wanted to explore was Design by Numbers—created by my graduate advisor's graduate advisor's graduate advisor, former Media Lab professor John Maeda.
So I downloaded a copy of DBN 3.0.1 from https://dbn.media.mit.edu/info/dbn-on-web/dbn.jar (which could to be run with jar→zip unarchive trick, though some contents got corrupted in the process), and much later, a more updated version from John Maeda's blog, which included a whole suite of example programs.
I'm always of the opinion that the best way to learn something is to make it myself instead. (This has dragged me into a good deal of trouble.) Therefore, as I played around with DBN, an evil idea started brewing in my mind — Since the syntax seemed quite straightforward, I thought it would be a fun weekend project to re-implement DBN for the web. It would also be fun to see examples in the DBN book running, and helpful for me to hone my interpreter-writing skills. As I was scheming and plotting, a message from Zach popped up: "a web based version of DBN would be really cool as a kind of side project—(not sure how hard that is, just thinking out loud...)".
Now that he was also onboard with the idea, Zach shared a "vocabulary document" he wrote in 2000 as a student—which is exactly the kind of thing one would need to implement a language from scratch. Instead of taking the easy route of writing a to-JavaScript transpiler as I've often done out of laziness, I decided to do it properly this time—by making a bytecode interpreter!
A transpiler converts programs written in one language into programs written in another language. It is like Google Translate for code. For example, for this DBN program `repeat i 0 10 {line i 0 100 100}`, a transpiler implementation would convert it to `for (let i = 0; i <= 10; i++) line(i,0,100,100);` as you can see, it is quite straightforward when the target language is more sophisticated than the source language, which is the case with DBN and JS. Another advantage is that we would be able to piggyback on the optimizations in the target language, such as V8's JIT. However, it can be harder to control what exactly happens after the translation is done, when the target language implementation takes over. One would also need to trace any run time errors back to original source, and features like stepping and inspecting are harder to inject.
A bytecode interpreter in comparison, first translates the source program into some sort of intermediate representation (IR). Since these IR often consist of bytes, they're called bytecode (Python bytecode, Java bytecode, etc.), but they're basically lower-level instructions. Then, one would implement a virtual machine (VM) to execute these instructions. A virtual machine (not to be confused with a virtual machine) is like a real machine, but you get to invent your own instruction set! For example the same program `repeat i 0 10 {line i 0 100 100}`, one might compile it to
Of course this is much simplified and doesn't account for the case of looping backwards and looping zero times—but notice that we get to define a variable, and have an instruction called "line"—things are probably not possible in real architectures but we can do whatever we like since it's our own virtual machine. Additionally, it is quite easy to make the machine run one step at a time—perfect for visualizing the process of execution (which I imagine could be quite useful for pedagogical tools such as DBN!)
There're of course other approaches, such as tree-walking interpreters (as you'll see later, our interpreter actually "cheats" by doing a little bit of tree-walking), or a straight-up compiler—which should be relatively manageable since the only type in DBN is a 32-bit integer.
To implement the compiler and the VM, I first carefully studied the vocabulary document. The syntax of DBN is quite fun. At first glance it resembles shell script, and uses spaces as primary delimiter—thus inserting math expressions requires parentheses (this is what I call "no free lunch" in syntax design).
Our parser first "tokenizes" the source code into a bunch of "words". Then it builds structure from the stream of words, into what they call an "abstract syntax tree". For example, the example `repeat i 0 10 {line i 0 100 100}` first gets turned into:
then into:
In addition to what you see above, each token is associated with its line number, for better error messages in case users write some buggy code (which we would expect the target audience of DBN to write a lot).
From this point, we can trivially write a JavaScript code generator (if we were going for a transpiler), but for our VM, we'd like things a bit flatter. This way we could avoid recursion or context-saving, and have the VM run exactly one step at a time.
For DBN, this boils down to the following tasks:
For numeric operations, I decided to make them atomic without the un-nesting. Thus they're not separate instructions, but recursively computed the same time their enclosing line is run (also known as tree-walking). Perhaps not a purist's approach, but saves me some trouble, and I doubt that users would need such fine-grained stepping into sub-expressions (and at the price of performance) anyways. In other words, there're no separate ADD, MUL, DIV etc. instructions.
For the VM, I designed a small set of instructions:
After I ran some small tests in the commandline, I confirmed that my parser and VM are indeed working. Time to make a web interface! You know, boring HTML/CSS stuff. I hacked together a somewhat convincing counterfeit of the original GUI with the help of a digital color picker. Since the button sprites were missing, I had to write some SVG to make my own, based on screenshots I found on the internet.
I wrote my own code editor from a `textarea` element instead of using popular libraries such as CodeMirror, which I find a bit bloated for my purpose. The hack is to overlay an uninteractable, styled div on top of the interactable, unstyled textarea, thus achieving the illusion of syntax highlighting. I've used this technique previously; It requires adjusting offsets for different browsers, and can be a bit slow when there's a massive amount of code in it.
I was quite happy with my little DBN implementation, and was looking for more example programs to test it on. Google search didn't yield much. I've been suspecting that the Future Sketches group already have a copy of the Design by Numbers book, and sure enough, it was right in the middle of our big bookshelf. What a beautiful work of typesetting! But who is going to type all of these printed examples into my computer? For a fleeting moment, UROPs no longer sounded like such a bad idea.
Then suddenly, an email hit my inbox, in it, there's nothing but a zip archive, called "dbn-examples.zip", which unzips into, a folder of, DBN examples. Thanks Golan!
Looking at the signatures on the examples, it seems that they've all become big names, but to think, when those programs were first written, perhaps they were still graduate students like me, trying to figure out how this "DBN" thing works!
What I also admired, is that many examples come with a little "poem" in the first block of comment. For example the one for "merging.dbn" reads:
My appreciation of historical artifacts was cut short, by a flood of error messages in my terminal. My interpreter crashed for every single example! None of them worked! What?
After a bit of head scratching, I figured out that the problems were partly due to many "secret" features that are not documented in the "vocabulary document" (perhaps they're later additions predated by the documentation), as well as some of my inaccurate assumptions about how DBN should work.
Another problem which tripped me up was how DBN was doing screen refresh. In DBN, you can explicitly request a redraw using the `refresh` command. But apparently many examples don't have it, while refreshing just fine in the original Java applet. Some mysterious magic is in work!
First I guessed that every time, right before the user sets the background color with the `paper` command, it would refresh. While this heuristic worked for a few examples, it was definitely not quite right for others. Then, I thought I would just refresh the screen every 100 commands run. That worked better, but it caused some flicker because some refreshes are happening at unfortunate times.
Just as I was typing up a frustrated complaint to Zach, it suddenly occurred to me: Since there're no such commands as `break` or `return` (except the `value` command in `number` blocks, but we made those atomic, remember?), there is simply no possible way to break out of a `forever` loop. Therefore, it logically follows that all `forever` loops are render loops (unless our programmer has messed up terribly). Therefore, we could simply do a refresh at the bottom of every `forever` loop.
Turned out that this heuristic worked well for all examples except maybe one. But I think it is a nice heuristic, so I decided that instead of fixing the problem, I'll fix the source of the problem—I added a `refresh` command to that particular example.
After fixing up the `refresh` problems, my implementation seemed to be pretty snappy, in fact, too snappy. Some of the more sophisticated examples are running at 1-2 FPS on the original Java applet while doing great on mine. I thought: doing creative coding back then must be hard! The original authors must have been imagining a fluid animation in their mind while looking at these choppy slideshows. Then after some Internet archeology, I discovered that you can add this line:
to the `dbn.properties` file, which, simply removes the slowdowns. Duh!
In addition to the original "play" and "stop" buttons, I added a "step" button, which lets you run one line at a time. Below the canvas, I added a table showing you the variables and values in each scope at the current moment. The line currently being evaluated is highlighted. It's rather fun to watch the computer tirelessly jumping around to trace out your little design!
You can also customize the refresh rate by changing the number of steps to run at each instant (or run as fast as possible by only allowing auto-refresh at the bottom of `forever` loops). There's also an option to set the maximum number of milliseconds you can get stuck for: just in case you wrote some bug and the page starts to freeze.
I showed the project to Golan, who was very happy that his old sketches were brought back. When I was doing my undergraduate studies at Carnegie Mellon, Golan taught this class called "Interactivity and Computation", where one of the assignments was to make a clock. Golan told the story about how he was given the same assignment while he was a student, and how he made his "banded clock". And so, perhaps it would be logical to conjecture, that these gradient stripes moving across a tiny rectangular portion of my screen, was the running of the original source code of that fabled "banded clock".
Taking Golan's suggestions, I added a "fullscreen" button and an "export GIF" button. As I did my own rasterization into a matrix and only use the web canvas API as a display, the blown up version of the sketches look crisply pixelated instead of blurrily anti-aliased as canvas graphics usually are.
Then I wrote my own animated GIF encoder from scratch. Well not entirely from scratch—I've previously written a tiny non-animated GIF encoder, which used a trick on LZW compression scheme to encode a palette of 127 without compression. It turned out animated GIF isn't that much harder than non-animated ones. You just need to know what additional bytes to write! The entire code is only 50 lines long. All the useful bits of info can be gathered from the Wikipedia page about GIF.
I repurposed the "print" button (who prints stuff these days anyways) into the GIF export button. (You can still access the browser print menu if you really want to print it). Then you could specify the starting frame, and the number of frames to record, and in super-realtime, a GIF will be generated and downloaded to your local disk. Of course, the GIF is totally uncompressed. You could then either use the ugly graphical interface of ezgif.com or the inscrutable commandline interface of gifsicle to process it however you like.
There are various other small (hopefully) improvements which you can find described in the help file.
Zach showed the project to John Maeda, who gave the project the final "seal of approval". Now DBN.JS is no longer some Chinese counterfeit (technically it is made by a Chinese person though).
I had great fun re-implementing Design by Numbers, and just as much exploring the example programs and writing a bit of my own. One might think that DBN is not nearly as powerful as Processing or openFrameworks, what is the value of remaking it beyond preservation? I think that it is also fun to play with limited systems, to push its limits and to learn from doing so, just like how I still enjoy woodblock printmaking when laser printers are a thousand times "better".
As I mentioned at the beginning, I was on a journey to explore all creative coding systems that exist or existed. DBN was the first, what comes next? Macromedia Flash? Hypercard?