Semi-Realistic Rotary Experiences is a set of six simulations of real life scenarios involving the gesture of rotating things. Those who wish to experience them need to perform such rotations on one to four physical knobs in an exacting and skillful manner.
During one of the Future Sketches group meetings, we had the idea of parametrically controlling generative graphics with a set of knobs. Having recently picked up physical computing, I was excited to prototype and see what fun experiments I can come up with!
In electronics, knobs are usually implemented with "rotary encoders". There's a wide variety of them available -- from a couple of dollars to hundreds of them (as Prof. Zach puts it: BMW of knobs); Some of them come with "breakout" boards (partial pre-designed circuitry). Seeing that the ones with or without breakout boards are virtually the same price, I got myself 5 breakout continuous rotary encoders for $10 for the sake of this experiment.
After inspecting the circuits on the breakouts, it's time to design my own circuit to match them. I've previously made a dev-board for the SAMD11D microcontroller, which I thought would suit this purpose -- it has many pins, but not much memory or power, so perfect for reading some simple buttons or some such. The resistors needed for rotary encoders are already on the breakouts, so all I need is some sort of "converter" that connects the lines between the breakouts and the dev-board. Thus I quickly routed the PCB in KiCAD and milled the board on the Roland SRM-20 in the basement of the Media Lab.
The next step was to design the packaging -- a nice box to hide all the messy circuits inside. For that, I use a simple JavaScript script I wrote specifically for generating cases for PCB's. I 3D printed the parts on a Ender-3 v1 at home. I am quite bad at using a caliper, remembering numbers, and calibrating a 3D printer, so I had to print the case three times to finally get all the dimensions right for them to press-fit.
The dev-board has its separate case, which can be slid snugly into the case containing the rest of the circuits; This modular design allows multiple projects to share the same dev-board. My only regret is that it takes somewhat too much horizontal space for the simple thing it does, due to all the mating pin-headers. Next time I'll probably use the rotary encoders without breakouts and stack the layers vertically.
Arduino environment comes with a handy library called "Keyboard", which allows your board to trick the computer into thinking it's a keyboard. Therefore, instead of coming with a custom protocal to communicate between my hardware and software, I can simply generate keystrokes that are universally recognized. Rotating the first knob anti-clockwise by a notch generates an "A" keystroke, and rotating it clockwise by a notch generates a "D" keystroke, pressing it down produces a "1" keystroke, and so on.
It's fun! You can even control ordinary softwares such as web browsers or even games with these keyboard-imposter knobs.
With the hardware and firmware problems out of the way, I can finally start what I'm good at: code! What interactions can I make with 4 knobs?
After some initial experiments with parametrically controlling various graphics, I became even more obsessed with the rotary gesture of turning knobs. My question becomes, what experiences in real life, that involves such gesture, can be simulated?
I immediately thought of screwdrivers. Having spent much time last semester screwing things (with my shaky hand), I want to make something to show how "fun" it is.
I quickly sketched the scene in my head: a screwdriver hovers around in mid-air, with its motion wobbled by Perlin noise, supposedly due to an (invisible) shaky hand grabbing it; A bolt half-planted in a deep orfice waiting to be fastened. The apparent challange is to wait for the screwdriver to line up with the hole before inserting and twisting (by holding down the rotary encoder and turning it).
For the graphics, I decided to use 3D wire-frames (They look nice and are easy to produce). Instead of using a modeling software (which I loathe), I generated them directly in openFrameworks (which I like).
Initially I was just using the built-in primitives such as cones and cylinders, but soon find them to include extraneous lines that reduce visual clarity (which is understandable, because meshes are triangles), and fall short when more complex geometries are needed. So I put together my own subroutine, for drawing vertically stacked frustums, which I call "ofDrawVase":
Later I also added "ofDrawTube", for more complex situations where segments can have differing orientations. However, the code for that is quite a bit hairier, so I'm not too proud of it to post it below.
The actual "gameplay" is slightly easier than it seems: when the screwdriver is somewhat near the hole and the user presses down the knob, the screwdriver can slide into the hole. However, if you accidentally let go of the knob, the screwdriver falls out of the hole again, requiring you to re-align and re-insert to finish screwing. (When play-testing, I was glad to see many people struggle with this part).
The raw data from the encoders are somewhat noisy, and can generate false or extraneous inputs. I heard that this can be solved by adding capacitors in the circuits. Well, next time, maybe. For now, I used linear interpolation to smooth things out.
What other rotary experiences are there? Maybe screwing two screws at once with two screwdrivers? I thought that'd be a good idea too, but I wanted to try something different first.
In this second experience, you get to un-cap a shaken-up bottle of soda, with finesse, such that it doesn't explode and make a huge mess: first, you unscrew a little bit and wait, to let the gas out, then you unscrew the rest.
The cool part is that I get to make some exploding particle effects (when the user fails the objective). I coded this with a bunch of tiny wireframe cubes, whose motions are determined by some simple physics.
Whenever I stay at a hotel or move into a new appartment, I always have a fun time figuring out how the shower knob works. They come in strange varieties, but almost always there's only but a tiny sweetspot where the water temperature is neither freezing cold or boiling hot. This is complicated by the fact that there's a delay between turning the knob and the water reaching that target temparature. But over time you remember that sweet-spot and never have to worry any more.
The third scene is an exact simulation of that: helping a stick-figure take a shower by turning a knob. Among the 20 notches on the rotary encoder, only 1 produces the comfortable temparature, after a small delay.
When the water is too hot or cold, the character will play a cooresponding animation to show you they're suffering. For cold, I simply make their legs tremble, and the non-knob-operating arm to hug themself. For hot, I add wild, random swings to all their limbs, making it look like they're screaming "Ahhh! I'm buring!!". The amplitude of the motions correlates to how much the current temperature deviates from the desired temperature.
Despite the crazy animations, one of the character's hand is always on the knob (ready to turn it at your command), thanks to some simple inverse-kinematics I implemented. Which sometimes look funny as if their hand is glued or cuffed there.
The water particles, in addition to obeying gravity, bounce off the character's body (and the ground). This is done by computing point-segment distance between a droplet and each of the limbs. If the distance is within a certain threshold, there's a random chance it will receive a force in a random direction. The random chance is to create an illusion of depth: those droplets that are not on the same vertical plane as the character does not interact with it. The random direction is not really physically accurate but looks convincing enough.
There's also an "easter egg". If the water temparature stays at a comfortable degree for a certain time, the character will start singing (i.e. music notations fly out of their mouth).
So far only one of the four knobs is used. Can there be an experience that requires all four? Leveling the bed of a 3D printer came into mind. When I first got my low-end 3D printer, I had a lot of headache making its bed level. If the bed is too low, the filament won't stick onto it; If it is too high, the nozzle will crash into it and filament cannot come out. There're four screws each on a corner, and by adjusting them you can get a combination of too high/too low from each quadrant of the bed. A surf on the Internet seems to confirm that this is a universal problem.
I'm very proud that this simulation is just as annoying as the real thing. Well, perhaps slightly less annoying, since there's no danger of wasting filament or worse, breaking equipment. But it's the best I can do.
Though it seems intuitive, the relation between the pose (position + rotation) of the bed and the vertical position of the four screws is a bit tricky to get physically accurate. This is due to the facts that the screws are supposed to have springs on them, and that only three points, not four, are required to determine the equation of a plane. I went for a simplified approach: The normal of the plane is calculated by averaging the normal at the position of the four screws (each calculated with regard to 2 neighboring screws); The center of the bed is calculated by averaging the position of the four screws. After some testing I find this to be a reasonable approximation.
The filament is simulated with spring physics. I had to tune the constants a bit, but when I suddenly experienced the traumatic feeling of "oh no this is exactly how it looks like when I forget to level the bed", I knew I got the parameters right.
There're four morning meetings tomorrow. It is three o'clock in the morning. Can you wake up in time? Better set the alarm!
The main challange with this one, is that on an old-school alarm clock, there's only one alarm hand that sets both the hour and the minute. You'll need to eyeball the minute by positioning the hand somewhere between two numbers.
On the left, there's a phone screen displaying the schedule of the coming day; while on the right, there's an alarm clock, which the player needs to set to a time before the first meeting.
It was a bit tedious to clone the user interface of the iOS calendar app, but it was nice to see it in wireframe style (and Hershey fonts!), especially after the perspective transform.
The initial time displayed on the clock and the phone screen matches the real local time of the user experiencing the software. When the alarm is set and confirmed, it fast forwards to the time of the alarm. So there is some tricky date-time math involved, and if you've programmed these before, you know it can be very tricky. Hopefully I got it right.
Turning on the stove is a thrilling activity. There's always the danger of burning or poisoning yourself, and when you fail to do it, you starve (or have to eat cold stuff).
After some research (through asking people about their stove and reading PDF's of stove manuals found online), I realized that there're several different logic for these stove knobs. For some, pressing down the knob activates the igniter, and turning adjusts the flow of the gas; For others, there is an angle to which the knob must first turn to ignite, and further turning adjusts the flow of the gas. For some, pressing down the knob is necessary for the knob to turn at all, for some, not. In the end, I went with the style of the knob I grew up with: Press to ignite, turn to flow gas.
Interestingly, most people who playtested this got it right the first try. "Oh it's a stove", and they proceeded to turn on the fire, just like that. However, they also tend to complain that there's no way to kill themself with leaked gas.
The fire is simulated with wireframe cubes, much like water droplets in previous scenes. Interesting how with the magic of animation, a bunch of cubes can be made to resemble anything.
The source code of the software and the hardware will be shortly made available on GitHub.