Concept and Context
What was I trying to create, and why?
Hope Chapel in Austin, TX used to offer a free Fall Festival each Halloween for the neighborhood kids. In 2009 I wanted to create a fun, interactive booth and I finally hit on an idea that would combine placemaking and technology: theme a room as an outdoor 50s-era drive-in and create an interactive environment using wiimotes and projectors and Flash.
I didn't come up with the name for the booth right away. I wanted it to communicate several ideas: 1950's setting, cheesy sci-fi movie, outdoor drive-in. Meanwhile, the folks that organize the event at Hope Chapel needed a name so they could ask volunteers to help. It was 23rd in the list, so it was first generically designated "Booth 23," and from there the name evolved to "Plan 23," an obvious reference to the infamous Plan Nine From Outer Space.
Not the most original or memorable name, I know! :)
Here's my concept: Create an "inside for out" environment where an indoor room is transformed into the desert/mountain region near Tucson Arizona at sunset, right as the latest 50s sci-fi movie is starting. Space is tight, so most of the set pieces were to be back-lit flats, creating the illusion of nearby mountains, the drive-in movie screen, neighboring cars and even the concession stand and playground in front of the screen. Most of the lighting in the scene was to come from the "sunset" behind the mountains. Tightly constrained blacklights behind the center stage elements would light stars overhead.
Although seated in a large, enclosed building, diners experience the illusion of eating in an outdoor restaurant at nighttime.
Two cars (miniature front ends and carseats to suggest convertibles) in the middle of the room will seat up to 4 children for a game that lasts not quite 2 minutes total. The wiimotes would be housed in old-school flashlight cases, color-coded to match the color of light they "shine."
Wii remotes (henceforth "wiimotes") are sophisticated devices that contain infrared cameras as well as motion sensors and use Bluetooth to communicate with the Wii console. An open source project called WiiFlash provides an easy way to connect the wiimotes to a Flash application on a Bluetooth-enabled PC. Using two projectors mounted near the ceiling and a custom array of infrared LED sensor bars, I could create a Flash game where the kids use the wiimotes like they were flashlights to scare away the flying saucers before they abducted all the cows grazing on the landscape.
I was inspired by a youtube video in which some developers were using a wiimote and some custom LED sensor arrays to playing Half-Life 2 in a large virtual reality theater environment.
Using measurements of the room where Plan 23 would take place, I created the scene within Blender, a 3D modelling and animation program. This gave me an idea of the amount of room I'd have for set pieces, how far back the projectors would need to be and most importantly, what kind of angles I was dealing with for the wiimotes and their sensors. No particular attention was paid to the lighting or textures within the scene, nor to the accuracy of the meshes representing the car fronts, etc. — I just needed to make sure everything fit.
Widescreen Wiimote Application
Thanks to WiiFlash, it was surprisingly easy to hook up the wiimotes to my PC and get access to their data within a Flash movie. What ended up being surprisingly difficult and time-consuming was to create a system which would allow the wiimotes to function in a widescreen environment such as my room. I went back and forth between trying to understand and adapt another system I found and simply creating my own.
Most people probably don't know this, but wiimotes don't direct a beam of light anywhere; optically, they are receivers rather than transmitters. They contain a camera that identifies up to 4 sources of infrared light and sends the size and coordinates of those points, along with accelerometer data, to the receiving unit via Bluetooth (radio). The light in a normal Wii setup actually comes from clusters of infrared LEDs at each end of the "sensor bar" that you put near your TV screen. The Wii calculates where the wiimote is pointing by using that point data and extrapolating: if the points are to the upper right of the wiimote camera image and the wiimote is being held with the buttons up, the user must be pointing the wiimote down and to the left.
The wiimote camera has a pretty narrow viewing angle: 20° horizontally and 15° vertically. This is fine for normal wiimote usage where you're far from a relatively small screen, but in my case this would not work.
In a normal setup, even if you point the wiimote at the very left edge of the TV screen the sensor bar is still well within the wiimote's viewing angle. For Plan 23, astandard sensor bar would leave users only able to effectively point at a very small portion of the playing surface. The same applies for tilt angle up and down. I needed to set up multiple sensor bars and figure out a way for the wiimote to know which one it was pointing at.
It's often best to reuse code rather than starting from scratch. I found some folks in New Zealand who had already done something similar (abstract, youtube) and I wrestled with their code for a while. It was in C#, which I didn't know, but is similar enough to other languages for me to get the gist of it. With 1000s of lines of code, however, I couldn't get an accurate read on their algorithm. Without going into detail, it became apparent that I would spend as much time understanding and adapting their code as I would making my own.
Their code was written for multiple sensor bars that are essentially similar and spread out over the viewing area. I decided to make each sensor bar unique. The code for handling this is much simpler (400 lines rather than 2400), but requires costlier and more difficult construction.
The placement and spacing of the sensor bars is sensitive because the wiimotes only identify 4 points of IR light at a time, and there's no control over which 4 it identifies. If there are more than 4 at a given moment, the wiimote latches onto the first 4 it sees until one of them drifts offscreen, at which point it picks up another one to track, not necessarily the brightest or the one closest to the others, etc. The same point of light might be identified as "point 1" at one moment and as "point 2" the next. The wiimote might latch onto 4 points each belonging to a different sensor bar! So it was important to place the sensor bars in such a way that one bar (two points of light) would be completely visible at any given angle and also be identifiable. If that sensor bar drifts out of range, two of the new dots that are picked up had better belong to another sensor bar!
By viewing the 3D Blender scene from the perspective of a virtual wiimote camera placed where the children would be sitting, I realized that my wall would need 9 unique sensor bars. It was simple enough to make the Flash movie look for "couplets," dots of light that were close enough together to belong to a single sensor bar. To give couplets a unique signature I used several variables:
- isHoriz — Some bars were horizontal, some vertical. Even accounting for the various viewing angles (and correcting for roll if the user isn't holding the wii with the buttons up), it's easy enough to tell if a bar is horizontal or not.
- isBalanced — The wiimotes look for "blobs" of light to report as a single dot. If multiple LEDs are placed close enough together (as is the case with each end of the standard wii sensor bar), the wiimote camera identifies that light source as one "point". This point might be larger or smaller depending on how spread out that cluster of LEDs is. If one of the clusters is sufficiently large, the sensor couplet can be identified as balanced or not.
- isSouthWest — Amongst the "imbalanced" couplets, they can be pointing left/right or up/down. This variable can't be used with the balanced couplets so it doesn't quite provide another degree of freedom, but we were going to need another variable anyway since we have more than 8 sensors to identify.
- isUpper — 5 of my sensor bars were positioned in the upper half of the wall and 4 in the lower half. Since the wiimote's accelerometers provide reasonably accurate tilt data, I could use this and my knowledge of the geometry of the room's layout to identify couplets as upper or lower. This last variable is the least robust. The accelerometer "sensorY" data only corresponds accurately with the wiimote tilt if the wiimote is being moved slowly and smoothly. Also, the use of this variable adds an undesireable constraint to the system: kids have to remain seated while playing. If they change the geometry by standing up, the algorithm might identify an upper couplet as a lower one!
Construction and Deployment
Although I ran out of time for the "placemaking" details that I so wanted to construct, the game worked and was a success. Since I was busy until the last possible second getting the game mechanics working, I have very little by way of photos or video to show for all my efforts.
It took several weeks longer than anticipated to choose, implement and troubleshoot the widescreen wiimote algorithm. I had to wait until this was nailed down and run tests with makeshift LED arrays before ordering parts so that I would know how many were needed. To make matters worse, the first supplier I tried to use took my money and delivered no LEDs, so there was extra delay in ordering them elsewhere.
It's no surprise that development on something unusual like Plan 23 should take extra weeks. The unfortunate downside, however, is that I simply ran out of time for crucial elements like set construction and lighting. I envisioned myself having the mechanics up and running a week before the event such that I could move in early, tweak things, run usability tests, take promo shots and video, etc. Ha! In my dreams. In reality, I had to open the booth a half hour late because I was still trying to get the code to behave properly with all 4 wiimotes running, which I hadn't had time to test yet!
So, I ran out of time for making playground and additional car flats. I had no time to sculpt the front ends of the cars and by the time they were out of the equation it made no more sense to bother using real car seats. The long and short of it is that I was able to dedicate only a few short hours to the slapping together of some minimal set pieces instead of being able to do the richly detailed theming that I had envisioned.
On the plus side, I did in fact get the game up and running, even though the wiimote "mouse movements" (projected flashlight spots) were erratic. This was no small accomplishment, and I looked forward to the next year when I could set the booth up again and have time to devote to the placemaking elements which are so important to me.
2009 Special Thanks to: Tim Jones and Melissa Rowland for extensive help setting up the booth, Todd Garza for cutting the video within the game, Tony Hyden for making our cool poster, and Adam Hull for running it better than I could.
2010 Version and work done in 2011
This was similar to the original version, but with the following major change: Hope Chapel discontinued the Fall Festival, so I set up the installation in my backyard garage and entered the booth as part of the inaugural Trail of Frights.
Remember what I said in 2009 about how I'd be able to devote tons of time to scenery in the next version? I hope you enjoyed that. As you may have guessed, there were no improvements to the scenery or lighting, and some set pieces actually took a step backwards! I spent all my time adjusting the infrared LED arrays to the new geometry of my garage (which required a new Blender model and new setup for hanging the projectors) and making improvements to the wiimote tracking software.
Actually, I spent a lot of time experimenting with pulse width modulation for the LEDs. I was trying to use something other than size to distinguish the points of light that the wiimote was seeing, and tried making some LEDs blink with different duty cycles. I ran into two problems with this idea.
One is that the wiimote reports data back to the base 100 times a second, and there was no easy way to synchronize the blinking of the LEDs to that cycle. This was not insurmountable, but there there was the second problem, which was a glitch somehow in the reporting which wasn't happening reliably every 100th of a second. These glitches in the regularity of the updates killed any chance I had at coming up with an error correction algorithm. So after much time and effort spent, I had to stick with the previous erratic wiimote movements.
I didn't re-enter the installation in the 2011 Trail of Frights. I was too swamped to make it happen, but I did do work to prepare the way for the 2012 entry.
This is worth special mention because I learned so much and went through so much trouble making it. Plus, it came out pretty cool, even though it's not strictly finished yet.
Since my original intention was to make two different ride vehicles (a Corvette and a Buick Super), I didn't want to go through the trouble of making a mold that I would only use once, and decided instead to make a positive form (plug) that I would simply lay fiberglass on top of. This turned out to be very troublesome, since fiberglass strand has a tendency to warp or buckle or trap air bubbles. If I do this again, I'll have to either use small, shredded pieces of strand mat or just go ahead and make a huge mold.
Dashboard #1 and Front Grill
Rear End (done much later)
2010 Special Thanks to: Amanda Walker and Dana Spolum for helping me create last minute set pieces and assemble it all. They went way beyond the call of duty.
Thanks to a successful Kickstarter campaign, I was able to improve upon the projectors, the set pieces and the lighting. Yay! I improved the Flash game as well, but didn't do anything earth-shattering with it. I still run into issues with the game freezing from time to time, but I think that those have to do with the hardware that I'm running the game on.
You might want to click through on that Kickstarter link and watch the promo video. It does a decent job of summarizing the project.
Improved Streetside Signage
Our garage is in our backyard so we really needed something attractive at the street to draw people. In addition to stringing up a bunch of lights to lead into the backyard, I spent a lot of time on new and improved signage and I'm delighted with the results. I wanted a Googie style sign on the street so I made one that incorporated both normal and fluorescent paint and lit it with regular and black lights. I did the design and painting.
List of Improvements
- Better projectors. Woo-hoo!
- Authentic-looking signage for the street.
- Entire fiberglass car for kids to sit in, including front end and back end.
- Improved intro spiel ("Grab your flashlights!...") with game instructions.
- Better game software: bug fixes, difficulty level adjusts automatically per user according to how well they're doing, flashlights beams have penumbra now, better collision detection with UFOs moving away from detecting beam.
- Better flats for the set pieces in front of you at the drive-in. Instead of hastily-shaped, vaguely-colored pieces of foam, I used our color printer to print out miniature playground pieces and cars and glued those to some thin foam board. Much better now.
- Fluorescent paint for the sunset and stars overhead.
- Space much more open with superfluous 2x4 beams removed from overhead and better "sky" in place.
- Better lighting within the set, including black lights which help the sunset and stars glow more and provide ambient lighting.
As you can imagine, I didn't take everything down the day after Halloween. I got close to achieving my kickstarter goals, but as October 31st approached I started to stress out until the thought occurred to me: "I don't have to get it all done before the Trail of Frights. I can actually keep working on it the day after!" So I relaxed and set my sites on some date after Halloween where I'd have a chance to get things fixed up and take some proper photos and video, which would require special lighting not appropriate to the running of the attraction anyway.
The main thing that desperately needed fixing was the "sky box," the cardboard hanging over and around the set to create the illusion of the sky. The cardboard on the ceiling held up fine since it was stapled in, but the pieces that were draping down buckled severely and didn't even begin to look right. So I at least had to redo that.
One big improvement I made to the Flash game was to include an intro spiel that explained the situation and gave instructions on how to play. So the drive-in screen would be looping on different things like "come by the concession stand" and then when guests arrived and were sat down, I'd start the instructional intro section and that would flow straight into the start of the game. Also, I added an "outro" when the game was over, with 50s-era "our playground is closed now" and "drive slowly" placards.
I continued working on upgrades until February when I finally dismantled it for good. I was sad to see it go, obviously, but I wanted my garage workspace back!
2012 Special Thanks to: all of my backers on kickstarter for your heart-warming support and for making improvements possible, Josh Cheatham for taking photos on Halloween night, Amanda Walker for taking video, and my amazing wife, Brie, for maintaining a positive, encouraging attitude during a difficult season!
Lessons Learned, Moving Forward
Still Needing Improvement
- I didn't think of this when I came up with the idea in 2009, but most kids today don't know what a drive-in theater is!! If I do this again next year, I need to provide some background info to help set the stage. The "queue line" going to my backyard should have little placards that the kids/parents can read.
- Wiimote tracking is still erratic; it works, but not as well as it should. If I install this again next year or do something again with wiimotes, this needs to be much improved.
- Sunset and clouds need a painterly touch. They look amorphous and "sprayed on." And the surface needs to be flat and smooth in order for it to effectively disappear— curtains and cardboard don't cut it because they give the brain visual cues that you're looking at something right next to you.
- Flash game needs major redesign. It lacks catchy narrative and fun replayability. The truth is, I sort of tacked on the game idea in 2009 without giving it proper thought. It doesn't stand up well.
Overall, due to a weak game concept and lack of details in scenery, the installation lacked the "wow" factor that I originally had in mind. In 2009, I wanted this experience to be so fun and so amazing that folks of all ages would be telling each other to check it out and that over the years the queue for it would get longer and longer. The reality is that I've put in an awful lot of work for very little result in this regard.
In retrospect I suppose the thing was not well designed or thought out, but even today I can't put my finger on one facet and say "the mistake was here." One thing I've learned is that it's not easy to pull off the "inside for out" concept in a tight space. There's a threshhold of believability that the guest needs to cross before they're willing to suspend additional disbelief. In spite of heroic efforts, the scenery fell short of this goal. I've noticed that in the Sci-Fi Dine-In Theater the atmosphere is simply painted on the walls instead of being built up in layers like I attempted. This may have been a better design choice for me; it's hard to see the drive-in walls as being walls when they're so close to you.
Project management is difficult — things always take much longer to accomplish than I initially think they should. This applies to hardware and software. So the challenge is to spend the available time and money on what makes the most difference rather than myopically focusing on some detail and perfecting it. I forced myself to leave the wiimote tracking software the way it was, even though it seriously needs improving, so that's a step in the right direction for me! :)
I learned some things about materials I've tried to use for the sky backdrop. I initially used corrugated cardboard (flat side out) for both the overhead sky and the surrounding "walls." That worked great where I was able to staple it in place overhead, but on the sides where it was hanging freely, it buckled and warped severely after I sprayed paint on it. Should have seen that coming. After Halloween, I tore that part down and bought some plastic sheeting to use on the sides instead. That worked better, but still is difficult to hang without it looking like plastic sheeting! If/when I try to create another environment in my garage, I'll have to come up with a better solution here.
Also, I've learned that my technique of applying fiberglass to a positive "plug" requires the use of many small bits of chopped strand mat rather than trying to use larger sheets. I went through lots of trouble to create a nice smooth plug for the carseat and then ran into lots of little bubbles and warps forming on it when I applied the chopped strand, no matter how much I tried to excise them! Ideally, I would spray on resin and then spray on the fiberglass strand as well, but lacking the hardware for that I can at least use lots and lots of little pieces. Next time...
Will UFOs threaten our skies again?
I had some folks ask me in 2013 if I was going to feature Plan 23 again in the Trail of Frights. Makes sense, since I had already put so much time and effort and money into it. Thing is, after many months of leaving the installation in place I finally (and somewhat tearfully) cleared out the garage so I could use it again as a workshop. Now that I have the space cleared, it would involve a lot of effort to reinstall it. I was needing a break.
The major factors in not bringing it back, though, were the aforementioned items that need improvement. The gameplay would have to be much more compelling and the placemaking more delightful for it to be worth the trouble. Until I have some sort of epiphany about the game portion, it's really not worth the trouble, and my time and money and effort would be better spent doing something new and fresh.
I keep little text files around my computer, some with little "readme"s to remind me why I did something a certain way, or in this case, a little to-do list of improvements I had in mind for Plan 23 should I get to it again. Here it is, without embellishment:
saucers are flying in too low adjust cow mask so they're not above the horizon! enable beam-crossing? show score at end runtime adjustment of NUM_WIIMOTES? use Sound Manager to centralize sound playback and volume add subtle cricket and other ambient sounds switch all tweens to Gtween or Tweenlite adjust Intro (add sound bite to intro placard, adjust timing) revisit sounds?