Case Study: The E8 Particle Simulator for Garrett Lisi
One of the projects that I have been privileged to work on in this last year, is for a friend named Garrett Lisi who is shaking up the theoretical physics field with his controversial yet elegant theory to unify pretty much everything in physics by mapping it onto the beautiful e8.
Today the video of his TED presentation came out, also available on youtube It's got a nice blend of humor and pretty profound What's cool is after working off and on on the project over about a year, finally seeing the video has helped me understand better what it is that we built, a real time particle simulation of his theory which you can play with at Garrett's website here.
It's an interesting case study, we weren't at sure that Flash was fast enough to do real time interaction of the 240 eight dimensional particles, and 10K interactions. But it turned out that the speed improvments in Flash9/AS3 to be fine. As a interaction designer it was an interesting challenge to help convey the various interactions in a user interface.
Part of the challenge was bridging the gap between Garrett's exceptional knowledge much of it requiring deep math and conceptual understanding that took years to acquire with that of the limitations of flash, and the practicalities of implmenting it. As is often in client projects, it's hard to convey what looks simple can be complicated, and contrarily what seems complicated can be super easy.
It helped greatly that Garrett had a mockup in Mathmatica, it was way less interactive, could only be stepped through, but is how he created his visualizations for the various videos. Mathmatica abstracts away much of the details that other programming languages like flash have to deal with to build multidimensional things. e.g. he would describe a single equation for a particle, and I would have to iterate through all the eight dimensions (e.g. x, y, z..) at 24fps to get it to work. Aparently rounding errors seem to be isolated from the user. However in our simulation, with the repeated calculations for every dimension, some care had to be taken to reuse math operations. To further improve performance, we also only compute changes when the user is dragging around.
One of the most tedious challenges was getting it to show the LaTex like symbol labels for the particles and dimensions. Rebuilding LaTex would have cost more than the project budget. There is a javascript+html solution, so we first tried just using fonts that contained many of the special symbols, but it turned out this required about 5 fonts that coordinating would have required rebuilding the routines in javascript, plus specifying them was tedious, it would have to be specified like "e^SQRT(1/SQRT(2)" might sigma to the power of ...you get the idea. The list of particles had to be specified in a particular way, and potentially there may be more than one particle system to map the physics properties onto. So Garrett had to render translate these from his mathmatica to an xml file that flash could consume. In the end, we threw away this approach there was no easy way to normalize the positions of the text fields. So we ended up punting and going to a movieclip with 240 frames, the symbols brought in as vectors after some Mathmatical->PostScript->Swf->decompile->Fla wizardry. This also reduced the xml overhead, since we just used an id to represent the frame of the label e.g.
labels.gotoAndStop(particle.label)
One of the cool parts of the application is the ability to select multiple particles that interact and produce others, and what of the eight dimensions to map to select for the current rotation. We spent many hours trying to eek out the exact way to signal options to the user, and deselect others. With the ability to mouse over a particle, indicate first, second and third selections deciding on colors and effects that didn't overload the already busy display. At first I start with CMYK as conceptually these are a color triangle and farthest apart, Garrett didn't like pinkish, so we went to RGB instead. We ended up using a static glow effect to indicate the selection state as since their identity was based on color, doing any other effect inside would make it hard to read. Since particles could be selected in different orders to produce similar results, the routine was far involved than most selection routines, rather than doing this at runtime, during the parsing of the xml, an object graph was created, largely linked lists so things could be very quickly traversed. This use of deeply linked object models I wish was better understood by many engineers, it can make huge performance differences, vs. using E4X, or relational database models.
Since particles show their selection state in multiple areas, the main viewport and the grid below it, we strongly used a MVC model, where all changes were to the model and these percolated up to multiple views. e.g.
public function setX(newVal):void{
if(curVal != newVal){
lastVal = curVal;
curVal = newVal;
dispatchEvent(new Event("ChangedX");
}
}
Since we had an onscreen display, there were actually two models, one for the underlying particle system and one for the onscreen presentations. So the rendering routine, reassociated and repositioned the onscreen particles which carried some additional view state (e.g. the reference to the sprite).
For the particles we ended up using the graphics API to draw them, as the symbols were reused at different sizes.
For the navigation, it took about 5 separate tries to come up with the tri-state indicator, as counterintutively any selected axis can be rotated in both the X and Y at the same time (as they are higher dimensions), in addition the entire set can be rotated in the plane, as if you had a pencil in the center. So here we have the term 'rotation' being used multiple times with very different implimentations. A common pitfall in projects.
Since Garrett's idea is an extention to the well accepted standard model, the need to show and hide these extra points, was necessary. We ended up normalizing the object model so that defining those common points in the 240 superset are indicated by a flag. What I ended up doing was using Tny to fade the extra particles in an out. In retrospect I would have preferred a slider.
The other cool but hard part of the application was the ability to project the particles into different coordinate systems. You can sort of imagine this like, the default projection was flatting a 3d image into a photograph (except 8 dimensions), and then another isometric projection...similar but different but able to be translated back and forth (all particles in 8 dimensions). During the information architecture phase we had though that the saved/bookmarked rotations would be independent. This lack of vision was very much like bringing to halves of a zipper together.
We started out with a simple 3d version, but in the end the 3d projection reduced the clarity relative to the 2d projection...a common thing I've discovered in doing infoviz. Since unlike a 3d model Garrett was more interested in the symmetries lining up, the browsing mechanism was changed to like a pan slider go from max left to max right, rather than rotate as much as you want. Here one of the intermittent bugs was due to math rounding errors, occasionally the rotation would over shoot the max by a tiny bit, and then this would spiral into weird skewing bugs. The key here was that at 24fps, small errors accumulate quickly, and in this case break the orthogonality of the intended rotation, so our solution was to add after the rotation another correction+constraint. As is typical with these projects, from his vantage it appeared to be something wrong with my code, and from my perspective there was something wrong with his formula. To solve, required bridging the gap, and stepping through the values to figure out what was going wrong, and it helped immensely that we were able to sit side by side in this phase. At one point it was immediately clear to him what the problem was, and the solution was immediately apparent to implement it. So like half a day of hassles over 3 lines of code!
Graphic Design is very important, so I went onto odesk and found a talented designer from Trigma who borrowed some of the style from other major apps to build what was done. This took about 3 iterations, primarily negotiating over how small to make the elements and to maximize the view area.
In the end, Garrett was happy with the results, and we achieved something that probably could not have been done easily with html, and will hopefully convey more information than just watching the video and certainly not by images alone. This is one of the rewards of flash. Now that the video is out one of the things we've discussed is synchronizing the display to the information presented in the video. Which would be pretty cool! But requires converting more code in mathmatica to rotate from any point to anyother point in 8 dimensions...
Garrett has made the code open source, so all the code is on code google so you can play with it if you want.