Archive for the 'tny' Category

Case Study: The E8 Particle Simulator for Garrett Lisi

One of the projects that I have been privileged to work on in this last year, is for a friend named Garrett Lisi who is shaking up the theoretical physics field with his controversial yet elegant theory to unify pretty much everything in physics by mapping it onto the beautiful e8.

Today the video of his TED presentation came out, also available on youtube It's got a nice blend of humor and pretty profound What's cool is after working off and on on the project over about a year, finally seeing the video has helped me understand better what it is that we built, a real time particle simulation of his theory which you can play with at Garrett's website here.

It's an interesting case study, we weren't at sure that Flash was fast enough to do real time interaction of the 240 eight dimensional particles, and 10K interactions. But it turned out that the speed improvments in Flash9/AS3 to be fine. As a interaction designer it was an interesting challenge to help convey the various interactions in a user interface.

Part of the challenge was bridging the gap between Garrett's exceptional knowledge much of it requiring deep math and conceptual understanding that took years to acquire with that of the limitations of flash, and the practicalities of implmenting it. As is often in client projects, it's hard to convey what looks simple can be complicated, and contrarily what seems complicated can be super easy.

It helped greatly that Garrett had a mockup in Mathmatica, it was way less interactive, could only be stepped through, but is how he created his visualizations for the various videos. Mathmatica abstracts away much of the details that other programming languages like flash have to deal with to build multidimensional things. e.g. he would describe a single equation for a particle, and I would have to iterate through all the eight dimensions (e.g. x, y, z..) at 24fps to get it to work. Aparently rounding errors seem to be isolated from the user. However in our simulation, with the repeated calculations for every dimension, some care had to be taken to reuse math operations. To further improve performance, we also only compute changes when the user is dragging around.

One of the most tedious challenges was getting it to show the LaTex like symbol labels for the particles and dimensions. Rebuilding LaTex would have cost more than the project budget. There is a javascript+html solution, so we first tried just using fonts that contained many of the special symbols, but it turned out this required about 5 fonts that coordinating would have required rebuilding the routines in javascript, plus specifying them was tedious, it would have to be specified like "e^SQRT(1/SQRT(2)" might sigma to the power of ...you get the idea. The list of particles had to be specified in a particular way, and potentially there may be more than one particle system to map the physics properties onto. So Garrett had to render translate these from his mathmatica to an xml file that flash could consume. In the end, we threw away this approach there was no easy way to normalize the positions of the text fields. So we ended up punting and going to a movieclip with 240 frames, the symbols brought in as vectors after some Mathmatical->PostScript->Swf->decompile->Fla wizardry. This also reduced the xml overhead, since we just used an id to represent the frame of the label e.g.
labels.gotoAndStop(particle.label)

One of the cool parts of the application is the ability to select multiple particles that interact and produce others, and what of the eight dimensions to map to select for the current rotation. We spent many hours trying to eek out the exact way to signal options to the user, and deselect others. With the ability to mouse over a particle, indicate first, second and third selections deciding on colors and effects that didn't overload the already busy display. At first I start with CMYK as conceptually these are a color triangle and farthest apart, Garrett didn't like pinkish, so we went to RGB instead. We ended up using a static glow effect to indicate the selection state as since their identity was based on color, doing any other effect inside would make it hard to read. Since particles could be selected in different orders to produce similar results, the routine was far involved than most selection routines, rather than doing this at runtime, during the parsing of the xml, an object graph was created, largely linked lists so things could be very quickly traversed. This use of deeply linked object models I wish was better understood by many engineers, it can make huge performance differences, vs. using E4X, or relational database models.

Since particles show their selection state in multiple areas, the main viewport and the grid below it, we strongly used a MVC model, where all changes were to the model and these percolated up to multiple views. e.g.


public function setX(newVal):void{
if(curVal != newVal){
lastVal = curVal;
curVal = newVal;
dispatchEvent(new Event("ChangedX");
}
}


Since we had an onscreen display, there were actually two models, one for the underlying particle system and one for the onscreen presentations. So the rendering routine, reassociated and repositioned the onscreen particles which carried some additional view state (e.g. the reference to the sprite).

For the particles we ended up using the graphics API to draw them, as the symbols were reused at different sizes.

For the navigation, it took about 5 separate tries to come up with the tri-state indicator, as counterintutively any selected axis can be rotated in both the X and Y at the same time (as they are higher dimensions), in addition the entire set can be rotated in the plane, as if you had a pencil in the center. So here we have the term 'rotation' being used multiple times with very different implimentations. A common pitfall in projects.

Since Garrett's idea is an extention to the well accepted standard model, the need to show and hide these extra points, was necessary. We ended up normalizing the object model so that defining those common points in the 240 superset are indicated by a flag. What I ended up doing was using Tny to fade the extra particles in an out. In retrospect I would have preferred a slider.

The other cool but hard part of the application was the ability to project the particles into different coordinate systems. You can sort of imagine this like, the default projection was flatting a 3d image into a photograph (except 8 dimensions), and then another isometric projection...similar but different but able to be translated back and forth (all particles in 8 dimensions). During the information architecture phase we had though that the saved/bookmarked rotations would be independent. This lack of vision was very much like bringing to halves of a zipper together.

We started out with a simple 3d version, but in the end the 3d projection reduced the clarity relative to the 2d projection...a common thing I've discovered in doing infoviz. Since unlike a 3d model Garrett was more interested in the symmetries lining up, the browsing mechanism was changed to like a pan slider go from max left to max right, rather than rotate as much as you want. Here one of the intermittent bugs was due to math rounding errors, occasionally the rotation would over shoot the max by a tiny bit, and then this would spiral into weird skewing bugs. The key here was that at 24fps, small errors accumulate quickly, and in this case break the orthogonality of the intended rotation, so our solution was to add after the rotation another correction+constraint. As is typical with these projects, from his vantage it appeared to be something wrong with my code, and from my perspective there was something wrong with his formula. To solve, required bridging the gap, and stepping through the values to figure out what was going wrong, and it helped immensely that we were able to sit side by side in this phase. At one point it was immediately clear to him what the problem was, and the solution was immediately apparent to implement it. So like half a day of hassles over 3 lines of code!

Graphic Design is very important, so I went onto odesk and found a talented designer from Trigma who borrowed some of the style from other major apps to build what was done. This took about 3 iterations, primarily negotiating over how small to make the elements and to maximize the view area.

In the end, Garrett was happy with the results, and we achieved something that probably could not have been done easily with html, and will hopefully convey more information than just watching the video and certainly not by images alone. This is one of the rewards of flash. Now that the video is out one of the things we've discussed is synchronizing the display to the information presented in the video. Which would be pretty cool! But requires converting more code in mathmatica to rotate from any point to anyother point in 8 dimensions...

Garrett has made the code open source, so all the code is on code google so you can play with it if you want.

AS3: Image Posterization + ColorUtils + Adaptively Colored UI’s

Ported this posterization algorithm to AS3 . It will be in the com.troyworks.ui.ColorUtil, when released.

Posterization is basically compressing the color range of a photo into something smaller. Anybody who has worked with converting to 'indexed color' in GIF or PNG knows what this means. Say you have a sun against a sky of blue.  Since there are only say 8 colors, there is no need to have room for 24bit color across the entire range, making it dramatically smaller.  The other use is for artistic purposes,

Porting one to actionscript has allowed me to create something nifty. One of the design challenges when working with dynamically loaded images is complimenting the colors in the loaded image in the surrounding UI elements., and from a useability perspective making sure that text and UI elements are sufficiently contrasting from whatever dominate color is chosen, else they dissapear...is that "cut or delete?"

So towards that end, I created this prototype, click on one of the indexed colors to change the ui, the text will choose from one of the other color bands. The title will pick either white or black to attempt whatever the best contrast is from the selected color.

Click on any of the colored bars to have the entire UI adjust..all colors originate from the thumbnail.

This was accomplished by first posterizing the loaded image (the second image) and indexing the colors with a bandpass filter (filtering out anything too black or too white) via com.troyworks.ui.ColorUtil , sorting by how often the color occurs in the image to generate the histogram. Used com.troyworks.core.tweeny.Tny to do the color tweening. com.troyworks.data.ArrayX to get a random color from the bands.

From a design perspective, it's interesting how the low occurance spot colors when selected makes the image seem more 'playful' while the more common dominate ones 'serious'. I find myself wanting to put a better color distance in, to filter out things close to grays from the options.

There are also method to get shades and hues from a given color, but no examples yet.

All in all a fun way to spend a few hours this evening, while listening to Lily Allen.

AS3: How low can you go? Tny, a 1.9KB Tweening engine for Banner Ads

Challenged by a Banner ad size limit, I was curious how small I could create a tweening engine with basic color and sound support. Got it down to 1919 bytes, it can get down to ~1500 bytes if you don't need sound or color. Leaving room for more assets (damn Myriad Pro is a hog!)

Of course it's not just for banner ads, it basically wraps a given DisplayObject (including TextFields), and behaves just like setting properties of a clip directly (x, y, xscale), except that it eases to get there. Setting the duration acts as the play().

This makes it pretty easy to prototype with, so much so I'm contemplating using it as the proxy for whever I normally position things, to force me to think about motion earlier, just set duration = 0, and it will jump to the end.  It shares the same basic API as Tweeny, so progressive enhancement (if you need the playhead API) is easy to do.
It' s not the fastest engine purely on code, but with rendering overhead, it's among the best available in GreenSock's great little starfield tweening benchmark util

Here's an example of tweening the color and position of textfields, note that the engine is 1.9K, this banner is 10K due to fonts:

Using this code.

It's open source, it will be up after the first of the year, still trying to figure out how to get google code's SVN to show up on the blog, and there may be a bug or two lurking I haven't found yet.

As an aside, as this post hopefully shows, the blog is improving technically in periodic jumps, with some exciting upgrades planned in 2008. While a webdev pro, It's my first time professionally blogging and first time using Wordpress. Over the weekend upgraded to the latest, moved it to the correct domain, added several plugins for flash, code snippets sand spam filtering so comments and registration have been reenabled should work correctly. Permalink structures have changed so hopefully google will start indexing correctly by post instead of category.

In theory you should be able to subscribe to changes on pages you're interested in.