Media Lab Intern Spotlight: Rui Hu's Metropolitan Triangle Garden

From Don Undeen, Senior Manager of Media Lab:

For the past two years, the Media Lab has been experimenting with low-cost, accessible 3D printing and scanning technology. We've previously used tools like AutoDesk's 123D Catch to turn photographs of many of our favorite objects into digital models which we then share on Thingiverse. However, now that many of these images are readily available through Open Access for Scholarly Content (OASC) via the Met's website, the Media Lab is excited to see all the creative uses our extended community has found for them. Free, low-cost, and demo software tools give us lots of options for incorporating Met objects into our digital creations, and we're hoping that blog posts like Rui's will inspire you to make your own digital art!

Rui Hu. Metropolitan Triangle Garden, 2014

«Last winter I attended a presentation at Harvestworks Digital Media Arts Center, where I had previously worked as an intern. One of the speakers was Don Undeen, senior manager of Media Lab at the Met, who presented many interesting projects his Media Lab colleagues were working on that used a number of emerging technologies. Among these technologies were 3D scanning and 3D printing, which I had a particular interest in thanks to my background in video, 3D animation, photography, and sculpture.»

To me, 3D scanning is a way of photographing, and 3D printing is a way of creating sculpture. The entity in between the early scanning and the final printing, however—the virtual 3D model—becomes something very interesting on its own. I later wrote to Don about a project idea to create a 3D animation using 3D-scanned models from the Met's wide-ranging collection of sculpture. When I was in school the 3D program we used was Maya, a very popular 3D animation program, but during the initial stage of the project I also came across a lesser-known program called Houdini. Houdini is a powerful program with a particular strength in dynamic simulation, and is used by technical directors in many of the feature films from the leading visual-effect studios.

One of the most unique features of Houdini is its procedural workflow. When users want to add an effect to a 3D model, instead of permanently altering the model (like many other 3D programs do), Houdini just adds a new step of instruction—represented by a node in the operation window—that will not affect the node representing the original model. Different branches of nodes can create completely different effects, and users can switch between branches to see the different effects, or go back and forth between different nodes in the same branch to see before and after the effect. In other 3D programs like Maya, this would not be possible: Once the user adds an effect to the model, the set of instructions that creates the effect will be fused to the model; and if the user wants to create a different effect or to see what it was before, the previously added effect has to be undone first. It is a very linear process that does not have much flexibility.

A demo of different effects applied to a single sculpture. Once created, switching between these effects is just a click away in Houdini, all while keeping the original model intact.

The procedural, node-based workflow of Houdini has a huge advantage in that it allowed me to explore many different ways of doing things without doing any "damage" to the 3D model, and I could easily switch between the renderings to compare the look of each. This workflow provided me with a lot of room for experimentation. I decided to use Houdini for this project because I didn't want to make a character animation with a clear narrative. Rather, I wanted to both actively engage with these originally static sculptures while also exploiting what the 3D program could do. I put the 3D-scanned models of the sculptures into the virtual museum space, and make them deform in ways that defy their physical properties. Some of them even burst into a floating stream of triangle pieces!

Houdini Test: Glitch Two

Glitch applied to the final animation

Computers usually use triangles to represent 3D shapes. After the simulation process the models are no longer sculptures, and the illusional image of computer data is broken—the animation becomes a performance of classical masterpieces and contemporary informational junk. The digital museum space was no longer just a space displaying static sculptures, but had become a theater of transformation and decay.

Houdini Test: Sculpture Factory

Houdini Test: Transformation

Many of the effects produced in the project are glitches, or misused simulation processes, created in ways that would probably be considered wrong by a professional visual-effects studio. One process breaks a solid sculpture into a pile of broken pieces, while another turns a sculpture into liquid so that it melts and flows away. But what if I fed the liquid sculpture data back to the breaking process, so that the program tried to "break" the liquid as if it were made of solid material? It would turn into a flow of disjointed triangles. This is an example of misusing the simulation process. Doing it too much might crash the process and you end up with nothing but an error message, since the glitch is a product of balance; you want to do something "wrong" and cheat the computer into thinking it's doing something correct, but not to go too far for the program to quit working.

A glitch in the final animation

Houdini Test: Glitch One

In the process of making the animation, I received valuable support and feedback from the staff, educators, and artists in residence at the Met. One particular example was the 3D architecture model of the Museum provided to me by the Met's Construction and Facilities Department, which was created by the Beyer Blinder Belle architecture firm. Without it, my idea for the animation would not have been possible.

The highly detailed architecture model of the Museum's Department of Greek and Roman Art galleries, as rendered​ in Houdini

The lengthy and tedious process of reducing the number of pieces in the model so that the file size is not too big to use for animation

I also experimented with 3D printing to create a new form of sculpture. One of the pieces I produced broke the sculpture models in the 3D program, to later remerge them into a pyramid-shaped pile of broken body parts. This turned out to be a challenging job to print in 3D because of many overhanging parts. The view of the object in the 3D program thankfully gives an idea of what it would look like.

Houdini Test: Fracture

The broken-and-remerged sculpture used to test 3D printing, as rendered in Houdini

The galleries of the Department of Greek and Roman Art proved to be an incredible source of inspiration throughout my project. These sculptures were created with care and mastery, treasured in their time, broken and tossed during historical turmoil, and then preserved and admired once again. The original delicacy and the torment of time—shown in the number of missing or broken body parts—combined to create an altogether new sensibility. In the virtual world, each of the 3D models is a character, a piece of sculpture, and digital data, all at the same time. The viewers experience the Classical beauty along with the digital glitches that distort the sculptures; it becomes a visual orchestra of cultural objects and living beings losing their individual identities and merging with each other in a timeless virtual space.

My experience in the Media Lab was extraordinary. The internship really focused on supporting individual interests, and there were abundant opportunities to meet staff from other departments and experts from outside the Met who provided feedback on and insight into my project. After completing the internship, besides my school's animation showcase, my project has been accepted into the Outcasting: Fourth Wall moving-image festival in Cardiff, Wales. I feel incredibly honored to have my work screened alongside that of artists I admire such as Jonas Mekas and Jordan Baseman. I see this project as a series of long-term experiments, and I hope that it can be seen by a broader audience in the future.