Bring an Island Deity to Life with Augmented Reality

The Met may have temporarily closed its doors, but the staff continues to find compelling ways to share the collection and their expertise. Today the Museum's Imaging Department is bringing to life a zemí cohoba stand—one of the most iconic sculptures that survives from the ancestral civilizations in the Americas—through augmented reality (AR).

This wooden ceremonial stand is the principal work in the exhibition Arte del mar: Artistic Exchange in the Caribbean, which explores the creative interaction around the rim of the Caribbean Sea before the sixteenth century between the Taíno civilizations of the Antilles archipelago and their powerful peers on the continental mainland.

Met Curator James Doyle says:

The Taíno concept of zemí pertains to the force of deities and ancestors that permeates the Caribbean landscape. This rare wooden image harnesses that environmental power into a particular zemí, a central figure in community ceremonies, including healing. To share this zemí beyond the walls of the Museum in a time of suffering keeps with the original intent of providing inspiration to all those who experience its beauty.

Follow the instructions below to view this magnificent sculpture in full 3D, accompanied by a guided, bilingual narration by Doyle—all from the comfort of your own home. And read on to learn how this 3D model was created.

See the Zemí in Your Space

Note: This AR model currently only works on iOS devices, and works best on an iPhone running iOS 13.4.1 or later. If you're unable to update your iOS, you can still experience the 3D artwork in AR, but without audio.

If you see this small cube icon on the top-right of the image below, that means your device is AR compatible. Tap the image below and your phone will give you instructions to place the model.

Navigation:

• You can move around the figure, shown at actual size. Tap it once to rotate, or pinch with two fingers to zoom in and out.

• Tap the English or Español button to begin the audio narration.

• Tap "X" to exit the window.

Tips:

• If you don't hear audio after updating, make sure your phone's silent mode is off.

• 3D models work best on opaque, well-lit, horizontal surfaces, such as a table or floor.

• Tap on the background to bring the camera button forward so you can capture video or a still image of your 3D experience.

• "Object" mode allows you to see the artwork against a plain background.

• USDZ files are shareable via iMessage, Mail, AirDrop, or other compatible third-party apps.

• AR models with audio can also be viewed on Mac OS Catalina (10.15.4 or later).

Modeling the Zemí Cohoba Stand

Photography

The Imaging Department's 3D modeling of the zemí began with photography. Using a standard DSLR, the artwork was photographed from hundreds of angles in a circuit of elevations using even illumination. Photogrammetry, the next step in the process, requires uniformly lit, overlapping images that depict the object at every angle.

Photogrammetry

Photogrammetry software tools detect points in overlapping images and use them to calculate depth. The result is a dense cloud of points mapped in 3D space.

The point cloud must be refined and cleaned, making the model's geometry accurate within a few thousandths of a centimeter. This refined model of connected 3D points is called a triangulated mesh.

In a process called texturing, the entire set of photographs was re-projected onto a 3D mesh, resulting in a file resembling the real-life sculpture. This large, textured source file was archived, and a derivative model, reduced through a process known as decimation, was made for distribution.

Optimization

The reduction and optimization of a multi-gigabyte 3D model for distribution was by far the most difficult stage in the workflow, and required constant refinement.

The original, highly detailed model was decreased in size while balancing surface texture and other visual elements that give it realistic detail. This required further decimation of the high-resolution mesh and re-applying optimized images to a vastly simplified model. The goal was to create a much smaller model while still conveying as much surface detail as possible, given the file size constraints.

The Met's archived zemí model was optimized from 2.43 gigabytes (or 20 million triangles) to a derivative that is 9.8 megabytes (4,500 triangles).

Refining Material Quality

3D digitization is accurate in terms of representing the artwork, but less so in reflecting how an artwork's surface reacts to light. The zemí's wooden surfaces and details, such as the shell teeth, were optimized using Adobe's Substance Painter tool to give the reduced model a more realistic viewing experience. Once completed, the model was exported as an open-format Universal Scene Description (USDZ) file.

Adding Audio and Interactivity

During this quarantine period, new audio capability for USDZ became available. Curator James Doyle's audio narration was recorded at home and embedded in the file, and interactive buttons were added using Apple's Reality Composer app.

Bring Imaging's technology preview of the zemí into your home by clicking the download image above, and enjoy!