One of the goals of the MediaLab is to explore the use of design and emerging technologies to improve the museum experience. Projection mapping, also known as spatial augmented reality, is a technology that can turn physical objects and buildings into a surface for projected light. This technique creates an enhanced experience for the audience by combining digital information with real objects. While the MediaLab was exploring how projection mapping could be used in the Museum, an opportunity arose to collaborate with the Department of Egyptian Art.
At that time, Maria Paula Saba and I joined the MediaLab as Creative Technology interns and collaborated with Erin Peters, who joined the Met as a Chester Dale Fellow to research The Temple of Dendur for her dissertation. Together we created Color The Temple, a tool that uses projected light to digitally restore color on The Temple of Dendur.
Did You Know The Temple Wasn't Always Beige?
The small square shows a cleaned surface on a temple in the Karnak Temple Complex. Image courtesy of the authors
Temples in Egypt, and in much of the ancient world, were not only carved with detailed reliefs, but also painted with vivid colors, like the example from the Karnak Temple Complex shown above. The small square shows a cleaned surface in an otherwise soot- and grime-covered relief scene. This small section at Karnak allows visitors to see the temple in new ways, and we set out to do this digitally with The Temple of Dendur.
The Temple of Dendur was originally located on an ancient site south of Aswan in the West Bank of the Nile, near the border between Egypt and the Sudan. Because the Nile flooded every year, the Egyptian government attempted to control the water through a series of dams. However, by the late 1920s, Dendur and the surrounding area was flooded for nine months out of the year. In the 1960s, the Egyptian government planned to construct a new dam that would have made this flooding permanent year-round.
The Temple was victim to flood waters, as can be seen in early photographs. George Gerster, Photo Researchers, Inc. Published in The Metropolitan Museum of Art Bulletin, Vol. 36, No. 1, Summer 1978, 18–19
In order to save Dendur and numerous other temples in the area, UNESCO initiated a salvage campaign in which Dendur was documented and dismantled. In 1965, the Arab Republic of Egypt offered Dendur to the United States in recognition of the assistance they provided during the campaign, and more than twenty American cities vied for the Temple in what the press called the "Dendur Derby."
Ultimately, Lyndon B. Johnson awarded the Temple to the Met in 1967. After being transported to the United States, the Temple was rebuilt in the newly constructed Sackler Wing and opened to the public on September 27, 1978. It has become one of the most beloved pieces at the Met.
Research on Colors and Patterns
To start the process of our digital recreation of the colors on The Temple of Dendur, we began by using a variety of sources—including the Temple itself, surveys of both Dendur and other temples, and objects from the Met's collection—to research the original colors of the Temple. Looking first to the Temple, we considered exploring several of the scenes. Ultimately, we focused on one scene in which the figures and glyphs were well preserved in the stone, because it would allow us to isolate various elements to tell the story of the scene. Logistically, we needed to be on the south wall, away from the direct sunlight coming through the windows, so that the natural light wouldn't wash out the projected light.
Scene on the south wall, in which Augustus presents offerings to the goddess Hathor and the god Horus. Image courtesy of the authors
We looked for remnants of color on this scene, both with the naked eye and a technique known as visible-induced luminescence (VIL) imaging, but, unfortunately and unsurprisingly, we found none. Although the arid desert is a good environment for preserving antiquities, sand and wind erosion over two thousand years, as well as flooding, deteriorated the surface. However, early reports regarding the Temple before it was flooded provided some key insights.
The pronaos of The Temple of Hathor at Dendera. Image courtesy of the authors
In his 1906 survey, Aylward M. Blackman recorded some of the visible paint in various scenes on the Temple's interior walls, and he even drew some diagrams for illustration. Although no paint was recorded anywhere on the exterior, the information on the interior provided a good starting point for the colors of our scene. Without concrete data, though, our results could only serve as a hypothetical recreation, with the purpose of giving a general idea of what the Temple might have looked like in antiquity.
We consulted nineteenth- and early twentieth-century surveys that reported on other temples as well, specifically the Napoleonic Description de l'Egypte, which included two temples illustrated with color. The two illustrated temples were the Temple of Isis at Philae and the Temple of Hathor at Dendera, both of which had recently been cleaned, revealing brilliant paint. The inner portion of the portico at the Temple of Hathor dates to the same early Roman period as Dendur, so it served, along with Blackman's record, as an example that helped us choose the colors for our digitally recreated projection.
We also looked at other objects in the Met's collection to get more information on the original color palette, including a painted column capital from the Temple of Amun at Hibis and Charles K. Wilkinson's Seth Slaying a Serpent, Temple of Amun at Hibis. Through conversations with Conservator Ann Heywood about her work on the capital, we learned there was originally a white gesso applied to the stone behind the colorful figures before paint, just like what is seen on the Temple of Hathor at Dendera.
Carrie Roberts, a former fellow of the Museum, shows us samples of Egyptian blue in order to compare our digital recreation with a surviving pigment. Image courtesy of the authors
Our research also revealed that certain objects in Egyptian culture, such as garments, bracelets, or crowns, were painted with highly decorative patterns. These detailed patterns, which included individual feathers or depictions of battle, were sometimes carved and then painted, like on the Temple of Kalabsha, and at other times, they were painted without being carved, like on the Temples of Kom Ombo and the Temple of Isis at Deir Shelwit. In our final digital recreation, we created versions with and without these patterns to show multiple possibilities.
The Temple of Kalabsha reveals carved patterns in the figures. Image courtesy of the authors.
An illustration of Dendera's eastern gate from Sylvie Cauville's Le Temple de Dendara 11: La Porte d'Isis
Making It Digital
The first step in the process of projecting colors onto the Temple was to create a digital version of the scene we chose. We used high-resolution photographs of the south wall, where our chosen scene was located, as a reference for our recreation. Our goal was to create a type of image that we could use to easily switch the colors of different parts of the scene, so we decided to make a vector-based image instead of a raster, pixel-based image because this type of image would allow us to easily change the style of each shape independently. We imported the photographs in Adobe Illustrator and added a layer to the image so that we could trace the original lines of the scene.
The process of making vector-based images on top of photographs.
The final result. Images courtesy of the authors
One of the benefits of our project was having access to the Temple as the primary resource for research. Once we started creating vector images and line drawings, it became apparent that the original carved lines of some of the hieroglyphs and figures were subject to erosion and therefore not very visible. We didn't want to simply copy and paste from what past scholars thought the Temple's carving should look like. Instead, we compared our line drawings and previously published versions of the hieroglyphs and figures with the stone itself, and came up with a version that combined that information.
Once we had an initial version of our line drawings as Illustrator files, our next goal was to get the basic projection-mapping workflow in place. We opened the drawings in openFrameworks as SVG files, which allowed us to easily change the outline, fill the shapes with color, and draw the shapes in our application window. We were then ready to send them to a projector and get them to align with the physical carvings.
We used MadMapper to align our drawing with the Temple because it allowed us to modify the graphics on screen such that they would appear correct if the projector was not facing the Temple straight on, or if we needed to make small adjustments to align with the carvings on the irregular surface of the stone. We bridged openFrameworks and MadMapper with Syphon, a plug-in that enables image sharing across applications in real time.
Until we had access to test our projection on the actual Temple, we printed out photographs of the Temple's scene at nearly full scale and taped them to the wall to practice projecting. MadMapper makes it pretty simple to get the alignment correct, so after this first test, we were ready to start tackling the issue of getting the colors to match our best guess of their original appearance.
We had to return to our drawing when we discovered that there were patterns in the background of the scene. Using documentation of Dendera's eastern gate as resource, we cut reference images and placed them on top of our drawing. We realized that vectorizing every shape of a pattern would take longer than painting the patterns as bitmap images. Since we already had the final selection of colors, we decided to open the first version of the scene, which featured solid colors, in Adobe Photoshop in order to colorize the patterns with more accuracy and agility.
The process of adding patterns to the safe image. Images courtesy of the authors
Making It Interactive
Projecting our digital recreation onto the Temple worked great, so we wanted to take advantage of this success by experimenting with different media. Technology opens up many possibilities, and we were dealing with the best medium for video, so we asked ourselves: How can animated scenes add to the process of learning about the Temple? Using our solid-color and patterned images as a base, we developed a series of animations that explored elements of emphasis and storytelling.
In the scene on the Temple, Emperor Caeser Augustus, depicted as Pharaoh, arrives and offers wine to the deities Hathor and Horus, so we created an animated version of this. In the original scene, the glyphs represent the dialogue between the emperor and the deities, so we highlighted the glyphs and made them larger. We also used the animation to explain how seemingly flat figures in Egyptian art actually represent 3D scenes, like in the case shown below, where Hathor seems to be behind Horus, but in reality they were side by side.
Left: The highlighted glyphs that represent the dialogue between the figures in the scene. Right: The animated Emperor Caeser Augustus arriving at the scene. Images courtesy of the authors
We knew that presenters using Color The Temple would need something to control what is highlighted or shown during the animation, so we thought the natural interaction would be to have a mobile controller. This led us to design an application with an interface that would trigger different scenes at the appropriate moment.
The remote controller that the presenters used to control the animations. Image courtesy of the authors
We connected an openFrameworks application to our interface controller, which was just a web page, using Spacebrew, an open toolkit to connect interactive platforms. Our interface featured sequenced buttons (which was decided based on the presenter's discourse) that the presenter could use during their presentation.
Evenings in the Temple
On-site tests were crucial to the success of the project, so we visited the Temple on several evenings. We even started our visits before we had the final results from our research so that we could test different equipment and tools throughout the process.
Former MediaLab Intern Matt Felsen adjusting the projector on top of the scene. Image courtesy of the authors
The colors slightly changed when projected onto sandstone instead of a white wall, and because of this, Erin took her research sessions to the Temple. We compared what the colors looked like in facsimiles and field photos to our projection on the Temple to try to get as close of a color match as possible. The ambient light also changed the way we perceived the final output, so we often tweaked colors in real time until we got the correct match.
The final result projected onto the Temple. Image courtesy of the authors
When we finished the project, Erin presented Color The Temple a few times to prove that the tool provided an enhanced way to present to a broader audience. Guests included illustrious people like Philippe de Montebello, the former director of the Met for over thirty years, current Director Thomas Campbell, and Associate Director Carrie Rebora Barratt.
We presented Color The Temple to the public for the first time during Teens Take the Met on October 16, 2015, followed by presentations to a large crowd during the Apollo Circle Supper Club and other private events. The project was well received by the public, which confirmed that there's an interest in learning from innovative, interactive experiences like projection mapping.
Watch a timelapse video of Color the Temple
What's Next
Through rigorous research, prototyping, discussion, and iteration, we have managed to cast new light on the Temple by presenting it in a fashion much closer to its original form for the first time in many millennia. Using relatively recent advances in software, we were able to experiment with restoration using nondestructive means (projected light rather than a material like paint) to temporarily display content without presenting any challenges for conservation.
It is our hope that this project will serve as the starting point for future discussion and work. There are several possibilities for future research—including expanding the projection area to include additional scenes beyond the one selected for our project, providing additional pedagogical layers to the existing content, and creating opportunities to present the work to broader audiences.
Teaching Museum visitors about The Temple of Dendur through projected light and animations opened up new avenues of inquiry, brought about new ways of thinking, and made us look for answers outside of traditional research methods. This project has been an engaging, collaborative experience, bringing together expertise in technology and academic scholarship in a way only possible in the MediaLab.
