Visiting Sleeping Beauties: Reawakening Fashion? You must join the virtual exhibition queue when you arrive. If capacity has been reached for the day, the queue will close early.

Learn more

New Perspectives on Storytelling, Artificial Intelligence, and Art

Kurt Behrendt
February 11, 2019
Workshop participants at a table

The author meets with other participants in the Storyteller group. From left to right: Emmanuel Awa, Kurt Behrendt, Lara Baladi, Matt Zumwalt. All photos by Natasha Moustache / Listen

Making the artworks in The Met collection meaningful for the public is one of my main goals as a curator, although it usually takes the form of exhibitions, publications, or lectures. When I go to exhibitions with family and friends, I find that viewing artworks with others leads to great discussions about how we can see the world in new and unexpected ways. So when I was invited to participate in a Met x MIT x Microsoft hackathon with the goal of leveraging artificial intelligence (AI) to interface with The Met's Open Access online collection of more than 406,000 images, I accepted, eager to see how we can creatively leverage digital and web-based technology to inspire and support new kinds of conversations and interactions with art. With this in mind, I headed off to Cambridge, MA, together with other curators and digital colleagues from The Met, to meet with engineers, researchers, students, designers, artists, and technologists from Microsoft and MIT.

Two views of workshop participants conversing

Left: Hackathon participants conversing on the first day (from left to right: Samaneh Kazemi Nafchi, Nina Diamond, Julie Arslanoglu, Griff Mann, Matthew Ritchie). Right: The Storyteller group gathers around a table to brainstorm (from left to right: Emmanuel Awa, Kurt Behrendt, Lara Baladi, Matt Zumwalt, Azra Akšamija, Griff Mann)

On the first evening, we all gathered over dinner to discuss how AI might be used in innovative ways. The plan was to break up into small groups and over the following two days build something useful. As I was getting ready for bed, I got an idea. I remembered a game we play in my family, where we pass a story from one person to the next. Each person improvises, and the story develops in unexpected ways. What would happen if we told stories like that while AI was listening in using voice recognition? As our personal narratives unfolded, the AI could reach into the massive data set of The Met collection and stream back relevant artworks. Drawing artworks from all periods across the world seemed like an inspirational, whimsical way to encounter The Met collection. Most importantly, it would be a way to interact with the collection that would be both personal for the storyteller and meaningful for the group listening. Instead of involving individuals in a solitary, inward-looking interaction with a tool on the web, it would be a platform that invited interaction and conversation. So late that night I drafted a short statement and pitched it as an idea the next morning when we all gathered to begin work.

Curator Griffith Mann talking with Azra Aksamija in front of a painting

Curator Griff Mann talks to Azra Akšamija, an artist and architectural historian, with Matt Zumwalt looking on. Behind them is the reproduction of a painting by Peter Paul Rubens from The Met collection

The flexibility of this approach appealed to a range of people from Microsoft, MIT, and The Met. Joining my group, they brought a host of new ideas, technical approaches, and possible outputs. We decided to call our project "Storyteller," and began brainstorming how it might be used in innovative ways. Storyteller could offer people the opportunity to create and preserve oral histories complemented by visually rich imagery from The Met collection. Since the tool would be driven by the spoken word, translation software could open up the platform to people speaking over sixty languages. Voice recognition technology could unlock the door for people with disabilities, and storytellers could dig into The Met collection to learn more about the artworks they discovered through their personal narratives. The AI could even generate a tour of the Museum so that a group of people contributing to or listening to a story could come to the Museum to see their artworks on display. Another output would assemble all of the relevant information about the artworks and produce a print-on-demand catalogue, a coffee-table memento of one's own personal story. Last but not least, the opportunity to share one's own story or to access someone else's story through social media clearly had tremendous potential; it could, for example, make the oral histories mentioned above freely available. Audiences could also enjoy collections of works created by different voices: a scholar, a curator, or perhaps a favorite YouTube star.

Three images: two bearded young men working on laptops; woman in a pink and black striped dress writing on an easel; black and white photo of the author speaking to a group

Clockwise from upper left: Azra Akšamija designs on a whiteboard. Johann Diedrick, a senior developer at The Met, and Chris Stone, senior software engineer at Microsoft, work on their laptops. Kurt Behrendt, the author, shares the Storyteller protoype as the hackathon concludes

I was fascinated to see how our group succeeded in just a couple of days in making this idea a technological reality. Superficially linking voice recognition software to the collection was straightforward, but figuring out how best to reach into the database to get relevant results was an issue fraught with complexity. I found myself especially interested in the question of how artificial intelligence could be made to understand an image and relate it to a story—a topic that extends far beyond the scope of our initial project and where it stands today as a prototype. Equally fascinating was seeing how the graphic website interface of the Storytelling project could be organized to allow someone to use the platform intuitively to create something that was truly a creative, personal expression.

An image from an event poster showing a painting of young farmers linking arms with the logos of the participating institutions above in black on white

Poster displayed behind the podium at the hackathon, showing the logos of the three participating institutions above a painting by Winslow Homer in The Met collection

On February 4, participants in the Met x Microsoft x MIT hackathon presented the five prototypes in the Great Hall of the Met Fifth Avenue. Looking back, I realized this project has changed how I think of artworks across the Museum. These diverse objects are so much more than simple traces of a distant past. Just thinking about how they might fit into a modern narrative transforms their meaning and the contexts that I might imagine for them. The Met truly offers us the possibility of bringing artworks to life and seeing the world from new perspectives.

Storyteller Project Participants

Azra Akšamija, Associate Professor, MIT Architecture, Future Heritage Lab, MIT
Lara Baladi, Lecturer, Art, Culture, and Technology Program, Ida Ely Rubin Artist in Residence, Centre for Art, Science & Technology, MIT
Kurt Behrendt, Associate Curator, Department of Asian Art, The Met
Johann Diedrick, Senior Developer, Digital Department, The Met
Griff Mann, Michel David-Weill Curator in Charge, Department of Medieval Art and The Cloisters, The Met
Emile Molin, Head of Design, The Met
Chris Stone, AI CAT, Software Engineer, Microsoft
Matt Zumwalt, Researcher and Reporter, Protocol Labs – Friend of MIT

Banner for The Met x Microsoft x MIT hackathon

The Met, Microsoft, and MIT held a two-day hackathon session to explore how artificial intelligence could connect people to art. Learn about this collaboration and the prototypes it generated.

Kurt Behrendt

Kurt Behrendt has been at The Met since 2006. He has curated a series of exhibitions, including Bodhisattvas of Wisdom, Compassion and Power (2021), Seeing the Divine: Pahari Painting of North India (2018), Tibet and India (2014), and Buddhism along the Silk Road (2012), and has published widely on the Buddhist art of the Indian subcontinent. Ongoing field research in India, Pakistan, Nepal, and Tibet provides a foundation for his exhibitions and publications. His 1997 PhD from UCLA was published as the book The Buddhist Architecture of Gandhara (Leiden: Brill, 2004).

Selected publications

Behrendt, Kurt. “Architectural Evidence for the Gandharan Tradition after the Third Century.” In Problems of Chronology in Gandharan Art: Proceeding of the First International Workshop of the Gandhara Connections Project, edited by Wannaporn Rienjang and Peter Stewart, 149-64. Oxford: Archaeopress Publishing Ltd., 2018.

———. “Buddhist Sculptures of Bihiar and Orissa from the 9th to 12th centuries: the Geographic Distribution of Iconographic types,” In Across South Asia: A Volume in Honor of Professor Robert L. Brown. edited by Rob Decaroli and Paul Lavy, New Delhi: D. K. Printworld, 2020.

MetPublications: Selected publications by Kurt Behrendt