A visitor using the Audio Guide. Photograph by Jackie Neale
«The Audio Guide is a long-standing service at the Museum with over three thousand audio messages attracting 250,000 users annually. It offers six to eight special exhibition tours and can be accessed in up to nine major foreign languages. In September 2013, the Museum launched a new version of the Audio Guide complete with a redesigned interface and repackaged content. This presented a fresh opportunity for us to take a more strategic look at the Audio Guide and see how well it has performed since its rebirth.»
In the summer of 2014, I worked with a team of consultants from Frankly, Green + Webb to conduct a comprehensive evaluation of the Audio Guide here at the Met (both at the Main Building and at The Cloisters). Below, I will explore some of our key findings and explain how the Museum is moving forward with the results of this research.
The Audio Guide as a Service
A Museum employee working at one of our Audio Guide stations. Image courtesy of the author
It's always tempting to think of an Audio Guide as an old-fashioned product that visitors can hold onto and carry with them throughout the Museum if they feel like it, but this perspective is simply too limited. Instead, the Audio Guide should be seen as a service that is a crucial part of the museum experience and one that combines both the digital and nondigital within the physical space of the Museum. This perspective, popularly known as "service design," is a way of creating and improving experiences (instead of stand-alone products) in order to better meet visitor needs within a specific context. It requires an understanding of the user's full experience (also known as the "user journey") in a robust way—from initial awareness of the offer to the full impact of the offer.
How We Got Our Data
We conducted the evaluation across a seven-day period. We created and used a wide range of methodologies in order to get a well-rounded picture of the Audio Guide service. This included a quantitative survey administered to 492 randomly selected visitors, a qualitative interview conducted with twenty prerecruited visitors, and guided usability testing. Below, I'll explain a few key insights that we gained during this process.
Who is the Audio Guide Serving?
Ideally, the Met would like our Audio Guide to be a service that appeals to all visitors from all walks of life, but this simply isn't the case; no service can be all things to all people. Therefore, it was extremely important for us to better understand who the Audio Guide was appealing (or not appealing) to. It was about time we met our users face to face!
A pie chart showing the percentage of visitors that are domestic and international tourists compared to local. Image courtesy of the author
The research has helped us more clearly identify two broad audiences currently using the Audio Guide at the Museum, each with their own unique set of needs. The first, or primary, audience group is the domestic and international tourist. This group makes up 81% of Audio Guide users and 40% of the visitors in this group are non-English speakers. Most of them are first-time visitors who are attracted to the Museum due to its impressive reputation as a must-see destination in New York City, and they are looking to use the Audio Guide as a way to help them navigate the Museum both conceptually and physically.
What we've found is that the Audio Guide's relatively small foreign-language offer has a negative impact on this particular subgroup of visitors. Where 90% of English-speaking visitors described their Audio Guide experience as "good" or "very good," only 73% of non-English speakers felt the same way. It turned out that our first-time foreign-language users often feel "underserved" because the majority of our Audio Guide is only available in English.
A bar graph depicting the percentage of Audio Guide users using it in languages other than English. Image courtesy of the author
In order to increase the value of the Audio Guide to this group, we are significantly expanding the foreign language offer. By the summer of 2015, we will have translated three hundred audio stops into nine languages: that's 2700 new stops. We're hoping to continue this pace in the coming year and onwards. We're also using data from our foreign-language analytics to be strategic about which stops we translate and when.
The second audience group is New York City and tristate-area residents, considered the Museum's "local" audience. They make up 19% of all Audio Guide users, are likely to be repeat visitors or Members and are a smaller, generally more diverse audience overall. As a result, we don't actually know very much about the specific needs and identity of this group. The most important thing we do know is that they primarily take the Audio Guide for special exhibitions. They think of the Audio Guide as being something that is for tourists and have trouble seeing its value because they feel that it doesn't fit their own interests and needs.
To address this, we're targeting our local audience where they already are in the Museum—special exhibitions—and promoting the availability of our newly launched web version of the Audio Guide in those spaces. This version of the Audio Guide, though still in beta, is free and accessible on personal devices and it gives us an opportunity to test whether or not our local audience will find this offer more appealing than the rentable device. This is just a starting point and it will hopefully help us better understand their needs as a group, which means we'll be able to provide a better user experience in the future.
Moving forward, we will implement new measures to improve our Audio Guide service in order to attract more local repeat visitors as well as expand the experience for our foreign-language speakers, who currently make up the largest "natural" market for the Audio Guide.
Is the Audio Guide Easy to Understand and Use?
Two young visitors using the Audio Guide. Photograph by Jackie Neale
The evaluation has shown us that our Audio Guide offer is big and ambitious, which can be a good thing, but what we've found is that this also creates a level of complexity that makes it difficult for visitors to make informed choices about their experience. For example, it is hard for staff members to give a clear and concise description of the Audio Guide to visitors because there is so much to go over and explain. As a result, many visitors fail to understand the true scope of what our Audio Guide has to offer and they can't take full advantage of it. This confusion was most clearly shown during our usability testing. Although 76% of our visitors stated that they found the Audio Guide "very easy" to use, much of the survey data indicates that the vast majority of visitors were only able to access a small portion of what it had to offer. This drove us to dive deeper into the reasons users were having difficulty getting the full experience of the Audio Guide.
So how do you reduce the confusion while keeping the depth and richness of a program that visitors value? According to the research, we've found that the answer is this: consolidate what is offered and shorten the learning curve for the visitor. This has many facets and the Met has started handling it in the following ways. First, we're making changes to the Audio Guide itself. We're reducing the number of options and rearranging the way tours are grouped and created so that they clearly reflect visitor needs. Second, we're making changes to the way the Audio Guide is represented in the physical space of the Museum by simplifying the numbering system and reducing the visual clutter. Our goal is to have one number on one label that links all of the content related to that artwork. Signage, logos, and other supplementary materials are being updated, condensed, and redesigned so that the information they supply also reflects visitor needs. We've also changed the placement of Audio Guide distribution points so that they mirror the most common visitor paths through the Museum.
Does the Audio Guide Deliver Value?
Visitors gaining a meaningful understanding of the artwork in front of them by using the Audio Guide. Photograph by Jackie Neale
Overwhelmingly, yes! Visitors see the Audio Guide as a good learning experience. They expect the Audio Guide to support them in learning new information about the artworks in the Museum and help them understand what they are seeing in a closer, more meaningful way. Visitors also said that the Audio Guide is more valuable to them when it is easy and fun to use.
The fact remains that the impact of our Audio Guide is no doubt far-reaching and deep. Almost all of the visitors who used the Audio Guide during their visit reported spending more time in front of the artworks and gaining a deeper understanding of what the artworks mean. The Audio Guide has shown itself to be the perfect conduit for conveying this deeper sort of engagement by delivering stories full of emotion, excitement, and character; it helps to foster a more intimate connection between the Museum and the visitor by becoming a unique voice for the institution. The value of the Audio Guide lies in its ability to excite, enthuse, and engross visitors during their visit.
A bar graph showing the percentage of Audio Guide users and nonusers who had a "better than expected" experience at the Museum. Image courtesy of the author
Overall, we were pleased to find that the Audio Guide has a very real and measurable impact on our visitors. A large majority of visitors were very satisfied with their experience of the Audio Guide, stating that it allowed them to look more closely at the art on view, which helped them foster a deeper sense of connection and engagement with what they were seeing. More than eight out of ten visitors felt this way! In fact, visitors who used the Audio Guide during their visit were exponentially more likely to report a better overall experience at the Museum compared to those who chose not to use it.
What's Next for the Audio Guide?
This evaluation helped the Museum get a better picture of our Audio Guide as an important part of the larger user journey. It also served as a reminder that our audiences are not static; they are always evolving. Now, the Museum is developing several new strategic approaches to refining the Audio Guide offer and adjusting the service.
This will require a lot of hard work from our Digital Media team and a huge investment on the part of the Museum. However, we are willing to buckle down and move forward because we know the end result will have an enormous positive influence on our visitors in the long run, and in the end, our visitors are what the Audio Guide is all about.
For more information, please view the full text of the paper: "A New Look at an Old Friend: Reevaluating the Met's Audio Guide Service" published by Museums and the Web 2015.