We are also interested in sharing our data with a larger network of like museums. We are an art museum. Would love to learn more about COVES and see the model applied more broadly. Happy to discuss/partner with anyone who is interested!
Original Message:
Sent: 06-25-2015 04:15 PM
From: Ellen Roth
Subject: Institutional Benchmarking through Audience Research
Hi Eric and Ryan,
We have a fairly well established visitor demographic survey that we currently do and have been doing for many years. This study is a little bit different from ours and we are definitely wary of the kiosks!! We are also a part of the Denver Evaluation Network, which has had a lot of success with collaborative surveys, but without any comparable institutions as far as content. I love this COVES model and we will definitely be looking in to that! Thanks for forwarding the message to Tania. My email address is ellen.roth@dmns.org if that helps for continuing the conversation.
Ellen
Original Message:
Sent: 06-25-2015 11:58 AM
From: Ryan Auster
Subject: Institutional Benchmarking through Audience Research
Hi Ellen,
As you note - and as Eric has already mentioned (thanks Eric!) - there is definitely interest in this type of network, and we've already started the work to build it. COVES (Collaboration for Ongoing Visitor Experience Studies) is designed to do a lot of what you ask about, including gathering data systematically from "like institutions" so that the data are comparable for benchmarking purposes. (Note: these "like" institutions are currently science centers, but vary in size, annual budget, geography, etc. We do plan to expand to other types of organizations once the system becomes self-sustaining.) The system is also designed to build evaluation capacity where it is lacking by leveraging evaluation capacity that exists elsewhere within our field - capitalizing on a strong collaborative approach to gathering visitor-level data. While the Museum of Science has helped to shepherd the work forward, this truly is a field-wide effort: we are partnering closely with ASTC to consider field-wide needs, have an incredible group of individuals from 6 other institutions to help guide the work (including Tania at NYSCI!), and we'll begin piloting this fall with 9 science centers across the country.
Similar to Eric's comments below, I would caution against contracting with an external agency looking to provide kiosks for data collection. While they may have a more sound methodology than initially meets the eye, you've picked up on at least one of the big setbacks you'll be facing - finding institutions worthy of being included in your group for comparison. This is initially what drove us to consider creating a collaboration for this type of data collection effort, and what seems to be a very enticing aspect of our work as we begin to spread the word.
We are working hard to ensure that many of the complications Eric notes below are taken into consideration ahead of time: creating instruments and methodologies that provide valid and reliable data, instituting data protection policies for participating institutions, considering a range of questions that are relevant to individual organizations and the broader field, and providing points of comparison that are truly worthwhile (e.g., not only comparing institution to institution, but institution to local census data). Our hope is that this system not only meets the needs of participating institutions initially, but also responds and adapts to changing needs, to the extent that the data we collect can be used for decision-making purposes.
We are in the process of building a website where anyone interested can learn more about us and begin to get involved (although it's currently not live), but members of our group will be at next month's VSA Conference and can share more information there, or at the ASTC Annual Meeting as well. We also posted a White Paper summary of the work leading up to this effort: Creating a Collaboration for Ongoing Visitor Experience Studies (C-COVES) White Paper.
| InformalScience |
remove preview |
|
| Creating a Collaboration for Ongoing Visitor Experience Studies (C-COVES) White Paper |
| This White Paper summarizes the work of C-COVES, a two-year IMLS-funded project designed to Create a Collaboration for Ongoing Visitor Experience Studies. Specifically, C-COVES was intended to research the feasibility of creating a multi-institutional network of science centers across the country united in studying the visitor experience within and across organizations nationwide. |
| View this on InformalScience > |
|
|
Please feel free to contact me directly if you won't be at either or if you'd like additional information before then. I hope this is somewhat helpful - good luck!
------------------------------
Ryan Auster
Senior Research Associate
Museum of Science
Boston MA
------------------------------
Original Message:
Sent: 06-25-2015 07:40 AM
From: Eric Siegel
Subject: Institutional Benchmarking through Audience Research
Hello Ellen:
If the group that is "hard selling" you this service doesn't have comparable clients, then I would approach it with caution. Furthermore, I am a bit skeptical of the idea of collecting demographic and other info through kiosks, as the users would be self-selecting. It would be better to do random surveys if you want statistically accurate information. I just found out about a year old project in which our institution is participating called COVES. Led by the Museum of Science in Boston, with several science museums as participants, it sounds like a very thoughtful attempt to develop the kind of community of shared information you are describing. The person at NYSCI who is leading this work is tania tiburcio, so you could reach out to her for more info.
Also, if you don't know about the Visitor Studies Association, they are a very active group of museum researchers that meets and publishes regularly.
In my experience, creating and sharing comparable data among different museums is more complicated that it sounds, as institutions have different needs, different communities, different audience demographics in terms of age, income, ethnicity, education, and different aspirations.
As I say the COVES group seems to be making a thoughtful effort in this direction from what I have heard. I have forwarded your note to Tania.
Good luck!
------------------------------
Eric Siegel
Director & Chief Content Officer
New York Hall of Science
Queens NY
------------------------------
Original Message:
Sent: 06-24-2015 02:16 PM
From: Ellen Roth
Subject: Institutional Benchmarking through Audience Research
Greetings from the Denver Museum of Nature & Science!
We are writing from the Department of Audience Insights within the museum and had a few questions about institutional benchmarking. We were recently approached by a market research group to install kiosks containing a basic demographics (age, race, gender)/reason for visit/satisfaction type survey. The hard sell is that if we participate, we will be benchmarked against other "like institutions" however, one of the problems we are facing is that there are few of those like institutions participating, so we're unsure if the benchmarking would be particularly useful.
My question is: Is anyone else using a different method of benchmarking for this same kind of data? If not, is there any interest in starting this kind of network?
Thanks for your input!
Ellen
------------------------------
Ellen Roth
Evaluation Project Manager
Denver Museum of Nature & Science
Denver, CO