How Surveys and Focus Groups Can Be Used Together to Evaluate Professional Development

Surveys for Data Collection

We’re all familiar with surveys. We use them whenever we seek information from groups of people, and they are our go-to data collection strategy for evaluating professional development. We may call them questionnaires or feedback forms, but the purpose remains the same – for leaders to gain insight into how educators are experiencing and learning from professional development, changing teaching practices in the classroom and impacting student learning.

But surveys don’t have to operate alone! In fact, to gain the deepest insight, you should use multiple data sources to evaluate professional learning.

Focus Groups for Data Collection

Using focus groups to collect data is an excellent strategy you can couple with surveys. Focus groups typically involve a small group of people (around 5-8 is best) who come together to participate in a discussion about a topic – in this case, professional learning. The discussion becomes your data; hence this is a qualitative method.

Unlike surveys, focus groups use mostly open-ended questions designed to elicit rich dialogue among participants, and participants are encouraged to respond to each other as well as to the moderator – the person who facilitates the focus group.

Focus groups have several key advantages:

• They allow you to interview multiple people in one session.
• You can ask focus group participants probing questions such as “Can you tell me more about that?” or “What exactly do you mean when you say …?”
• Having people come together for a focus group means that they can build off of each other’s responses, and spark each other’s memories.
• Because you’re meeting in person, you see people’s facial expressions and hear their tone of voice as they’re responding to questions (and that is additional data!).

Of course, as with any data collection method, there are also some potential challenges:

• Logistics are required to plan for a space and time for everyone to meet.
• While not required, it is a nice gesture to offer focus group participants light refreshments, or even a meal if the time slot crosses a meal time, so there can be some cost involved.
• It’s best to have a notetaker in addition to a moderator – someone who can skillfully script the conversation as it happens (although you can audio or video record).
• Analysis can be a bit tricky and time-consuming.

Which Data Collection Method to Use When

Before you decide if, how and when to use focus groups and surveys, you must articulate clear evaluation questions that will drive your selection of data collection methods. These are your “big” questions – your research questions, if you will. These are not the same as the questions you would ask individuals after they have participated in professional learning. You won’t know whether a survey, focus group or any other data collection strategy will help you answer your big evaluation questions until you’re absolutely clear on what the questions are.

Once you have your evaluation questions ready, then consider:

• Do you know what questions you would need to ask of individuals who have participated in the professional learning that will help you answer your evaluation questions? (In other words, do you have a set of survey questions in mind?)
• Do you need data from a large group of people (more people than you could feasibly talk with if you were to call or visit them individually)?
• Will a set of closed-ended questions (e.g., multiple choice, rating questions) give you all the insight you need to answer your evaluation questions?
• Is there detailed information about attitudes, perceptions, learning, or teaching practice that would be easier for people to describe to you in person than to write down in a few short sentences or paragraphs?

A Surprising Twist

You may be thinking that you would naturally use a focus group after you collect data with a survey to have a selection of respondents elaborate on some key points. This is a powerful strategy and very common in evaluation.

Perhaps you look at your survey data and find that a majority of respondents enjoyed the professional learning, but are not putting new ideas into practice in classrooms. Or, respondents indicated that they need more professional learning on a topic, but you have several options for programming that you can put into place but don’t know which might work best for teachers. Use focus groups to dig deeper and explore these survey findings.

But did you know that you can also use focus groups before administering a survey to inform the survey questions? Let’s say you offered professional learning on a new topic, or in a new format, or on a larger scale than ever before. Perhaps you’re not quite sure which survey questions would best help you answer your overarching evaluation questions. You can use a focus group to help you develop a survey!

In this case, you might ask your focus group participants very general questions about how they experienced the professional learning and then just listen. You will get a sense of what is on their minds and can use this understanding to develop a specific set of (mostly) closed-ended questions for a survey to administer to all professional learning participants.

Focus group participants, especially if they are either members of the respondent population or closely connected to the topic of study, can provide greater understanding of a topic and lead researchers to explore aspects of a topic that might otherwise be overlooked (Robinson & Leonard, 2019, p. 78).

Want to Learn More?

For more about evaluating professional learning check out my eBook, a free Frontline Education resource. To learn more about conducting focus groups, download my free guide, How to Conduct Focus Groups for Data Collection. For an in-depth look at how to design effective surveys, see Designing Quality Survey Questions.