Selecting Measures for Change Ideas

Baseball great Yogi Berra famously warned, “You’ve got to be very careful if you don’t know where you’re going, because you might not get there.” Yogi’s words are as relevant to implementing and measuring change ideas as they are to baseball. They remind us that we need to answer some essential questions before we can implement and before we can decide how to measure what we have implemented.

And the most important question is this: After you have selected a change idea, how will you know if it worked?

Planning change ideas of course starts with understanding what the current issue is and what we believe the root cause of that issue to be. Second, we need to understand the desired performance, skill, or outcome we want students to achieve. Third, we identify a change idea, which is part of a theory of addressing the challenge we identified.

And then there is the final question: How will you know? How will you know if students have gotten to the place you want them to get to? How will you know if students have gotten better at the key skill or performance you have identified? This is where measurement comes in. After you have defined a problem, outcome, and change idea, then we can select a couple of ways to measure whether or not students are closer to that outcome.

In our case, we are looking to see if our change ideas are impacting proficiency with our identified skill, as well as engagement as an assumed precursor to that improved proficiency. And in all cases, we need to be able to compare: Compare students to their previous performance or to compare our students getting the change idea with peers not currently participating in the change idea (Although they could participate if we determine it works!).

The table below illustrates how we might use the words in our guiding questions to select measures.

* Claim, Evidence, Reasoning rubric on the parts of persuasive writing

As the table suggests, measures of engagement can include simply counting students who engage in a behavior or complete an assignment. As Yogi said, “You can observe a lot by just watching.” But make it easy on yourself by using clear, simple categories or checklists (e.g., Did/did not complete). Other simple engagement measures can be three-question surveys with four or five-point agreement scales (I like four points for simpler analysis), asking students to reflect on their sense of connection or confidence. For proficiency measures, teachers have been doing those since forever. They key is to give yourself something similar enough to the original assessment (the “before”) to be able to assess progress (the “after”).

As you craft your measures and match them to your change ideas, then, consider these key principles:

1. Make them simple: Straightforward measures make it more likely that you’ll do them a few times in a cycle.
2. Make them consistent: The measure needs to be similar enough at each point in time that it makes a suitable comparison. A short writing assignment using the same rubric or the same, 3-question survey gets at this idea.
3. Repeat them: Administer each one twice, at a minimum: At the start and end of the cycle, so you can compare where the students are. No need to measure more than weekly for most change ideas.

At the end of a cycle, simple, common, repeated measures will offer you a great set of discussion points with your team to see what is working, for whom, and under what conditions. With this information, we know what practices to adopt, adapt, or abandon. Simple, comparative measures can help us make those decisions.