In and of itself, data does not improve school outcomes. Improvement happens when schools collect the right data, interpret it carefully, and use it to guide informed decisions and action.
The importance of getting the right data
Too often, schools collect data because it is available, not because it addresses a clearly identified issue or problem of practice.
Schools must move beyond ad hoc and indiscriminate data collection and instead adopt a structured process for identifying what information is genuinely useful for improvement.
Good data and good decision making begin well before surveys are administered or spreadsheets are created. They begin with disciplined thinking about the landscape that matters, the constructs that influence valued outcomes, and the kinds of evidence that best explain these outcomes.
In this article I want to focus on a part of the data collection process that typically does not get enough attention: the intellectual muscle that needs to be invested before the data are collected.
Educational problems are typically complex and multifaceted. Schools need real clarity before they gather evidence.
Deciding what data to collect
Good data collection demands careful thought about concepts, priorities, explanatory frameworks, and intended uses. This involves asking important questions such as:
- What aspect of learning or wellbeing are we trying to understand?
- What factors are most likely to influence this outcome?
- What conceptual or guiding framework best explains these factors?
- What indicators validly represent the constructs we are targeting?
- Which stakeholders can provide the most meaningful evidence?
Here I will describe an 8-step process designed to answer these questions – shown in Figure 1.
Rather than collecting everything possible, this process narrows attention towards data that is conceptually sensible, practically useful, and strategically relevant.
Importantly, if a school has already collected data, the 8-step process can still be used retrospectively to distil key information for coherent analysis and decision-making.
In fact, schools do not always need more information; sometimes they need better frameworks for interpreting what they already have.
Figure 1. 8-Step Framework for Data Collection
8 Steps to Data Collection
This 8-step framework helps make sense of the many moving parts in school data and decision making.
It is also aimed at helping educators avoid common pitfalls such as collecting excessive data, using measures not well aligned to school priorities, or analyzing information without focus or clarity.
1. Identify the landscape that matters
Schools should first determine the broad area of concern or interest. In my world (as an educational psychologist), a key landscape is student wellbeing – and in particular academic wellbeing. This will be the focus for the article, but equally it could be social-emotional wellbeing.
2. Identify target domains on this landscape
Once the broad landscape is identified, we need to identify the key domains on this landscape. For example, three major domains of academic wellbeing are motivation, engagement, and learning.
3. Identify an explanatory, organizing, or theoretical framework
When the domains are decided, it is time to identify a conceptual or theoretical lens that brings clarity, specificity, and order to the selected domains. For motivation and engagement, I use the Motivation and Engagement Wheel. For learning, I use Load Reduction Instruction.
4. Identify specific indicators under the framework
Conceptual or theoretical frameworks are typically multidimensional, representing the key indicators in a given domain. Indeed, this is one of the most helpful features of a good conceptual lens – it unpacks a domain into its specific indicators, that can then be the focus of data collection. As Kurt Lewin famously observed, “There is nothing more practical than a good theory.”
The Motivation and Engagement Wheel comprises 11 specific indicators of motivation and engagement illustrated in Figure 2.
Figure 2. The Motivation and Engagement Wheel, reproduced with permission from Lifelong Achievement Group.
Load Reduction Instruction unpacks the 5 specific principles of effective teaching and learning shown in Figure 3.
Figure 3. Load Reduction Instruction, reproduced with permission from Martin (2016).
5. Identify measures of specific indicators
For many conceptual frameworks, there are established and validated instruments available that can be used to collect data.
For the Motivation and Engagement Wheel, there is the accompanying Motivation and Engagement Scale (MES). The MES has a primary school, high school, and university/college version. The MES comprises 44 questions completed by students – 4 questions for each of the 11 parts of the Wheel.
For example, on a scale of 1 (Strongly Disagree) to 7 (Strongly Agree), students will rate themselves on one of the four MES ‘self-belief’ items as follows: “If I try hard, I believe I can do my schoolwork well.”
Based on students’ responses to the other three self-belief items, schools can create an average self-belief score for students and use this score in any analysis that is focused on understanding and making practice decisions about students’ self-belief.
For Load Reduction Instruction, there is the accompanying Load Reduction Instruction Scale (LRIS). The LRIS comprises 25 questions that students and teachers can report on – 5 questions for each of the 5 parts of the LRI framework.
Then, for instance, on a scale of 1 (Strongly Disagree) to 7 (Strongly Agree) students will rate their teacher on one of the five LRIS ‘difficulty reduction’ items as follows: “When we learn new things in this class, the teacher makes it easy at first.”
Based on students’ responses to the other four difficulty reduction items, schools can create an average difficulty reduction score and use this score in any analysis that is focused on understanding and making practice decisions about how to reduce the difficulty of instruction in the initial stages of learning.
If a validated instrument does not exist, the selected conceptual lens can be very helpful for schools in guiding the development of questions, items, or measures.
The selected framework usually has a good working definition of each indicator (e.g., of ‘self-belief’) under each domain (e.g., under ‘motivation’) and so the school can start writing items to map onto this definition.
6. Know who, what or where counts
Before any data are collected, it is vital to decide who, what or where to collect it from. That is, schools need to identify the people, factors, or places that have the biggest impact on the target outcomes.
This helps schools prioritize high-impact sources of information rather than dispersing effort across less influential variables. For example, research shows that students and teachers explain the most variance in motivation, engagement, and learning outcomes – so they should be a focus of data collection.
7. Collect the data
Only after Steps 1-6 should data be collected.
This is perhaps the most important take-home message from the article: a lot of thinking goes into good data collection well before the data are collected.
Put another way, good data collection is primarily a thinking exercise before it is a measurement exercise.
8. Use data for understanding and decision making
Once the data are collected, schools can interpret findings and apply them to practice, strategy, and improvement efforts.
For example, data collected using the MES allows schools to move beyond general impressions toward targeted intervention. Rather than simply concluding that “engagement is declining,” schools can identify which dimensions of engagement are changing and which student groups are most affected.
Many theoretical lenses are the basis of evidence-based practice and this guides decisions about which supports may be needed. For example, there is motivational practice advice available for each part of the Motivation and Engagement Wheel.
Likewise, because the LRIS data are connected to a clear framework (Load Reduction Instruction), teachers can make practical decisions about difficulty reduction, scaffolding, practice opportunities, feedback processes, and when students are ready for greater independence through approaches such as inquiry-based or problem-based learning.
Thus, rather than relying on vague judgments about “good teaching,” schools can identify specific instructional practices associated with stronger student outcomes and support teachers in implementing them consistently.
Final thoughts
Schools today face no shortage of data and information. The challenge is not access to data, but knowing what data matters and how to use it effectively.
When schools invest intellectual effort before collecting data, they are far more likely to gather evidence that is focused, meaningful, and actionable. That is the foundation of effective, evidence-informed school improvement.
