We all want to help students learn. When it comes to education technology (edtech), school leaders and developers alike want to provide the best tools—ones that truly make a difference. This is especially critical for students who are not being served by current instruction.
To support this goal, Digital Promise’s Marketplace initiative is dedicated to identifying “effective, engaging, tech-enabled learning environments.” But what does “effective” really mean? To identify effective digital learning tools, we have to know what kind of impact we are looking for. At its core, being effective is about reaching a learning goal.
What makes a learning app good? Where is the checklist that leads to a seal of approval? As a researcher, I get asked these questions all the time. Researchers are in the business of figuring out how to identify and measure things. Frustratingly, there is no one right answer. With so many different tools for a wide range of learners, the trick is to know it when we see it.
To figure out if a learning tool is effective, ask these five questions:
Who are the learners?
Learners come in all shapes and sizes. Good tools are designed to take into account the developmental level of the target learners, the experience they bring with them to the activity, and the challenges they may face. When choosing digital tools to help someone learn, I look for thoughtful design that pays attention to language (vocabulary levels and multiple language support), interaction modes (speaking and listening; looking, reading and writing; touching, clicking, and gesturing), and ethical design (unbiased algorithms and data privacy). A tool that does not fit the learner is unlikely to support learning. But a tool designed with a deep, research-based understanding of the learners can be highly impactful.
What are students learning?
To know that a given tool “works” requires an idea of what learning looks like. What will someone be able to do or say differently because they’ve used this app, game, or site? Outcomes can be anything from having enough confidence to start conversation in a new language to knowing the elements of the periodic table. Facts, skills, attitudes, and choices are all in the mix. When evaluating digital learning tools, I look for clarity around both success on a test and how that relates to decision making in real life.
Once it’s clear what someone needs to learn, I focus on barriers. Like good teachers, the best edtech designers really understand what’s hard about learning, whether the goal is to grasp fractions, solve algebra problems, understand HIV, spot fake news, or interpret literary texts. Only then can developers do something about it.
How are the tools teaching?
The body of research on how people learn is full of tips that edtech developers can use to make learning more effective. For example, research shows that exploring contrasting cases before a lecture is better than just hearing the expert solution. Other studies show that students learn more from science activities that simulate authentic problem solving than from excellent traditional lectures. And a sense of social belonging activity has been shown to raise both academic and health outcomes of marginalized students over time. When evaluating learning tools, I look for one or more interactions that align with the science of learning.
How does the technology help?
The fact that the internet delivers huge quantities of information every day is a game changer when it comes to access. Who hasn’t watched a video online to learn a tricky technique? And computers are really good at quickly reporting if an answer is right or not, which may be why practicing facts is the mainstay of so many learning apps. But there are many more ways that edtech developers can use new technologies to support learning, particularly for marginalized students too often ignored by the edtech industry.
Interactive apps can let students manipulate functions or visualize electrons to build better mental models. Mobile devices can help connect people and places. Apps and toys can map abstract ideas to the physical world we live in. And taking on a helping identity in virtual reality has been shown to improve pro-social behaviors. If the technology doesn’t do something beyond what we can do with paper and pencil, why bother? I look for a digital learning tool to use technology in ways that amplify or transform the activities that help people learn.
How would we know if the tool is effective?
Let’s be honest: there’s never going to be a randomized control trial for every new app. We can’t “prove” a tool will work in every district’s unique context. Instead, we have to look for a fit between our needs and the answers to the questions above. When we tie all these parts together, we can trace a “theory of change” to see if the answer makes sense. Using the Digital Promise Edtech Pilot Framework? The answers to these questions will help you to determine if a product meets your students’ needs.
Efficacy Matters in Edtech
Technology is great at making some tasks more efficient. Efficiency is about maximizing impact while minimizing cost. But without efficacy, efficiency of digital learning tools is meaningless. All the effort and expense is wasted if you don’t achieve learning gains. To level up the impact of edtech, we need to keep insisting on effective digital learning tools.
Explore the Edtech Marketplace Today blog series to hear other voices from the field share important perspectives on challenges and strategies to improve the edtech market. Visit the Product Certifications website to learn more about the edtech marketplace and subscribe to Digital Promise’s Action Report to stay up-to-date on their work.
