Flipping the Status Quo: Outcomes Based Contracting May Ensure Edtech Reaches Its Full Potential

Making the right decisions for students is at the core of what our district partners do every day. However, when it comes to edtech, this has been a long-standing challenge.

Before making any purchase, we typically ask ourselves: Why am I buying this? What will it cost? Will it be effective and meet my needs? Historically, edtech procurement has not followed this thoughtful process. Education leaders often feel overwhelmed by pitches; some have noted that responding to all the provider emails and calls could easily become a full-time job. In addition to direct outreach, many major education conferences feature expansive vendor floors filled with demos, creating pressure to stay updated on emerging tools and technologies.

Recently, our team had the honor of joining the team leading the Center for Outcomes Based Contracting at the Southern Education Foundation to dive into discussions about disrupting the status quo at the Outcomes Based Contracting (OBC) Annual Conference. We held two sessions aimed at empowering education leaders in edtech procurement to share our perspective on the marketplace and learn together with education leaders committed to ensuring that every dollar spent is tied to positive student outcomes. After everyone in the room raised their hand in agreement that they had sat through a pitch where the provider claimed their tool would solve all of their district’s challenges, most participants raised a second hand to confirm that these claims were made without the provider asking a single question about their community. Our session shifted to the data on fundamental questions we should all be asking about the edtech marketplace.

When asked who had written down what outcomes they expected from their products, only a few hands were raised, and those hands belonged to district teams that had been involved in the OBC work. It’s essential that we emphasize that the leaders who didn’t raise their hands are absolutely not alone: That’s the norm!

There has been a large, consistent increase in the use of edtech tools in recent years. In the 2023-24 school year, districts accessed an average of 2,739 distinct edtech tools, an increase of 8% from the previous school year. Looking more closely at the classroom, students accessed an average of 45 tools last school year — an overwhelming number to incorporate, learn, and use effectively. This makes it extremely challenging to build consistent learning experiences and measure student progress. As more tools are developed and introduced each year, it becomes vital for districts to be selective and ground their decision-making in intention and evidence. We can move away from the inundation of tools driven by the need to keep up with peer districts, emerging trends, and new technology, and instead focus on decision-making that emphasizes student participation in 21st century learning, empowerment, and success.

We advocate for flipping this approach on its head and starting with the vision. What learning experiences do you want, for which students, and why? What outcomes and metrics will allow you to measure progress toward this vision? What tools, resources, and services will enable your school community to achieve that vision? Finally, how can you partner with your provider to ensure effective edtech implementation that propels student learning?

As we described the booming market — edtech jumped from an estimated budget of $26B in 2018 to an estimated $145B in the 2022-23 school year, globally — what resonated most deeply with our audiences was that all of this was based on estimations from a completely opaque market. Some of our district partners have discovered that peer districts have paid as much as 50% more for licenses for the exact same school. In fact, one district found that school-level contracts across the same district ranged dramatically in price for the same tool and by bringing several of these contracts to a district level with a pre-negotiated rate, their system saved over $250,000 without cutting any tools!

This hit home. How can districts effectively negotiate rates with no sense of what others are paying? At Digital Promise, we’ve been collecting data from our product certification applications for over four years, asking 100’s of edtech vendors who apply if they offer pricing transparency. Over 90% of vendors do not and require individual calls to the sales team for a district to get any pricing information. To add to the chaos, this pricing information is generally only referencing student licenses — pricing for professional learning (to ensure educators know how to use the tool) is a separate bill that districts must additionally figure out how to fund.

Short answer: No.

Slightly longer answer: We aren’t even collecting data to know.

SETDA’s 2023 State EdTech Trends Report found that, while there have been some improvements from the 2022 to 2023 school year, there are still a lot of unanswered questions around usage. Only 5% of state leaders reported that their state collects ample data on frequency of use and effectiveness of edtech tools, while nearly 40% reported that their state does not collect data on use or effectiveness of edtech tools at all.

Data suggests that of the 100 most used edtech products, only one quarter have a demonstrated rationale (i.e., logic model) and less than 10% show any level of evidence of impact, that is, very little research that it should, or does work. In addition, with students trying to use around 45 different tools on average each year, it’s unclear how they could use any of these tools effectively. Research supports this challenge. A recent review of most of (the few) efficacy reports for edtech tools, found that, while for students who used the tools “as recommended,” these tools were effective at promoting student achievement, the vast majority of students (95%) were not using the tools “as recommended” (e.g., 30 mins a day, three days a week). That is, 95% of students are not using tools effectively, and are not able to reap the promised benefits of the tool. In addition, some research found that the students who are less likely to use the tool as recommended tend to be those students who are already experiencing opportunity gaps in their learning, including lower income and students of color).

The Elementary and Secondary School Emergency Relief (ESSER) cliff, coupled with the new wave of AI in education, has created a critical moment to pause and evaluate the status quo for edtech procurement and its impact on student learning. With significantly tighter budgets and media headlines about district investments gone wrong with AI, district leaders have a golden opportunity to build formal evaluation systems to make edtech procurement decisions. We believe that the path forward calls for collaborative and innovative solutions. Product certifications can support districts’ need to readily find quality products that support safe, equitable, and effective learning for all students. By contracting under an outcomes based contract, districts and providers stay mutually accountable to what matters most: student learning.

Outcomes based contracting enables district leaders to collaboratively identify their vision for students, based on data, to understand where the need is, and articulate how the tool explicitly will move learners toward the intended goal. Under this model, districts also set the pricing based on a core set of inputs grounded in their local context: their budget, the number of students they need to serve, the baseline data, and the outcome/metric pairs that will measure success. This means that under an OBC, Requests for Proposal (RFPs) decisions are not determined by the lowest priced bid, but instead by the quality of the product and evidence that it will meet the needs of students served. This enables education leaders to incorporate points to the scoring rubric based on the status of product certifications, helping education leaders elevate their priorities around ensuring the tools are research and evidence based and inclusively designed. The mutual accountability and continuous improvement language in the RFP and subsequent contract ensure the tool is used as recommended, driving toward meaningful student learning instead of use of edtech for the sake of edtech alone.

As Theo Shaw, an National OBC Expert, shared at the convening, “The time of a district paying for a $2 million contract for two students to use a tool is over.” At Digital Promise, we’re thrilled to be a strategic partner and the edtech cohort implementation research partner to support the expansion of this work around the country. Join the movement of empowered education leaders who partner with providers to identify what they need for their community based on a data-driven vision for their students, and prioritize working with products designed for success. This powerful partnership will ensure your system follows a meaningful implementation plan with continuous improvement cycles, enabling effective use of edtech tools that lead to measurable, meaningful student success.

Get started!

• Check out our presentation to learn more.
• Explore our district resources to learn how to integrate product certifications into your evaluation and procurement processes.
• Learn more about our partnership with a variety of nonprofit organizations to build the 5 Quality Indicators for Edtech & AI Products.
• Hold providers, districts, and schools mutually accountable for the dollars spent on edtech by tying outcomes to payment through partnership with the Center for Outcomes Based Contracting.

References

Harvard (2016). DREAMBOX LEARNING ACHIEVEMENT GROWTH in the Howard County Public School System and Rocketship Education: Key findings report. https://cepr.harvard.edu/files/cepr/files/dreambox-key-findings.pdf

Rauf, D (2023). Why a major school district wants outcomes-based standards for vendors. EdWeek MarketBrief. https://marketbrief.edweek.org/sales-marketing/why-a-major-school-district-wants-outcomes-based-standards-for-vendors/2023/01

OBC Impact Report- looking back at 2023, looking ahead to 2024
OBC IMPACT REPORT_FINAL.pdf

Holt, 2024. https://www.educationnext.org/5-percent-problem-online-mathematics-programs-may-benefit-most-kids-who-need-it-least/

Council of Chief State School Officers. (2024). States leading: How state education agencies are
leveraging the ESSER set-aside. https://learning.ccsso.org/states-leading-how-state-education-agencies-are-leveraging-the-esser-set-aside

Diliberti, M. K., Schwartz, H. L., Doan, S., Shapiro, A., Rainey, L. R., & Lake, R. J. (2024).
Using artificial intelligence tools in K–12 classrooms. RAND Corporation. https://www.rand.org/pubs/research_reports/RRA956-21.html

Green, A., Tench, B., Weinstein, E., & Lara, E. (2024). Teen and young adult perspectives on
generative AI: Patterns of use, excitements, and concerns. Common Sense. https://www.commonsensemedia.org/sites/default/files/research/report/teen-and-young-adult-perspectives-on-generative-ai.pdf

Instructure. (2024). The edtech top 40: A look at k-12 edtech engagement during the 2023-24
school year. https://www.instructure.com/resources/research-reports/edtech-top-40-look-k-12-edtech-engagement-during-2023-24-school-year

ISTE. (2024). Easing the burden on schools: Five quality indicators for edtech & AI products.
https://iste.org/blog/easing-burden-on-schools-five-quality-indicators-for-edtech-ai-products

McLemore, C., & Rae, J. (2024). How district leaders make edtech purchasing decisions. EdSurge. https://www.edsurge.com/news/2024-06-24-how-district-leaders-make-edtech-purchasing-decisions

SETDA. (n.d.). Collaborative groups. https://www.setda.org/about/collaboratives/

Silverman, R. D., Keane, K., Darling-Hammond, E., & Khanna, S. (2022). The Effects of Educational Technology Interventions on Literacy in Elementary School: A Meta-Analysis. Review of Educational Research, 00346543241261073.

Reyes, E.F. & Popoff, E. (2023). 2023 state edtech trends report. SETDA.
https://www.setda.org/priorities/state-trends/

U.S. Department of Education. (2023). A first look: Students’ access to educational opportunities
in U.S. public schools. Office for Civil Rights. https://www2.ed.gov/about/offices/list/ocr/docs/crdc-educational-opportunities-report.pdf

U.S. Department of Education (2024). A Call to Action for Closing the Digital Access, Design, and Use Divides: 2024 National Educational Technology Plan. Office of Educational Technology. https://tech.ed.gov/netp/