Rapid Cycle Pilots: Lessons Learned from Math Trials in Six Districts

Over the past year, Digital Promise identified common challenges in K-12 education that might be addressed using innovative education technology (ed-tech) products, and supported school districts as they piloted products designed to address these challenges. The goals of this effort were to influence ed-tech providers to improve their products, to influence school districts to improve their ed-tech piloting practices, and to increase awareness of ed-tech pilot best practices among K-12 district leaders.

In spring 2016, Digital Promise worked with six League of Innovative Schools districts to conduct pilot studies of two middle school math products. We also supported both companies in designing and conducting the pilot study, and provided formative feedback throughout the process. At the end of the study, district staff reported an improved understanding of the pilot process and the value of conducting pilots before purchasing new ed-tech products. The companies reported that the feedback they received ultimately improved their products.

The following recommendations, based on key findings from this study, can help other districts and product developers conduct successful edtech pilots. These include:

• Establish Educator Buy-in: When educators are involved in choosing and deciding to pilot a tool, their engagement with the education technology is higher and the tools are implemented with greater fidelity.
• Consider Pilot Size: Pilots should include a sample of the student and teacher population who might benefit from the tool, leaving a non-user group with whom to compare results. Smaller-scale pilots are also easier to implement because they require less time for professional learning, IT integration, and data analysis.
• Maintain Open Communication Among Stakeholders: Cooperation and communication between educators, school leaders, product companies, and researchers is essential to ensure common goals and expectations throughout the process.
• Prepare for Purchasing: When planning for an ed-tech pilot, consider what type of evidence will be needed in order to make an informed purchasing decision at the end of the process.
• Academic Rigor is Important: Educator buy-in is improved when they feel the tools are academically rigorous and well aligned to the curriculum.
• Setup and Rostering: Simplifying student and teacher account registration can help prevent frustration for educators and IT departments, and ensure the pilot rollout is
  successful.
• Offer Educator Support: Educators appreciate having use cases to observe when trying a new tool, and benefit from ongoing support throughout the pilot’s implementation.