It takes extra time, money, and effort for ed-tech developers to use research as they design and develop their products. And with no requirement or obvious incentive for developers to do this, we’ve heard many in the education community skeptically ask, “Are any of these products actually research-based?”
The answer is yes. Earlier this year, we issued a call for developers to tell us how they use research. Fifty-three companies shared an array of approaches: from applying learning science theory in the design of new products, to conducting user research, to launching evaluations of product effectiveness.
Experts at Teachers College, Columbia University reviewed applications and selected one exemplary company and two honorable mentions in each submission category. Congratulations to these companies:
We talked with leaders from Woot Math, Cogent Education, and BrainQuake, Inc., to learn more about their strategies for using research. Check out their stories on EdSurge. You can also read our analysis of all 53 submissions.
We received many impressive applications, and want to recognize all participating companies, which you can view below, for their efforts to use research. We hope their examples inspire other developers to use research, ultimately creating a marketplace filled with strong ed-tech products that are more likely to improve student learning outcomes.
The companies and products highlighted below are not endorsed by Digital Promise or Teachers College, Columbia University.
Product: 21Teach
One way they use research: “During the prototype development phase, early adaptors evaluated the usability and function of the app. This feedback was directly related to the product design and development, with their suggestions being used for implementation.”
Product: 30hands Storyteller
One way they use research: “The methods used have included interviewing teachers, observing student usage, analyzing end-products created by students and teachers, and surveying teachers during our testing process.”
Product: STEMscopes
One way they use research: “It is the goal of STEMscopes to provide an inquiry-based curriculum, which incorporates the research-based, constructivist phases of the BSCS 5E Instructional Model (Bybee et al., 2006), the Key Findings from the National Research Council’s report, How People Learn (2000), and the individual state standards for science education.”
Product: Abcmouse.com® Early Learning Academy
One way they use research: “A randomized, double-blind experimental study was implemented over a three-month period … A total of 281 families with kindergarten students completed this study.”
Product: Run Marco!
One way they use research: “Running workshops with students (having teachers and/or parents present) and then combining analysis from usage data helped us design/improve the visual programming language and the hints/feedback system.”
Product: BeeLine Reader
One way they use research: “The short-cycle study lasted for 4 weeks. Each week, the students were individually assessed to measure reading fluency. The reading fluency scores for the BeeLine class was compared to: (1) the baseline scores from before the study began, and (2) the scores of the control class.”
Product: Boom Cards by Boom Learning
One way they use research: “We used research on engagement, mastery, and best practices in game design to make the student experience of playing the applets created by teachers more than just practicing concepts.”
Product: Wuzzit Trouble
One way they use research: “To date, two universities have conducted independent outcomes research on our products, leading to published efficacy studies in 2015 in the peer-reviewed International Journal of Serious Games.”
Product: Cell-Ed: Literacy on the Go!
Product: Cell-Ed: English on the Go!
One way they use research: “We also conducted a ‘Cell-Ed User Experience Assessment’ with a University of California Berkeley UX/UI team. This study was based on two sets of informational interviews with different study cohorts. The interviews covered the following topic areas: attraction, adoption, retention (technology endurance and learning acquisition), completion, and an overall Cell-Ed journey for each participant.”
Product: Community in Crisis
One way they use research: “The compelling storylines and problems require students to engage in ‘meaningful play,’ which research has shown enhances student learning (Gee, 2007; Salen & Zimmerman, 2005), while also providing experiences that are ‘pleasantly frustrating’ in order to hook and challenge students (Gee, 2007).”
Product: codeSpark Academy with The Foos
One way they use research: “We gathered feedback from hundreds of students and dozens of teachers and parents. We tested our first paper prototype with approximately 150 kids and 20 adults. We shared our first digital prototype with over 200 kids and 50 adults.”
Product: Interactive Cases
One way they use research: “…analyses of student learning in the early stages of product development identified areas where students were struggling. We used these data to embed more scaffolding elements for these concepts to support the learner.”
Product: InferCabulary
One way they use research: “In November and December of 2015, Dr. Michael Kennedy at the Curry School of Education at the University of Virginia conducted an independent six-week study comparing the use of the InferCabulary iPad app with ‘business as usual instruction’.”
Product: DreamBox Learning K-8 Math
One way they use research: “To study the relationship between student learning and the use of blended learning software programs, the Center for Education Policy and Research (CEPR) at Harvard University used a quasi-experimental design to analyze the impact of DreamBox Learning K-8 Math on student achievement in grades 3-5 in two school systems.”
Product: BlocksCAD
One way they use research: “The pilot was designed to evaluate usability and ease of understanding in using blocks-based code for 3D design, and to continue discussions of creative entrepreneurship and innovation.”
Product: Frontier
One way they use research: “Our product team visits classrooms at least once a week, gathering feedback on the existing product and prototypes of updates and new features.”
Product: Inspira Program
One way they use research: “To estimate the impact of Inspira we used a Difference in Difference approach, where a control group (students who have not worked with Inspira) was compared to the treatment group (students who have worked with Inspira) before and after the intervention.”
Product: Get Published
Product: Fab@School Maker Studio
One way they use research: “We collected feedback from students as well as teachers. The students loved being able to tell us what they wanted changed, and they were able to see changes implemented.”
Product: Flocabulary
One way they use research: “Flocabulary uses several methods to gather feedback, including digital surveys, in-person classroom observations, product preview webinars with select super users, video conference focus groups, and educational technology conferences and summits.”
Product: Ideaphora
One way they use research: “The design and development of Ideaphora’s web-based Knowledge Mapping Environment has been informed by three areas of scientific research: Concept Mapping, Scaffolding and Supports, and Making Thinking Visible.”
Product: Kindergarten Math Home
Product: Kindergarten Math School
One way they use research: “Infinut piloted the Kindergarten Math application in two local kindergarten classrooms and obtained continuous feedback from teachers and students. As such, based on this feedback, we modified some aspects of the application to improve its usability.”
Product: Team Builders
One way they use research: An independent evaluator…”analyzed pre-post test data from students, teacher surveys, and coding of video segments to determine if student collaboration improved from pre to post.”
Product: Xenos
One way they use research: “As part of the evaluation of the Xenos-ISLE program, RTI [research organization] administered an English language assessment test called BEST Plus to measure students’ English language learning progress before and after their participation in the program. Using both quantitative and qualitative methods, RTI also examined how participation affects the students’ motivation, confidence, and language skill development.”
Product: Legends of Learning
One way they use research: “We used a controlled quasi-experimental design to compare outcomes for a sample of 1,080 students in 46 classrooms taught by 15 teachers, located in 10 schools across seven states, and located in urban (54%), suburban (23%), and rural (23%) contexts.”
Product: Levered Mathematics
One way they use research: “Watching students using the curriculum and listening to their feedback has led to numerous changes, including longer wait times before triggering inactivity prompts, the refinement of the platform’s error recognition to provide more targeted tutorial videos based on the nature of student errors, and the inclusion of more simulation-based questions.”
Product: Lexia Reading Core5
One way they use research: “Data are currently being analyzed for a longitudinal, two-year study with 487 elementary school students in Southeastern Massachusetts (15% EL) and for a comparison between two Core5 schools to two ‘business as usual schools’ in Central Massachusetts (860 treatment and 774 control students).”
Product: Lifeliqe
One way they use research: “Pilot studies on schools aimed to verify the efficiency of the product and users‘ satisfaction, and quantitative surveys as well as qualitative structured interviews and focus groups were made. Also, observations of selected classroom lessons were conducted.”
Product: LightSail
One way they use research: “A team of researchers from Johns Hopkins University studied 280 New York City Middle School students, 17 teachers, and five coaches from New York City’s Middle School Quality Initiative (MSQI) over seven weeks as they participated in LightSail’s SummerSail program.”
Product: Newsela
One way they use research: “Research conducted over the past 30 years has demonstrated that when teachers use student progress monitoring, students learn more, teachers’ decision-making improves, and students become more aware of their own performance (Deno, 2003; Good & Jefferson, 1998). Newsela’s PRO Binder shows student activity and assessment performance by article, and gives teachers the tools to dive deeper into individual student information.”
Product: Easyread System
One way they use research: “We run RCT research projects directly with Easyread schools as part of their on-boarding process. Each school taking up Easyread runs an internal research study with an experimental structure, in order to fully evaluate the impact prior to taking up a full school account.”
Product: Bibliotech
One way they use research: “The development of an engaging narrative combined with the multimedia elements – mini-games, infographics, interactive diagrams, videos, dynamic illustrations, and digital notebook – is supported by Mayer’s cognitive theory of multimedia learning (Mayer 2009).”
Product: Foundations
One way they use research: “To keep up-to-date with the most current research, Reasoning Mind invites academic researchers to visit and give seminars, organizes book clubs to read and discuss relevant journal articles and books, and sends staff to conferences where the latest research is presented.”
Product: SAS Read Aloud
One way they use research: “We’re lucky to have an internal research team comprised of professionals from a variety of academic disciplines, including individuals with doctoral degrees in educational psychology, computer science, public administration, and mathematics.”
Product: SmartyPal, Inc. Interactivity and Personalization Platform
One way they use research: “We have conducted research on user responses to our technology via both Phase I and II Small-Business Innovation Research grants through the National Science Foundation.”
Product: Smithsonian Learning Lab
One way they use research: “Rapid prototyping, first with paper prototypes and then with a functional test site, informed multiple rounds of product iterations tested with educators and students in diverse educational settings.”
Product: Teachley Operations
One way they use research: “Because we know from research that effective strategy use is essential to mathematics learning, and kids with mathematical learning difficulties (MLD) do not develop these strategies on their own, we explicitly incorporated these research-based strategies into our games. Our apps are designed to promote advanced strategy acquisition through the use of carefully-designed virtual manipulatives.”
Product: TenMarks Math
One way they use research: “The design and development of TenMarks Math is informed by three bodies of research related to content, pedagogy, and access. For each of these, TenMarks conducted extensive reviews of the research literature, giving the greatest weight to approaches validated by rigorous, empirical evidence.”
Product: Common Online Data Analysis Platform
One way they use research: “A series of qualitative studies using cognitive interviews with middle and high school students took place in the first two years of the project, a pilot study in a summer class in year two, and a larger, confirmatory study during a field test in year three.”
Product: ThinkCERCA
One way they use research: “Feedback from small test groups allows us to make changes until we reach a design we believe in and can prove using data from testing…[so we can] clearly and effectively fulfill the needs of the user.”
Product: Tiggly Math
One way they use research: “We conducted a pilot study with twenty teachers to investigate best practices of implementing Tiggly Math in their curriculum, based on tablet implementation in their school: 1-to-1 programs, shared tablets, centers, or one tablet per classroom. We then gathered these best practices in our Tiggly Math Teacher Handbook, which outlines activities for implementing technology-enabled manipulatives into the curriculum.”
Product: Up Learn
One way they use research: Using the theory of retrieval practice, UpLearn “tests students’ ability to recall knowledge, ability to recall definitions, and ability to apply knowledge and definitions to answer exam questions. When students do not score highly enough, the adaptive learning system programs that material back into their learning schedule.”
Product: Woot Math: Fractions Unlocked
One way they use research: “Phase I evaluation was designed to determine whether the adaptive engine developed for Woot Math could improve learning. We tested two versions of Woot Math against each other in an RCT Phase II trial, which will look for impact on learning for classroom instruction with (vs. without) augmentation by Woot Math.”
Product: Zinc Reading Labs
One way they use research: “We looked to the academics and experts in literacy to direct us on how best to augment Zinc Reading Labs to achieve our goal. The key research finding we used was their number one recommendation to improve adolescent literacy: to provide explicit vocabulary instruction.”
Product: Chatbot
One way they use research: “We randomized a group of 7,527 admitted students into a control and treatment group. The control group continued to receive regular email communications from the school, while the treatment group had access to the chatbot. We monitored user text-messaging data throughout the study and their completion of key tasks related to enrollment through the university’s database.”
Product: 1word.us
One way they use research: “The team looked at Marzano’s (2004) Six Step Process for teaching academic vocabulary, and incorporated the latest in mobile usability and marketing research to determine the layout and features to incorporate in the application.”
Product: Pixel Art Academy
One way they use research: “The learning theories used include andragogy, a model of assumptions developed to describe learning in adults (Knowles, 1970); self-directed learning model with its three overlapping dimensions of self-management, self-monitoring, and motivation (Garrison, 1997); and the four-phase model of interest development, which we used in relation to the shift from extrinsic to intrinsic motivation (Hidi & Renninger, 2006).”
Product: The Explorer’s Library
One way they use research: “We gathered research and decided that simulations were an effective way to teach physics concepts, especially when paired with physical experimentation. As a result, we designed our app to simulate the physics of simple machines, and included examples of physical activities to pair with the app in our handbook.”
Product: Professional Development for Novice Teachers of ELs
One way they use research: “Instructional videos were evaluated with a rubric developed from the Danielson teacher evaluation framework, and educational research on the effective use of instructional videos in the classroom (Canning-Wilson & Wallace 2000; Cruse 2011; Danielson 2013).”
Product: Inq-ITS
One way they use research: “…we leveraged literature about students’ misconceptions in science, their difficulties with inquiry, and teachers’ assessment and instructional practices, and used these along with design frameworks for conceptualizing inquiry skills/tasks, and computational techniques for scoring our assessments.”
Product: Mathspace
One way they use research: “Mathspace was chosen to be part of an efficacy study with three Silicon Valley middle schools that piloted Mathspace…A pilot was conducted in different contexts.”
Product: Comprehensive Mindprint
One way they use research: “We licensed our battery from the Brain Behavior Lab at Penn Medicine. The researchers normed the battery with funding from an NIMH (National Institute of Mental Health) grant. Through our research agreement, we commercialized the battery to ensure its reliability and efficacy for the educational community.”
Product: SAS Reading Records
One way they use research: “With the help of our Teacher Institute reading specialist and research team, we identified key skills at each reading level and coded comprehension questions and prompts according to those skills.”
Product: Moodle Accreditation Platform
One way they use research: “System data was continuously being captured and we would look to correlate it against qualitative data every four weeks. That was the average time it would take us to obtain user feedback, apply changes based on the feedback, and seek testing from the same users again.”
Product: WriteReader
One way they use research: “Chomsky’s research and the ‘whole language’ approach is the fundamental basis of our product. The mission has been to create a digital tool that makes it easy and engaging for the students to learn through this pedagogical approach.”
About