Handstablet

Sharing the Results of the 2016 Research-Based Products Campaign

It takes extra time, money, and effort for ed-tech developers to use research as they design and develop their products. And with no requirement or obvious incentive for developers to do this, we’ve heard many in the education community skeptically ask, “Are any of these products actually research-based?”

The answer is yes. Earlier this year, we issued a call for developers to tell us how they use research. Fifty-three companies shared an array of approaches: from applying learning science theory in the design of new products, to conducting user research, to launching evaluations of product effectiveness.

Experts at Teachers College, Columbia University reviewed applications and selected one exemplary company and two honorable mentions in each submission category. Congratulations to these companies:

  • Top Submission in the Learning Sciences Category: BrainQuake, Inc.
    • Honorable Mentions: Teachley and Woot Math
  • Top Submission in the User Research Category: Cogent Education
    • Honorable Mentions: Accelerate Learning and Smithsonian Learning Lab
  • Top Submission in the Evaluation Research Category: Woot Math
    • Honorable Mentions: Cell-Ed and Dreambox Learning


We talked with leaders from Woot Math, Cogent Education, and BrainQuake, Inc., to learn more about their strategies for using research. Check out their stories on EdSurge.

We received many impressive applications, and want to recognize all participating companies, which you can view below, for their efforts to use research. We hope their examples inspire other developers to use research, ultimately creating a marketplace filled with strong ed-tech products that are more likely to improve student learning outcomes.

The companies and products highlighted below are not endorsed by Digital Promise or Teachers College, Columbia University.

 

Curriculum Products | School Operations | Parent Home Use | Teacher Needs

Curriculum Products

21st Century Education

Product: 21Teach

One way they use research: “During the prototype development phase, early adaptors evaluated the usability and function of the app. This feedback was directly related to the product design and development, with their suggestions being used for implementation.”

30hands Learning

Product: 30hands Storyteller

One way they use research: “The methods used have included interviewing teachers, observing student usage, analyzing end-products created by students and teachers, and surveying teachers during our testing process.”

Accelerate Learning, Inc.

Product: STEMscopes

One way they use research: “It is the goal of STEMscopes to provide an inquiry-based curriculum, which incorporates the research-based, constructivist phases of the BSCS 5E Instructional Model (Bybee et al., 2006), the Key Findings from the National Research Council’s report, How People Learn (2000), and the individual state standards for science education.”

Age of Learning®, Inc.

Product: Abcmouse.com® Early Learning Academy

One way they use research: “A randomized, double-blind experimental study was implemented over a three-month period … A total of 281 families with kindergarten students completed this study.”

Allcancode

Product: Run Marco!

One way they use research: “Running workshops with students (having teachers and/or parents present) and then combining analysis from usage data helped us design/improve the visual programming language and the hints/feedback system.”

BeeLine Reader

Product: BeeLine Reader

One way they use research: “The short-cycle study lasted for 4 weeks. Each week, the students were individually assessed to measure reading fluency. The reading fluency scores for the BeeLine class was compared to: (1) the baseline scores from before the study began, and (2) the scores of the control class.”

Boom Learning Inc

Product: Boom Cards by Boom Learning

One way they use research: “We used research on engagement, mastery, and best practices in game design to make the student experience of playing the applets created by teachers more than just practicing concepts.”

BrainQuake Inc

Product: Wuzzit Trouble

One way they use research: “To date, two universities have conducted independent outcomes research on our products, leading to published efficacy studies in 2015 in the peer-reviewed International Journal of Serious Games.”

Cell-Ed

Product: Cell-Ed: Literacy on the Go!

Product: Cell-Ed: English on the Go!

One way they use research: “We also conducted a ‘Cell-Ed User Experience Assessment’ with a University of California Berkeley UX/UI team. This study was based on two sets of informational interviews with different study cohorts. The interviews covered the following topic areas: attraction, adoption, retention (technology endurance and learning acquisition), completion, and an overall Cell-Ed journey for each participant.”

Classroom, Inc.

Product: Community in Crisis

One way they use research: “The compelling storylines and problems require students to engage in ‘meaningful play,’ which research has shown enhances student learning (Gee, 2007; Salen & Zimmerman, 2005), while also providing experiences that are ‘pleasantly frustrating’ in order to hook and challenge students (Gee, 2007).”

codeSpark

Product: codeSpark Academy with The Foos

One way they use research: “We gathered feedback from hundreds of students and dozens of teachers and parents. We tested our first paper prototype with approximately 150 kids and 20 adults. We shared our first digital prototype with over 200 kids and 50 adults.”

Cogent Education

Product: Interactive Cases

One way they use research: “…analyses of student learning in the early stages of product development identified areas where students were struggling. We used these data to embed more scaffolding elements for these concepts to support the learner.”

Communication APPtitude

Product: InferCabulary

One way they use research: “In November and December of 2015, Dr. Michael Kennedy at the Curry School of Education at the University of Virginia conducted an independent six-week study comparing the use of the InferCabulary iPad app with ‘business as usual instruction’.”

DreamBox Learning

Product: DreamBox Learning K-8 Math

One way they use research: “To study the relationship between student learning and the use of blended learning software programs, the Center for Education Policy and Research (CEPR) at Harvard University used a quasi-experimental design to analyze the impact of DreamBox Learning K-8 Math on student achievement in grades 3-5 in two school systems.”

Einstein’s Workshop

Product: BlocksCAD

One way they use research: “The pilot was designed to evaluate usability and ease of understanding in using blocks-based code for 3D design, and to continue discussions of creative entrepreneurship and innovation.”

eSpark Learning

Product: Frontier

One way they use research: “Our product team visits classrooms at least once a week, gathering feedback on the existing product and prototypes of updates and new features.”

EvoBooks

Product: Inspira Program

One way they use research: “To estimate the impact of Inspira we used a Difference in Difference approach, where a control group (students who have not worked with Inspira) was compared to the treatment group (students who have worked with Inspira) before and after the intervention.”

FableVision Learning

Product: Get Published

Product: Fab@School Maker Studio

One way they use research: “We collected feedback from students as well as teachers. The students loved being able to tell us what they wanted changed, and they were able to see changes implemented.”

Flocabulary, Inc.

Product: Flocabulary

One way they use research: “Flocabulary uses several methods to gather feedback, including digital surveys, in-person classroom observations, product preview webinars with select super users, video conference focus groups, and educational technology conferences and summits.”

Ideaphora

Product: Ideaphora

One way they use research: “The design and development of Ideaphora’s web-based Knowledge Mapping Environment has been informed by three areas of scientific research: Concept Mapping, Scaffolding and Supports, and Making Thinking Visible.”

Infinut

Product: Kindergarten Math Home

Product: Kindergarten Math School

One way they use research: “Infinut piloted the Kindergarten Math application in two local kindergarten classrooms and obtained continuous feedback from teachers and students. As such, based on this feedback, we modified some aspects of the application to improve its usability.”

ISA Learning

Product: Team Builders

One way they use research: An independent evaluator…”analyzed pre-post test data from students, teacher surveys, and coding of video segments to determine if student collaboration improved from pre to post.”

Learning Games Studios

Product: Xenos

One way they use research: “As part of the evaluation of the Xenos-ISLE program, RTI [research organization] administered an English language assessment test called BEST Plus to measure students’ English language learning progress before and after their participation in the program. Using both quantitative and qualitative methods, RTI also examined how participation affects the students’ motivation, confidence, and language skill development.”

Legends of Learning, Inc

Product: Legends of Learning

One way they use research: “We used a controlled quasi-experimental design to compare outcomes for a sample of 1,080 students in 46 classrooms taught by 15 teachers, located in 10 schools across seven states, and located in urban (54%), suburban (23%), and rural (23%) contexts.”

Levered Learning

Product: Levered Mathematics

One way they use research: “Watching students using the curriculum and listening to their feedback has led to numerous changes, including longer wait times before triggering inactivity prompts, the refinement of the platform’s error recognition to provide more targeted tutorial videos based on the nature of student errors, and the inclusion of more simulation-based questions.”

Lexia Learning, A Rosetta Stone Company

Product: Lexia Reading Core5

One way they use research: “Data are currently being analyzed for a longitudinal, two-year study with 487 elementary school students in Southeastern Massachusetts (15% EL) and for a comparison between two Core5 schools to two ‘business as usual schools’ in Central Massachusetts (860 treatment and 774 control students).”

Lifeliqe

Product: Lifeliqe

One way they use research: “Pilot studies on schools aimed to verify the efficiency of the product and users‘ satisfaction, and quantitative surveys as well as qualitative structured interviews and focus groups were made. Also, observations of selected classroom lessons were conducted.”

LightSail Education

Product: LightSail

One way they use research: “A team of researchers from Johns Hopkins University studied 280 New York City Middle School students, 17 teachers, and five coaches from New York City’s Middle School Quality Initiative (MSQI) over seven weeks as they participated in LightSail’s SummerSail program.”

Newsela Inc.

Product: Newsela

One way they use research: “Research conducted over the past 30 years has demonstrated that when teachers use student progress monitoring, students learn more, teachers’ decision-making improves, and students become more aware of their own performance (Deno, 2003; Good & Jefferson, 1998). Newsela’s PRO Binder shows student activity and assessment performance by article, and gives teachers the tools to dive deeper into individual student information.”

Oxford Learning Solutions

Product: Easyread System

One way they use research: “We run RCT research projects directly with Easyread schools as part of their on-boarding process. Each school taking up Easyread runs an internal research study with an experimental structure, in order to fully evaluate the impact prior to taking up a full school account.”

Partnership in Education, Duquesne University & Simcoach Games

Product: Bibliotech

One way they use research: “The development of an engaging narrative combined with the multimedia elements – mini-games, infographics, interactive diagrams, videos, dynamic illustrations, and digital notebook – is supported by Mayer’s cognitive theory of multimedia learning (Mayer 2009).”

Reasoning Mind

Product: Foundations

One way they use research: “To keep up-to-date with the most current research, Reasoning Mind invites academic researchers to visit and give seminars, organizes book clubs to read and discuss relevant journal articles and books, and sends staff to conferences where the latest research is presented.”

SAS Curriculum Pathways

Product: SAS Read Aloud

One way they use research: “We’re lucky to have an internal research team comprised of professionals from a variety of academic disciplines, including individuals with doctoral degrees in educational psychology, computer science, public administration, and mathematics.”

SmartyPal, Inc.

Product: SmartyPal, Inc. Interactivity and Personalization Platform

One way they use research: “We have conducted research on user responses to our technology via both Phase I and II Small-Business Innovation Research grants through the National Science Foundation.”

Smithsonian Center for Learning and Digital Access (SCLDA)

Product: Smithsonian Learning Lab

One way they use research: “Rapid prototyping, first with paper prototypes and then with a functional test site, informed multiple rounds of product iterations tested with educators and students in diverse educational settings.”

Teachley

Product: Teachley Operations

One way they use research: “Because we know from research that effective strategy use is essential to mathematics learning, and kids with mathematical learning difficulties (MLD) do not develop these strategies on their own, we explicitly incorporated these research-based strategies into our games. Our apps are designed to promote advanced strategy acquisition through the use of carefully-designed virtual manipulatives.”

TenMarks Education

Product: TenMarks Math

One way they use research: “The design and development of TenMarks Math is informed by three bodies of research related to content, pedagogy, and access. For each of these, TenMarks conducted extensive reviews of the research literature, giving the greatest weight to approaches validated by rigorous, empirical evidence.”

The Concord Consortium

Product: Common Online Data Analysis Platform

One way they use research: “A series of qualitative studies using cognitive interviews with middle and high school students took place in the first two years of the project, a pilot study in a summer class in year two, and a larger, confirmatory study during a field test in year three.”

ThinkCERCA

Product: ThinkCERCA

One way they use research: “Feedback from small test groups allows us to make changes until we reach a design we believe in and can prove using data from testing…[so we can] clearly and effectively fulfill the needs of the user.”

Tiggly

Product: Tiggly Math

One way they use research: “We conducted a pilot study with twenty teachers to investigate best practices of implementing Tiggly Math in their curriculum, based on tablet implementation in their school: 1-to-1 programs, shared tablets, centers, or one tablet per classroom. We then gathered these best practices in our Tiggly Math Teacher Handbook, which outlines activities for implementing technology-enabled manipulatives into the curriculum.”

Up Learn Limited

Product: Up Learn

One way they use research: Using the theory of retrieval practice, UpLearn “tests students’ ability to recall knowledge, ability to recall definitions, and ability to apply knowledge and definitions to answer exam questions. When students do not score highly enough, the adaptive learning system programs that material back into their learning schedule.”

Woot Math, Inc.

Product: Woot Math: Fractions Unlocked

One way they use research: “Phase I evaluation was designed to determine whether the adaptive engine developed for Woot Math could improve learning. We tested two versions of Woot Math against each other in an RCT Phase II trial, which will look for impact on learning for classroom instruction with (vs. without) augmentation by Woot Math.”

Zinc Learning Labs

Product: Zinc Reading Labs

One way they use research: “We looked to the academics and experts in literacy to direct us on how best to augment Zinc Reading Labs to achieve our goal. The key research finding we used was their number one recommendation to improve adolescent literacy: to provide explicit vocabulary instruction.”

School Operations [top]

AdmitHub

Product: Chatbot

One way they use research: “We randomized a group of 7,527 admitted students into a control and treatment group. The control group continued to receive regular email communications from the school, while the treatment group had access to the chatbot. We monitored user text-messaging data throughout the study and their completion of key tasks related to enrollment through the university’s database.”

Lea(R)n

Product: LearnPlatform

One way they use research: “Lea(R)n conducted rigorous psychometric work and numerous studies to inform the development and validation of the Lea(R)n grading protocol and the associated rubric, which educators use to evaluate educational technologies.”

Parent/Home Use [top]

21st Century Education

Product: 1word.us

One way they use research: “The team looked at Marzano’s (2004) Six Step Process for teaching academic vocabulary, and incorporated the latest in mobile usability and marketing research to determine the layout and features to incorporate in the application.”

Retronator

Product: Pixel Art Academy

One way they use research: “The learning theories used include andragogy, a model of assumptions developed to describe learning in adults (Knowles, 1970); self-directed learning model with its three overlapping dimensions of self-management, self-monitoring, and motivation (Garrison, 1997); and the four-phase model of interest development, which we used in relation to the shift from extrinsic to intrinsic motivation (Hidi & Renninger, 2006).”

Tinybop

Product: The Explorer’s Library

One way they use research: “We gathered research and decided that simulations were an effective way to teach physics concepts, especially when paired with physical experimentation. As a result, we designed our app to simulate the physics of simple machines, and included examples of physical activities to pair with the app in our handbook.”

Teacher Needs [top]

Allenwood Company LLC

Product: Professional Development for Novice Teachers of ELs

One way they use research: “Instructional videos were evaluated with a rubric developed from the Danielson teacher evaluation framework, and educational research on the effective use of instructional videos in the classroom (Canning-Wilson & Wallace 2000; Cruse 2011; Danielson 2013).”

Apprendis

Product: Inq-ITS

One way they use research: “…we leveraged literature about students’ misconceptions in science, their difficulties with inquiry, and teachers’ assessment and instructional practices, and used these along with design frameworks for conceptualizing inquiry skills/tasks, and computational techniques for scoring our assessments.”

Mathspace

Product: Mathspace

One way they use research: “Mathspace was chosen to be part of an efficacy study with three Silicon Valley middle schools that piloted Mathspace…A pilot was conducted in different contexts.”

Mindprint Learning

Product: Comprehensive Mindprint

One way they use research: “We licensed our battery from the Brain Behavior Lab at Penn Medicine. The researchers normed the battery with funding from an NIMH (National Institute of Mental Health) grant. Through our research agreement, we commercialized the battery to ensure its reliability and efficacy for the educational community.”

SAS Curriculum Pathways

Product: SAS Reading Records

One way they use research: “With the help of our Teacher Institute reading specialist and research team, we identified key skills at each reading level and coded comprehension questions and prompts according to those skills.”

VerveEd

Product: Moodle Accreditation Platform

One way they use research: “System data was continuously being captured and we would look to correlate it against qualitative data every four weeks. That was the average time it would take us to obtain user feedback, apply changes based on the feedback, and seek testing from the same users again.”

WriteReader

Product: WriteReader

One way they use research: “Chomsky’s research and the ‘whole language’ approach is the fundamental basis of our product. The mission has been to create a digital tool that makes it easy and engaging for the students to learn through this pedagogical approach.”