The Good and Bad of Ed-Tech: Improving 21 Century Student Outcomes

According to a recent The74 article "Texting, Personalization and Free Computers for All", while the Ed-tech industry is expected to generate revenues of $252 billion dollars by 2020, research on Ed-tech effectiveness is still in its infancy. As late as 2015, for instance, a Sesame Education Workshop education technology study reported that 77 percent of the most popular literacy apps still did not provide information about the research behind them and two-thirds also did not mention a guiding curriculum.

As such, while findings still need to be taken with a grain of salt given the paucity of ongoing Ed-tech effectiveness research, a recent National Bureau of Economic Research (NBER) report on the association of various types of Ed-tech and student achievement gains is receiving some important buzz within education circles.

So what's working according to NBER?

One of the most promising uses of educational technology is learning software, especially programs geared toward improving math skills through personalization. Of the 22 math studies that NBER reviewed, 15 reported positive effects, six reported no effects, and only one reported negative effects. A math curriculum in Texas that relied on computer software had one of the largest effect sizes of all the programs analyzed in NBER’s review, for its improvement of math scores for seventh- and eighth-graders.

Computer-assisted learning programs also highlight how personalization through technology can help students. A math homework program in Maine called ASSISTments gives students feedback and guidance as they solve problems. Teachers can then use the data to tailor their instruction toward student needs. This program in particular required only 30 to 40 minutes per week and had positive effects. New Classrooms has also developed a computer assisted learning program in math that provides teachers with a personalized student "lesson play book" for each day of math learning. The most recent third party evaluation conducted by New Classrooms, on the effectiveness of its "teach to one" model also found that students made annual academic gains that were equivalent to 1.5 times the national average—equivalent to half a year of additional learning.

While teachers have long attempted to differentiate learning for groups of students, many have struggled in finding time to personalize daily lessons for each and every student. It is exciting to think about how programs such as ASSISTments and New Classrooms, can now do this but I wonder how Ed-Tech companies that focus on the personalization of learning will be able to scale up operations given the inevitable increasing demand for thousands of personalized student lesson plans on a daily basis. Do any readers have knowledge on how to scale out teach to one models to ensure deep academic impact?

Research on interventions aimed at influencing behaviors — like encouraging parents to read to their kids or spurring a student to complete the FAFSA — have shown positive results. NBER has found that this goal can be accomplished as cheaply and as simply as sending a text message. For instance, a preschool literacy program called READY4K! sent parents three texts each week suggesting activities for building their children’s vocabularies, such as taking their young student to the library or pointing out words that rhyme. This program showed improved learning outcomes as well as increased parent engagement in home and schools and cost, on average, less than a dollar per family. While positive results were also found for programs that texted parents about student progress and attendance, these more personalized text messages require extended program management time and, as with personalized learning software, the scale up of operations may prove to be a challenge for Ed-tech entrepreneurs. The NBER review also noted that, while texts may be effective right now because they are a relatively new technology intervention, if parents are overwhelmed with texts then effectiveness could decrease.

Social and emotional learning might also be influenced by mindset interventions that encourage students to alter their attitudes toward challenges. Twelve NBER reviews found mostly positive results for online programs that gave students reading and writing prompts centered on facing adversity, rethinking setbacks, and overcoming feelings of isolation. In another LinkedIn article, I also talked about the potential of VR in this regard.

As with the potential of technology that personalizes learning, I am excited about the potential for learning that texting and mobile aps might have on education - particularly in the developing world. For instance, a 2015 Pew Research Center study found that, while only two percent of African households have a landline, cell phones are as common in Nigeria and South Africa as they are in the United States, with about 90 percent of adults owning mobile phones. Kenya as well has significant mobile telephone infrastructure. Pew also found that, while an average of 17 percent of people in Sub-Saharan Africa still do not own a cell phone, more than half of Sub-Saharan Africans sometimes have access to one. Quick Key, as an example, is capitalizing on cell phone use in Africa by partnering with Flying Kites, in Kenya, on helping teachers assess student learning and in more easily communicating with parents using mobile aps. Image: Lama Zoo.

Which Ed-Tech Merits Additional Study on Effectiveness?

For students in primary and secondary grades, studies revealed that providing free technology, like computers, had no positive or negative effects on academics. At the same time, however, the NBER report pointed out that free computer distribution programs, at a minimum, can help increase technology access for students who need it, such as the 67 percent of children who live in households with incomes less than $25,000 that do not have access to computers. Computers, at least, provide the basic infrastructure needed to participate in online coursework.

As early as 2012, several states already had either mandated or strongly recommended that high school students engage in online learning classes. At the same time, approximately 620,000 students were already taking at least one virtual class according to data collected by the Evergreen Education Group. The number of online learners has only increased in the intervening years. Given this, it is sobering that the authors of the NBER report have concluded that there is evidence that online-only courses at the college level can hurt rather than help student performance, while, at the high school level, programs aimed at credit recovery also show that online-only learning is not as helpful for students as is in-person instruction. Given the growth in the online learning market, it will be important that online learning be combined with in-person instruction given that the NBER report also found an association between higher student outcomes and blended learning approaches.

On the high school level, high schools might want to ensure that time be built into the daily schedule (i.e. through an online learning period and/or access to teachers providing online support during study hall) so that teaching staff can interface with students taking online courses to check on student progress and/or to address any concerns. High schools also might want to partner with a select number of online course providers to better manage the number of students taking online courses and the type/number of courses being offered to bolter student motivation as well as to limit the negative effects that learning in isolation can have on virtual students. For example, Windsor High School (CT), which serves 1,500 students in grades 9–12, started the Learning Lab - an online learning program to support students who were at risk of not graduating on time and to support students with special needs. Students in the Learning Lab used Edgenuity for credit recovery, initial credit, or supplemental learning. After one year of implementing Edgenuity courses, the graduation rate of special education students in the program increased from 64 percent in 2014 to 86 percent in 2015. The key to the program’s success, according to school administration, was the blending of online courses with in-person teacher support.

High schools that lack the resources to provide in-person supports - such as those offered to Windsor High School students taking courses online - might wish to partner with state authorized virtual schools since these schools often provide supports for high school students taking online courses. The North Carolina Virtual Public School, for instance, employs peer mentors - high school students who have successfully completed at least one online course offered by NCVPS - to assist new students as they progress through their online course work.