All school year, Kaylee Carrell has been watching online math videos using a free software platform called Algebra Nation.
What the Florida 8th grader didnât know: The software was also watching her.
As part of her nightly homework, Carrell might start a video, watch an instructor explain a concept, rewind to review, press pause when she was ready to solve a problem, and post messages to the Facebook-style âwallâ if she needed help. Occasionally, a brief survey might pop up.
Behind the scenes, the software was diligently tracking all that activity, anonymously logging the clicks and keystrokes of Carrell and more than 200,000 other students. As part of an $8.9 million federal grant project, researchers then used machine-learning techniques to search for patterns. Their ultimate goal: improve student learning by teaching the software to pinpoint when children are feeling happy, bored, or engaged.
Itâs just one example of a growing push to use educational technology to measure, monitor, and modify studentsâ emotions, mindsets, and ways of thinking.
The trend is provoking strong and conflicting reactions throughout the K-12 world.
âI can see how it could be really helpful,â Carrell said after a reporter explained the mechanics of the Algebra Nation research in which she was unknowingly taking part.
âBut home is also supposed to be a safe space. You donât want to feel like your computer is watching you.â
Personalized Learning for the âWhole Childâ
For years, thereâs been a movement to personalize student learning based on each childâs academic strengths, weaknesses, and preferences. Now, some experts believe such efforts shouldnât be limited to determining how well individual kids spell or subtract. To be effective, the thinking goes, schools also need to know when students are distracted, whether theyâre willing to embrace new challenges, and if they can control their impulses and empathize with the emotions of those around them.
To describe this constellation of traits and abilities, education experts use a host of often-overlapping terms, such as social-emotional skills, non-cognitive abilities, character traits, and executive functions.
For many parents and teachers, itâs common sense: Kids do well when they pay attention, work hard, and get along with others.
An emerging body of research backs that intuition, tying these non-academic factors to improved school achievement, future workplace success, and long-term well-being. Learning scientists are also increasingly convinced these traits and abilities can be improved with practice.
The result has been a groundswell of interest. A major international test of studentsâ social-emotional skills will be unveiled in 2019. The recently passed federal education law, known as the Every Student Succeeds Act, tried to push states and schools to broaden their definition of success. And major philanthropies and venture-capital firms are lining up to support the movement with hundreds of millions of dollars.
Sensing opportunity, a fresh crop of companies has sprouted up.
One of the most popular now administers online surveys to more than 7 million students a year, generating a massive database about childrenâs âgritâ and âgrowth mindset.â
Others claim they can improve childrenâs âimpulse controlâ through video games; provide parents with a âhigh-dimensional psychometric profileâ of their preschoolers; and allow school staffers to use smartphones to continually record their observations of studentsâ feelings.
Meanwhile, cutting-edge researchers are also exploring facial recognition, eye-tracking, wearable devices, and even virtual reality as ways to better gauge what students are feeling.
Proponents view this push to understand and respond to the âwhole childâ as a promising path to dramatically improving student learningâand an antidote to the K-12 worldâs longstanding focus on standardized tests.
Critics fear an Orwellian surveillance state, in which government and corporations alike invade studentsâ privacy, encroach on their individual liberty, and try to manipulate their behavior based on spurious measurements.
Recent fiascoes involving misuse of sensitive data harvested by consumer platforms such as Facebook have only amplified the stakes.
Add it all up, and the K-12 sector finds itself walking a tightrope.
âSocial-emotional learning is really about building relationships and communities,â said Jeremy Taylor, the director of assessment for the nonprofit Collaborative for Academic, Social, and Emotional Learning. âWe hope ed tech can support that, but we have to make sure things are done in a way that is responsible and not outpacing peopleâs comfort with what is happening in their schools,â
More Than Just a Test Score
Among the most prominent groups seeking to make sense of this new landscape is the Chan Zuckerberg Initiative.
A private venture-philanthropy organization founded by Facebook CEO Mark Zuckerberg and his wife, pediatrician Priscilla Chan, CZI plans to devote hundreds of millions of dollars per year to âwhole-child personalized learning.â
As part of that work, the group teamed up in May with the Bill & Melinda Gates Foundation to scour the field for the latest insights on how to measure and support studentsâ âexecutive functions,â such as the ability to focus and filter out distractions.
Jim Shelton, who heads Chan Zuckerbergâs education efforts, explained his thinking in an interview.
âKids have physical development, mental-health development, identity development, cognitive-skills development, and social-emotional skills development, as well as academic-skill development,â Shelton said. âUnless you understand how they are relating to each other, it is very difficult to optimize any of them.â
Creating that kind of 360-degree view of each individual child is the key to reaching millions of students who have been failed by the existing education system, Shelton believes. Technology may eventually help make such a holistic understanding possible at scale, he said, but efforts to develop such tools are still in their infancy.
Nevertheless, real companies are already working to pursue various parts of what Shelton described.
Take Panorama Education. Founded in 2012, the data-analytics company has raised $32 million in venture capital (including multiple rounds from CZI.)
It already works with 500 school systems, including the 29,000-student Spokane, Wash., district.
In addition to questions about school safety and their relationships with teachers, all of Spokaneâs 4th to 12th graders now take online surveys asking things like:
⢠In school, how possible is it for you to change how easily you give up?
⢠How often do you stay focused on the same goal for several months at a time?
⢠During the past 30 days, how carefully did you listen to other peopleâs points of view?
âWe have a bigger responsibility than just cranking out kids who can pass tests.â
âTravis Schulhauser,
director of assessment and instructional technology, Spokane Public 69´ŤĂ˝
Panorama stores the responses in a central database, then feeds the information back to Spokane administrators and educators via customized dashboards.
As part of this yearâs pilot, a small group of Spokane teachers was able to see how each individual child in their classrooms scored on a 0-5 scale for the social-emotional domain corresponding to each question: grit, growth mindset, and social awareness.
Principals and counselors also analyzed the Panorama data alongside other information, such as class rosters, to identify students with an attitude that might help them thrive in Advanced Placement courses, even if they didnât have the highest PSAT scores. And district leaders examined how their schools compared with national benchmarksâwhich Panorama develops by analyzing the anonymized data of millions of students who take its surveys each year.
Spokane officials describe their multi-pronged use of Panorama data as key to making better decisions.
âAs we get more and more information from kids, I think weâll respond to each student better and better,â said Travis Schulhauser, the districtâs director of assessment and instructional technology.
Plenty of schools are embracing social-emotional learning without using technology or trying to measure individual studentsâ development.
New research has raised questions about whether classroom-based interventions around concepts like growth mindset are likely to have a significant impact.
And some of the leading pioneers in the field have warned about the limits of using surveys and questionnaires to measure social-emotional skills. Even Shelton said CZI invested in the company primarily because of its surveys related to school climate and cultureâand that he wasnât familiar enough with its âentire product suiteâ to say whether it provides âa valid and reliable measure of a developmental area.â
Regardless, Spokane is just one of hundreds of districts embracing the digital tools and data theyâre currently being offered.
âWe have a bigger responsibility than just cranking out kids who can pass tests,â Schulhauser said. âWe take seriously our responsibility to develop students who are well-rounded and able to achieve their hopes and dreams.â
âJust Un-Americanâ
Why wouldnât someone want schools to look beyond test scores, or use technology to encourage kids to persevere in class and build healthy relationships with peers?
âSorry, but I donât want the government knowing how my childâs mind works,â said Jane Robbins, an attorney and senior fellow with the American Principles Project Foundation, a think tank that promotes individual liberty.
Like many conservative and libertarian parents and activists, Robbins is strongly opposed to using technology to track student emotions and mindsets.
An individual teacher encouraging students to try hard is one thing, she said. But surveys like Panoramaâs are âborderline mental-health assessments,â she believes, and they encourage a view of children as potential patients in need of treatment.
In addition, Robbins argued, public schools are an extension of government. And government should not be trying to monitor and mold what individual citizens feel.
Plus, some critics on the right view the social-emotional learning movement as a thinly disguised effort to promote ostensibly left-wing causes, such as gay and transgender rights. (âMight a studentâs ârelationship skillsâ be deemed deficient if, in keeping with the influence of his family and faith, he rejects the LGBT agenda such as same-sex marriage and normalization of gender dysphoria?â asked conservative activist Karen Effrem in a 2016 essay, â69´ŤĂ˝ Ditch Academics for Emotional Manipulation.â)
And all those concerns get amplified dramatically when large-scale digital data collection is added to the mix.
âYou donât think these databases are going to be interesting down the line to employers, or prosecutors?â Robbins asked.
Given such opposition, itâs not hard to imagine the reaction to newer entries into the social-emotional learning space, such as San Francisco-based startup Emote.
The companyâs core service is a mobile app that makes it easier for a wide range of school staff, from bus drivers to teachers, to record and share their observations of when students appear sad, anxious, angry, and frustrated.
Launched in 2016 with $120,000 in support from Y Combinator, one of the most high-profile âacceleratorsâ in Silicon Valley, Emote is already used by 16 districts in more than a dozen states.
How does it work?
Imagine a student gets in an argument with his parent before school, said Julian Golder, the companyâs 32-year old CEO. That morning, the boy stomps into the school building. A staff member at the front desk notices, so she enters the observation directly into the Emote app on her smartphone, selecting from a menu of keywords (âsadâ) and a color-coded scale (âblue.â)
That, in turn, prompts a notification to be sent to each of the studentâs teachers.
If those teachers then notice something similarâthe student has his head down in 1st period, or seems disconnected in 3rd periodâthey also record their observations in the app.
If a pattern emerges, Golder said, the student may be at risk of âescalation,â such as getting into a fight or flunking his 5th period math test.
Despite their best efforts, he said, schools often miss these kinds of developments in the moment, leaving them to respond to the trouble after it happens.
But Emote helps adults notice and communicate about kidsâ feelings and behaviors in real time, Golder said. That way, they can respond before something bad happens.
âSorry, but I donât want the government knowing how my childâs mind works.â
âJane Robbins,
attorney and senior fellow, American Principles Project Foundation
The company also allows schools to track studentsâ feelings longitudinally: Maybe the student feels angry and disconnected every Monday morning because thereâs some underlying issue at home that needs to be addressed? Or perhaps the schoolâs African-American boys are consistently feeling disconnected after a particular class, and some observations of the teacher are warranted?
âI think this is a really exciting vision of what school can look like,â Golder said.
âThereâs more interest than we can handle at this point.â
Still, Robbins of the American Principles Project Foundation isnât buying it.
âThe idea of telling children that even their feelings are not private, and that weâre going to constantly surveil them and analyze them, is just un-American,â Robbins said. âThe only good thing I can see about this company is that theyâre not hooking kids up to wearables.â
âExpand the Realm of the Possibleâ
Broadly speaking, however, thatâs exactly where the field appears to be heading.
In 2016, the World Economic Forum and the Boston Consulting Group issued a report on the future of ed tech and social-emotional learning. It highlighted wearable devices as a way to âexpand the realm of the possible.â
Among the companies trying to deliver on that promise: Cambridge, U.K.-based Tinylogics, which is currently testing a wearable it calls FOCI. When clipped to a waistband, the device will track usersâ breathing patterns, then tell them when theyâre feeling focused, relaxed, fatigued, or stressed. Tinylogics describes an accompanying app as a âfocus-enhancing mind coach.â A recent press release from the company billed the FOCI as a tool for managing digital distractions in the classroom.
Prominent ed-tech venture-capitalists are also touting virtual reality as an âempathy technology.â A company called Mursion, for example, has been working with the Alexandria, Va., public schools to use immersive VR simulations as part of an effort to help students with autism develop their social-emotional learning skills.
And the World Economic Forum and the Boston Consulting Group also touted the field of âaffective computing,â in which machines are trained to recognize, interpret, and simulate human emotions.
Which brings the story back to West Palm Beach and Algebra Nation.
The effort to use clickstream data from the online platform in order to identify student emotions and engagement is being led by University of Colorado Boulder professor Sidney DâMello, a national leader in the affective-computing field.
So far, all the information harvested by the team on the Algebra Nation project has been collected anonymously, in a way that is âintentionally insufficient to build affective profiles for individual students (which is not our goal at all),â DâMello wrote in an email.
Because of that approach, the institutional review board tasked with approving the project determined that no parental consent was yet needed.
But that could change soon.
Beginning in spring 2019, pending the review boardâs approval of a series of new consent protocols, some students may interact with the current version of Algebra Nation. Others might interact with a version of the software that will analyze their individual click patterns in real-time to identify when theyâre becoming bored or frustrated. If the predictions prove sufficiently accurate, it will respond in the moment with personalized prompts or supports. The researchers will see if the software that adapts to studentsâ emotional states helps them learn algebra better.
At Conniston Middle School, Kaylee Carrell and her classmates had nuanced reactions, calling the idea both âcoolâ and âcreepy.â
The one place where students consistently drew a line: facial recognition.
âI would feel like Iâm being watched, like someone is spying on me,â said 8th grader Merlin Aguilar.
But even there, related technology is already being used in K-12.
Through the Emotive Computing Lab that DâMello runs, for example, heâs conducting a separate research project using eye-trackers and webcams to track consenting studentsâ eye movements and facial expressions. Algorithms examine the resulting data to determine when kidsâ minds are wandering. If a student is zoning out, the software will intervene.
DâMello acknowledged the privacy concerns associated with such technologies. But he said some level of risk is necessary in order to pursue a promising new vision of education.
âIf I could always have teachers that are adapting to me, looking at my mistakes, giving me motivation and supports, and then backing away and giving me room, I think Iâd be a much better learner,â he said.
Clearly, that view has support from high places. The Algebra Nation project, for example, is being funded with an $8.9 million grant from the federal Institute of Education Sciences, the research arm of the U.S. Department of Education.
But recent controversies in other sectors have started to cast the use of technology to track student emotions in a critical new light.
Take the recent Facebook-Cambridge Analytica scandal, in which millionsâ of users had sensitive information about their preferences and personalities misused as part of an effort to influence their votes during the 2016 presidential election.
The resulting headlines and Congressional hearings have helped make the K-12 world more aware that similar technologies are already making their way into schools, said Ben Williamson, a lecturer at the University of Stirling in the United Kingdom.
That, in turn, is heightening awareness of the possibility of unintended consequences.
Data breaches are the most obvious potential downside, Williamson said, although he wasnât aware of any such incidents involving emotional or affective information collected from students.
But more pernicious, Williamson believes, is the potential for âpsycho-compulsion and behavior modification.â
âIf you generate detailed information about studentsâ feelings, then it becomes possible to target them in sophisticated ways in order to nudge them to behave in ways that conform with a particular, idealized model of a âgood student,ââ Williamson said.
For supporters, thatâs part of the promise: The possibility that technology might help students develop grit and focusâand improve their grades, job prospects, and long-term healthâis reason to invest millions now.
For critics, though, itâs the wrong end of a slippery slope: Government agencies and Silicon Valley companies deciding how students should be thinking and what they should be feelingâthen collecting massive amounts of data and deploying invisible algorithms to enact that agendaâis something to be fought now, before the horse is all the way out of the barn.
The one thing both sides agree on?
âThe technology is powerful,â Williamson said, âand it could have real consequences.â
