So-called “brain training” programs designed to help boost students’ attention and working memory are coming under scrutiny from the Federal Trade Commission, as some claims have outpaced both the initial hopes and subsequent evidence.
Last year, the FTC declared open season on brain-training programs, both those directed at classrooms and for general public use. Last month, it announced a $2 million settlement and $50 million judgement for misleading advertising against Lumos Labs Inc., the creators of Lumosity, a suite of computer- and app-based brain-training programs. A judge found that the company’s in-house research did not prove its video games could “improve school, work, and athletic performance” or protect against cognitive impairments caused by attention deficit disorders, traumatic brain injury, pediatric chemotherapy, or adult dementia as its advertisements had claimed.
Earlier in the year, the FTC also settled with the Texas-based Focus Education, which inappropriately claimed its Jungle Rangers computer game had “scientifically proven memory- and attention-training exercises” that give students “the ability to focus, complete school work, homework, and to stay on task.”
The Every Student Succeeds Act has created a tiered evidence structure to help states and districts evaluate programs’ effectiveness in nascent research areas like memory training, but actions against Lumosity and Jungle Rangers give warning to researchers and educators alike that the promise of some interventions can turn into hype.
“We are hoping to crack down on overhyped claims in this area,” said Michelle Rusk, a lawyer in the FTC’s bureau of consumer protection. “We may in the next year or two be bringing other cases for similar kinds of brain training for children.”
Booming Industry
Lumosity is one of the best-known of a slew of new app- and computer-based programs intended to help students improve working memory, the system the brain uses to hold information during decisionmaking and analysis.
Working-memory capacity naturally expands as a child grows—12-year-olds can handle about twice as many individual pieces of information as 7-year-olds, for example. In the early 2000s, neuroscientists and cognitive psychologists studying neuroplasticity found evidence to suggest working-memory capacity could be expanded in other ways, too.
“Originally, we were really excited about this; we found what we thought was [learning] transfer to other tasks,” said Michael Dougherty, a professor of psychology and the director of the Decision, Attention, and Memory Lab at the University of Maryland, College Park. He was one of the “optimists in the field,” he said, but did not work on projects for Lumos Labs or Focus Education.
Annual sales of brain-training products in the United States and Canada climbed past $67 million last year, according to a recent analysis by MarketsandMarkets, a private business research group.
Worldwide sales of such products were more than three times higher, approaching $227 million.
SOURCE: MarketsandMarkets
“The claims of scientists and the claims of companies—I don’t think they differ too much, unfortunately,” Dougherty said. “I think these initial promising results were taken to mean there were very strong cognitive results and changing [those indicators] could change your life.”
Since 2013, the business research firm MarketsandMarkets estimates the “brain training” market has grown from $48.5 million in annual revenues to more than $67 million in North America alone, and it is projected to grow to nearly $200 million by 2020. Worldwide, the market could top $700 million, it says.
Lumos Labs declined to comment for this story, but its advertisements said outright: “Brain training has the potential to change lives.” Through the Lumosity Education Access Program, Lumos Labs provided free copies of the games to students in more than 500 schools, in part to help build a research base to prove the interventions’ effectiveness. The company also built up a network of researchers, dubbed the Human Cognition Project, to dig through the data, and presented its research at conferences such as the Society for Neuroscience.
Research Rules
In FTC documents, investigators noted Lumosity offers 40 different games that purported to target specific brain areas, using 10- to 15-minute sessions several days a week (at costs ranging from $14.95 a month to nearly $300 for a lifetime subscription). Advertisements pointed to studies that suggested the training could improve students’ math skills, including for girls with Turner syndrome, a genetic disorder that can result in cognitive impairments, particularly in math. Advertisements also noted a study of more than 1,200 students in 40 schools. Those who used Lumosity to supplement their regular curriculum, the advertisement said, “improved more than a control group on a battery of cognitive assessments.”
That’s true, as far as it goes. But Lumosity’s studies and many others across the brain-training field had increasing problems showing that the improvements in scores on their games actually transferred to better learning, memory, and attention beyond them.
“Lumosity generated millions and millions in sales and spent millions and millions in marketing, so they clearly had the money to do a couple of high-quality studies,” Rusk said. “The best way to stay on the right side of the law is to be very careful not to get ahead of where the science is.”
Eric B. Gordon, the president and CEO of Atentiv, another cognitive-improvement program, did not comment on the FTC’s settlements but said his own company has been building up evidence on Atentiv using clinical, home, and school studies, and that any communications on its programs goes through “independent scientific and regulatory approval” before being released.
“Only good practices deliver valuable and viable health-care solutions. Evidence comes first,” he said.
But big, generalizable cognitive improvements have been much harder to come by than researchers and educators initially hoped.
In a 2013 meta-analysis in the journal Developmental Psychology, researchers at the University of Oslo, Norway, looked at 23 studies on working-memory training. They found many programs did improve short-term working-memory skills, but the improvements were not sustained, and the skills did not translate into more general cognitive skills associated with working memory, such as attention control, verbal and nonverbal ability, or word decoding.
In the same year, Thomas Redick and colleagues at the Purdue University Applied Cognition Lab found no evidence of improvement in general cognition after 20 sessions of a working-memory training program that used a structure that is similar to many games: the “N-Back test.” The most basic form of the task requires a student to remember a target, such as a number or word, which appears in a series, while dealing with distractions.
Finding Broader Learning
Dougherty found similar problems in his own experiments: 69ý and adults regularly improved in speed and accuracy on training tasks like these, but the further the task was from complex, real-life attention and decisionmaking, the less those benefits transferred.
“It would be great if we could move everybody’s IQ by 2 or 3 points—it would be amazing,” Dougherty said, “and I was very optimistic, and then in study after study, both in my own labs and other people’s, we just didn’t find it.”
In 2014, the Stanford University Center on Longevity and Germany’s Max Planck Institute for Human Development convened international cognitive psychologists and neuroscientists to hash out a consensus on the state of evidence on brain training, which was blunt: “Claims promoting brain games are frequently exaggerated and at times misleading. ... To date, there is little evidence that playing brain games improves underlying broad cognitive abilities, or that it enables one to better navigate a complex realm of everyday life.”
An intervention that meets ESSA’s rules for research evidence on some level could still include claims on promotional materials that run afoul of the FTC’s rules, which are based on the federal Unfair Claims Act and are more black-and-white than the new education research requirements. The agency can consider a claim misleading if it doesn’t rely on “competent and reliable scientific evidence,” based on “human clinical testing ... randomized, double-blind, and adequately controlled; and conducted by researchers qualified by training and experience to conduct such testing.” That’s roughly equivalent to ESSA’s top tier of evidence, but the education law lays out lower tiers which states can choose to use—particularly in areas of nascent research.
Rusk of the FTC said the agency would continue its investigations, but warned school leaders to scrutinize any program claiming to improve students’ intelligence or attention. “The more dramatic the improvements promised, the more skeptical they should be,” she said. “It is the case unfortunately that we have to go after these companies one at a time, and we don’t have the resources to investigate every product on the market, so it does require some due diligence on the part of the educator.”
The scientists who developed the Stanford-Planck consensus statement recommended that future cognitive research on brain training also examine “opportunity costs ... If an hour spent doing solo software drills is an hour not spent hiking, learning Italian, making a new recipe or playing with your grandchildren, it may not be worth it.”
They and Dougherty also noted that the one activity so far that has been shown to have some improvement on general cognition is physical exercise—the opposite of sitting in front of a computer screen or app.
