Corrected: A previous version of this story cited an incorrect figure for the share of K-2 early reading and special education teachers who said they use Fountas and Pinnell鈥檚 Leveled Literacy Intervention.
Two of the nation鈥檚 most popular early literacy programs that have been at the center of a debate over how to best teach reading both faced more new critiques in the past few weeks, receiving bottom marks on an outside evaluation of their materials.
EdReports鈥攁 nonprofit organization that reviews K-12 instructional materials in English/language arts, math, and science鈥� Tuesday, finding that the program didn鈥檛 meet expectations for text quality or alignment to standards. The release comes on the heels of the group鈥檚 from the Teachers College 69传媒 and Writing Project, another popular early reading program.
Together, the two reports received the lowest ratings EdReports has given for K-2 curricula in English/language arts, and they鈥檙e among the three lowest for ELA in grades 3-8.
鈥淭he materials don鈥檛 reflect the shifts鈥攖ext quality and complexity鈥攅specially in K-2,鈥� said Stephanie Stephens, EdReports鈥� ELA content specialist for early literacy, referencing key components of the Common Core State Standards鈥攁 big part of the organization鈥檚 review criteria.
These two literacy brands, both published by Heinemann, command large shares of the early reading market.
In 2019, a nationally representative EdWeek Research Center survey found that 43 percent of K-2 early reading and special education teachers use Fountas and Pinnell鈥檚 Leveled Literacy Intervention, the intervention companion to Fountas and Pinnell Classroom. The same survey found that 16 percent of teachers used the Units of Study for Teaching 69传媒.
Recently, these programs have faced criticism from educators and researchers that the instructional methods they use don鈥檛 align with, or in some cases contradict, the research on how to develop strong readers. Fountas and Pinnell has pushed back against these characterizations. Lucy Calkins, the director of the Teachers College 69传媒 and Writing Project, has announced an upcoming revision to the Units of Study, set to be released in summer 2022. (EdReports reviewed the current version of the materials.)
How these programs attend to foundational skills鈥攖eaching students to recognize and manipulate the sounds in words, and then matching those sounds to written letters鈥攊s one of the main focuses of the critique. It鈥檚 also something that EdReports turned a renewed attention toward.
Fountas and Pinnell Classroom and Units of Study are two of the three K-2 reading programs to have gone through EdReports鈥� updated review tools for English/language arts, which teaching. These new evaluation criteria also look for what EdReports calls 鈥渂loat,鈥� whether all of the content in a set of materials can be taught in one year. Open Court, the third program evaluated with these new tools, partially met expectations.
In its two responses to the reviews on the EdReports website, Heinemann wrote that the EdReports鈥� rubrics aren鈥檛 a good fit for programs like Fountas and Pinnell Classroom and Units of Study.
鈥淔PC greatly values the importance of responsive teaching and the teacher agency required to adjust, extend, and enrich learning based on individual student needs,鈥� reads one response. 鈥淭he EdReports rubric provides no way to measure these deeply valuable components of an effective literacy system.鈥�
This ethos of teacher agency is one of the reasons that Fountas and Pinnell and the Units of Study are so popular with educators. Both give teachers, and oftentimes students, choice over materials and activities. Still, the EdReports reviews could affect whether schools continue to use them, said Morgan Polikoff, an associate professor of education at USC Rossier who studies K-12 curriculum and standards.
The reviews may influence state-level recommendations, or district leaders might reference them the next time they have to choose curriculum, Polikoff said. It鈥檚 also possible that parent advocates, like , could use these reviews. 鈥淎 bad EdReports rating could be another piece of evidence that those parents could potentially bring to bear鈥� in attempts to jettison these programs, he said.
Kareem Weaver, a member of the Oakland NAACP Education Committee and the co-founder of FULCRUM, an Oakland group that advocates for evidence-based literacy instruction, said that the reviews could provide the impetus for school districts to reconsider the use of programs that he says don鈥檛 work for all kids.
鈥淚鈥檓 really hoping it will make people do a double take,鈥� he said.
Reviews critique text complexity, foundational skills
Since its launch in 2015, EdReports has recruited educators鈥攖eachers and other instructional leaders鈥攖o conduct its reviews, and to develop the rubrics used to judge materials. These rubrics measure alignment to the Common Core State Standards, usability in a classroom setting, and other indicators of quality, such as text complexity.
The company is one of the few organizations that provides external evaluations of curricula, and its reviews have a wide reach: As of 2020, EdReports said that at least 1,084 districts use its reviews, including 89 of the 200 largest districts in the country. (There are about 13,400 school districts in the United States.)
Still, not everyone agrees with EdReports鈥� conclusions. Publishers have critiqued the group鈥檚 methodology and rating system in the past, claiming that reviewers failed to consider supplemental materials and taking issue with the organization鈥檚 鈥済ateway鈥� system, which requires that a program meet the standards set for alignment before it can be evaluated on other features. EdReports made a few changes to its process after publishers pushed back on its first set of math reviews, though the gateway system remains.
Fountas and Pinnell Classroom failed to pass the first gateway. In K-2, reviewers said that core texts didn鈥檛 meet standards for quality or complexity, and that speaking and writing assignments didn鈥檛 require students to use evidence from the texts they read. EdReports also critiqued Fountas and Pinnell鈥檚 text leveling system, which it said was 鈥渘ot accompanied by an accurate text complexity analysis and a rationale for educational purpose and placement in the grade level.鈥� The group gave a similar evaluation for the program in grades 3-8.
While the K-2 program鈥檚 word study lessons teach phonics, 鈥渢he program does not present a research-based or evidence-based explanation for the sequence鈥� of instruction, reviewers found. The report also claims that the program doesn鈥檛 consistently devote enough time to systematic instruction in phonological awareness, phonics, and fluency.
Units of Study also didn鈥檛 pass EdReports鈥� first gateway, which measures alignment to the common core. For grades K-2, reviewers said that texts featured in the materials 鈥渁re not appropriately complex for the grade level and do not build in complexity over the course of the year.鈥� They also noted that the program focused mostly on reading skills instruction, rather than 鈥渜uestions and tasks aligned to grade-level standards,鈥� like asking students to use information from the text to support opinions.
Instruction in foundational reading skills like phonological awareness and phonics, they said, 鈥渓acks a cohesive and intentional scope and sequence.鈥� The review also notes that the materials rely on cueing strategies for word identification: prompting students to draw on pictures, context, and sentence structure鈥攁long with letters鈥攖o figure out what words say. But research has shown that pulling students鈥� attention away from the letters can lower the chances that they鈥檒l use their knowledge of letter-sound correspondences to read through a word, making it less likely that they鈥檒l be able to map the spelling to the spoken word in their memory.
Reviewers found text complexity lacking in grades 3-8, as well, and they said that the program lacks 鈥渁 variety of regular, standards-aligned, text-based listening and speaking opportunities,鈥� as well as opportunities for on-demand writing and systematic vocabulary development.
Not every program reviewed against EdReports鈥� new rubric received low marks. Open Court, the third program reviewed with the new tools, fared better. It partially met expectations at the first gateway, and also at the second gateway, which measures knowledge building. In grades K-2, reviewers reported a research-based approach to foundational skills instruction, but noted that there wasn鈥檛 enough practice with encoding鈥攈earing sounds and converting them into written language.
Reviewers said that only some texts were 鈥渁ppropriately complex for the grade level,鈥� and also said that the program missed opportunities for standards-aligned activities. They had the same critique for the grades 3-5 materials.
Publishers claim that EdReports tool is mismatched to their approach
As part of the review process, EdReports solicits publisher responses to its evaluations, posted publicly on its website. McGraw Hill, which publishes Open Court, and Heinemann both critiqued the review process in their responses.
The McGraw Hill response claimed that EdReports had overlooked end-of-unit opportunities for students to demonstrate knowledge, citing the curriculum鈥檚 unit-long 鈥淚nquiry鈥� process.
Heinemann criticized the EdReports review process for omitting texts that students read outside of whole-group instruction.
In Units of Study, students only spend limited time in a whole-group 鈥渕inilesson,鈥� before moving on to the reading workshop, during which they apply the skills taught in the minilesson to independent reading, reading with a partner, or working with the teacher one-on-one or in small groups. FPC is structured similarly, with whole-class minilessons but also guided reading, independent reading, and student book clubs.
Heinemann鈥檚 responses argue that EdReports鈥� review design prioritizes textbook-style reading curricula, and fails to capture the quality of texts that students might read on their own or in small groups. The publisher did not respond to EdReports鈥� critiques of foundational skills instruction.
Stephens, of EdReports, said the group is not discriminating based on design and approach, but rather evaluating whether students have guaranteed access to grade-level text. 鈥淚f they鈥檙e using independent reading at their level, there鈥檚 not a guarantee that鈥檚 at grade level,鈥� she said.
Separately, Irene Fountas and Gay Su Pinnell, the program鈥檚 namesakes and founding authors, have begun to publish a 10-part blog series rebutting claims that their program is not aligned to reading science.
In , the authors defend their program鈥檚 use of cueing and other strategies that are central to their materials but which studies have shows are ineffective, like leveled reading groups.
鈥淚f a reader says 鈥榩ony鈥� for 鈥榟orse鈥� because of information from the pictures, that tells the teacher that the reader is using meaning information from the pictures, as well as the structure of the language, but is neglecting to use the visual information of the print,鈥� one of the blogs reads. 鈥淗is response is partially correct, but the teacher needs to guide him to stop and work for accuracy.鈥� This idea is in direct contrast to what most cognitive scientists say about how strong readers process new words.
The Teachers College 69传媒 and Writing Project, which writes the Units of Study, has also separately responded to the EdReports reviews. A argues that the program has a different approach to meeting common-core standards than EdReports does. 鈥淎t a fundamental level, ours is a paradigm where choice matters, where agency matters. EdReports uses a rubric that does not value those things.鈥� TCRWP cited, for example, that when teachers were provided with a choice to assign on-demand writing, EdReports didn鈥檛 award full marks because the writing was not a requirement.
鈥淭his is always the challenge of applying a rubric to things that differ in a lot of ways. It鈥檚 an imperfect science,鈥� said Polikoff, of USC Rossier. 鈥淭he question is, is it better than not having it? And to me, the answer is yes.鈥�
EdReports is working with one set of criteria, and can give teachers information about how programs line up according to that criteria鈥攊nformation that is often hard to come by, Polikoff said. There aren鈥檛 many avenues for teachers to find third-party evaluations of materials, he added.
Matthew Alexander, the director of elementary literacy and numeracy for Hall County 69传媒 in Gainesville, Ga., said his district relies both on outside evaluation and internal data in making decisions about what programs to use.
Hall County uses one piece of Fountas and Pinnell Classroom鈥攖he Phonics, Spelling, and Word Study component鈥攁cross its 20 elementary schools. The district also use its Benchmark Assessment System.
Alexander plans to discuss the review with other leaders in the school system, as it relates to their phonics instruction. But he鈥檚 hesitant to make any quick changes, because Hall County only started using Phonics, Spelling, and Word Study in the 2019-20 school year, right before the pandemic hit.
鈥淚f we were seeing that in our schools, that our kids were not making gains as readers, we would certainly look to see if we would shift our resources in a different direction. But with just three years of non-typical data, it鈥檚 hard to make that statement,鈥� Alexander said.
Review tool changes address foundational skills, program 鈥榖loat鈥�
The low ratings on some indicators in these reviews stem from changes to EdReports鈥� review tools.
In 2020, EdReports announced its first revision to its criteria and its evidence guides鈥攁 sort of handbook for reviewers that helps them identify evidence that programs meet, or don鈥檛 meet, the criteria. Part of this update are two key changes to how reviewers evaluate English/language arts materials.
One has to do with how reviewers approach foundational skills instruction in K-5. Criteria and evidence guides are more specific about when and how these skills should be taught.
For example, criteria that require systematic and explicit teaching in the alphabetic principle, phonemic awareness, phonics, and other skills has now been split into four subcategories, each with its own grade-by-grade breakdown of what students should be able to do in the evidence guide.
EdReports has also cut guidance that says programs 鈥渟hould instruct the teacher to employ syntactic or semantic cueing systems when the phonics patterns do not work or to confirm a word choice.鈥� These changes have come as and the work of reading researchers have turned increased public scrutiny toward cueing over the past few years.
The revision brings the comprehensive ELA reviews more in line with the stricter criteria in stand-alone reviews of foundational skills, which EdReports launched in 2019, said Stephens. This way, she said, comprehensive reading programs will be judged as rigorously on their foundational skills components.
Still, Stephens thinks that the programs reviewed under the revised tools would have fared similarly under the originals. The revision provided 鈥渃larity,鈥� she said, rather than an entire new scoring system.
The other change to the review process concerns what EdReports calls program 鈥渂loat.鈥�
If a program says, for example, take 15 minutes a day for reading and 20 minutes for foundational skills, is that actually doable with the materials provided? Or is there too much content to feasibly get through? The program should offer a 鈥渃lear and concise鈥� pathway through the standards, Stephens said.
EdReports has also made some changes to its math review process, and has updated its criteria for gateway 3, which measures usability, across all subjects.
Louisa Moats, an early literacy expert and the lead writer of LETRS, a professional development program for reading teachers, . She said that the new review tools are more closely aligned with research-based practice in reading instruction.
鈥淭hese standards are much better for identifying practices in programs that are wildly off base. They鈥檙e a pretty good firewall in recognizing the programs that are wildly misaligned with reading science and with practices that have been shown to be ineffective with most kids. 鈥� That鈥檚 really good,鈥� she said.
Still, she said, even if a program passes the review, its success or failure is going to come down to how the skills are taught in the classroom.
Weaver, in Oakland, said that the field needs more information about the effectiveness of popular reading materials. 鈥淲hat [EdReports] doesn鈥檛 do is it doesn鈥檛 talk about student achievement results. It doesn鈥檛 talk about how kids do with the program. And that鈥檚 fine, because they don鈥檛 claim to do that. But a lot of districts think they do,鈥� he said.
鈥淎lignment with the standards is the bare minimum that we should be able to expect from the curriculum,鈥� Weaver said.
