Ask teachers whether the drive to raise academic standards in the nation’s schools is a “move in the right direction,” and the overwhelming majority answer yes. But ask teachers whether instruction focuses too much on the tests used to measure those standards, whether they have enough time to teach all the expected knowledge and skills, or whether they have enough materials and training to do the job, and their responses are more critical.
After more than a decade, standards-based initiatives to improve schools are, indeed, reaching into classrooms. While some studies suggest the results have been positive--including a more demanding curriculum and better test scores--evidence also suggests that teachers are being pushed to place too much emphasis on tests and test-preparation activities.
This fifth edition of Quality Counts tries to provide a classroom-eye view of how standards-based education is playing out. It is based on a new, nationally representative survey of public school teachers conducted for Education Week.
The report also includes the most comprehensive survey to date of what the 50 states are doing to set academic standards, create testing and accountability systems linked to those standards, and provide support for students and educators to reach the higher expectations.
Quality Counts 2001 comes at a crucial juncture. The vast majority of states have been working diligently on policies related to standards-based changes. Almost all now have standards for what students should know in core subjects, tests to measure student learning, and at least the beginnings of an accountability system to hold schools responsible for results. But some large pieces of the puzzle are still missing.
In particular, the assessments now being used to judge whether students and schools have met state standards leave much to be desired. In too many states, the tests still focus too much on low-level, multiple-choice questions and are poorly aligned with the standards they are designed to measure. In too many states, students’ academic fates rest too heavily on performance on a single test. And, in some states, the passing scores for students may be set so high that thousands of youngsters risk repeating a grade or failing to earn a diploma.
Moreover, while states have begun to provide materials, training, and other standards-related help to teachers, students, and schools that fare poorly on state exams, those efforts have only scratched the surface.
The conclusion is simple and one voiced repeatedly by classroom teachers: If states really want to improve teaching and learning, they must find a better balance among standards, tests, and the support needed to do the job.
“Everything that we do in our classroom has to revolve around the standards,” says Solange M. Pohlman, a 7th grade teacher at Brentwood Middle School in Charleston, S.C. “It’s given us a good guideline, a good basis to go by.”
But, she adds, “I don’t feel like there’s been enough training. I don’t feel like we have the materials--the hands-on things--to do it.”
Conversation Shifts
The amount of ground traveled by states since the early 1980s is striking. At that time, most states were focused on minimum-competency tests for students to ensure that they had mastered basic reading and math skills before graduating from high school. Few states were committed to significantly higher expectations for all students--a goal that has since become a mantra of the school improvement movement.
“I think the most powerful, positive effect of standards-based reform has been that it’s focused attention on student learning and shifted the conversation from one about different groups of students’ meeting different standards to all students’ being taught to meet high standards,” says Warren Simmons, the director of the Annenberg Institute for School Reform. “That’s a fundamental sea change in the goals of American education.”
The push to improve education based on more rigorous academic standards dates back at least to the late 1980s. In 1989, the nation’s governors and then-President George Bush agreed to set ambitious education goals for the nation. With the goals Bush unveiled the following year, the leaders pledged that by 2000, among other objectives, American students would leave grades 4, 8, and 12 having demonstrated competency in challenging subject matter in English, mathematics, science, history, and geography.
A panel of experts convened by Congress concluded in 1992 that to measure progress toward the goals in a meaningful way, Americans should consider the creation of national education standards and a voluntary system of assessments to measure whether students had met those standards. “In the absence of well-defined and demanding standards, education in the United States has gravitated toward de facto national minimum expectations,” warned the National Council on Education Standards and Testing, “with curricula focusing on low-level reading and arithmetic skills and on small amounts of factual material in other content areas.”
Today, the idea of a single set of national standards and tests for all remains mired in controversy, making both unlikely to get off the ground. And the national education goals are already fading into memory, a year after their target date passed with the goals unmet.
But states have embraced the broad feature of standards-based education with astonishing alacrity.
Forty-nine states now have statewide academic standard for what students should know and be able to do in at least some subjects; 50 states test how well their students are learning; and 27 hold schools accountable for results, either by rating the performance of all their schools or identifying low-performing ones.
All of which poses critical questions: Is standards-based reform working? Are schools actually changing in response to state standards, and are students learning more? Have states adopted the right balance of policies to arrive at their desired goals?
Critics assert that the whole premise of standards-based school improvement is fundamentally misguided because it shifts the locus of control from individual schools, teachers, and parents to distant state bureaucrats. They also contend that the tests being used to drive higher expectations are, ironically, squeezing out high-quality instruction, and that the standards are producing mindless and undesirable standardization in schools. Equally important, they say, states are highlighting the wrong problem. If they really wanted to improve learning, critics suggest, states would pay more attention to the quality of teaching, inequities in school financing, and the excessive size of many high schools.
Conversely, supporters of the standards movement point to rising test scores in some states and districts as proof that standards are on the right track. They argue that the emphasis on results has caused educators to zero in on academics and to provide more support for low-performing students and schools. And, they maintain, the focus on standards has encouraged states to increase funding for education and to devote greater attention to teacher quality.
Trying to find a firm footing across such a philosophical divide is difficult, particularly because the details of state policies and practices vary so widely.
I think [standards] are making all teachers more aware of what needs to be taught.”
Some states set standards in only a few subjects; rely primarily on commercially developed, multiple-choice tests to measure whether students are learning; and do not reward or punish schools based on performance. Other states set standards in English, math, science, social studies, and more; test every student every year in grades 3-10; and use test results to determine whether students graduate or schools are labeled as failing. While some states have been working on a set of standards-related policies for more than a decade, others have barely begun. All of which make it hard to generalize about the positive or negative effects of the effort as a whole.
The Rumblings of Change
Even so, some patterns are beginning to emerge. In particular, some of the states making the greatest gains in reading and mathematics on the National Assessment of Educational Progress in the 1990s, such as Connecticut, Kentucky, North Carolina, and Texas, also were early and consistent supporters of state standards and assessments. Studies by economist John Bishop at Cornell University have found that students in states with a combination of high school exit tests and end-of-course exams perform better on national tests than students in states without such exams.
A report released last fall by the American Federation of Teachers highlighted 11 urban districts where students have made academic progress, based on tests, and sustained it for at least three years--a trend that was not evident at the beginning of the 1990s. Many of those districts, including Boston, Philadelphia, and Corpus Christi, Texas--have placed strong standards and aligned assessments at the centerpiece of their school improvement efforts. Experts warn that some of those test-score gains could be inflated by teachers’ spending too much time preparing students to pass the tests.
But there is also a growing body of evidence that suggests schools serving high proportions of poor and minority children can achieve at high levels, based on strong leadership, rigorous academic expectations, a close attention to data, and a strong, shared sense of professional responsibility. (See related story, Page 19.)
In addition, surveys and case studies of schools and teachers in Illinois, Kentucky, Michigan, New Jersey, New York, Vermont, and Washington state have found that districts, as well as individual schools, are trying to revise their curricula to match state expectations. Teachers reported that they were adding new topics to their lessons to reflect what’s on state tests. In some cases, they were changing their teaching strategies.
Many of the 1,019 teachers surveyed for Quality Counts similarly reported that improvements are occurring in their schools. A majority said that, compared with three years ago, the curriculum is more demanding of students, teacher expectations are higher, teachers in their schools are collaborating more, and students are writing more. Nearly half also reported that students are reading more. Of those who said the curriculum is more demanding, more than six in 10 attributed the change to statewide academic standards. Just over half said their students are working harder because of the new standards.
“I think [standards] are making all teachers more aware of what needs to be taught,” says Kenneth H. Knowles, an English teacher at Carmel High School in Carmel, Ind., in a sentiment echoed by many teachers.
“For the first time in the 29 years since I’ve been teaching, I’ve seen charts published by the state department of education on those standards and what each student should know by what grade.”
A large majority of those surveyed also said they had modified their curricula to reflect state standards, and that they had adopted or created modules, curriculum units, or lesson plans linked to state expectations.
“Teachers report teaching a much more balanced curriculum today than they were reporting 10 years ago,” says Andrew C. Porter, the director of the Wisconsin Center for Education Research at the University of Wisconsin- Madison. “By balance, I mean a better balance across various kinds of cognitive skills: memorizing facts, computational skills, problem-solving, being able to communicate about ideas in science and mathematics.”
But, in most cases, the research suggests that such changes do not run deep. Although standards are beginning to find their way into classrooms, those classrooms still look much as they did 10 years ago.
“People are doing more of the content that’s being tested,” says William A Firestone, a professor of educational policy at Rutgers University, who is studying the influence of state standards and assessments on math and science teaching in New Jersey classrooms. “What’s harder to do is get teachers to change the ways they’re teaching, so that it really encourages students to think mathematically or think scientifically.”
A survey he and his colleagues conducted of some 245 4th grade teachers in New Jersey found that many reported using hands-on materials, such as geometric solids, to teach math. “But when we go into the classroom and watch teachers use the manipulatives with kids,” he says, “my impression is there’s a lot of basically drill and practice.”
In Illinois, a longitudinal study of how districts are implementing state standards “definitely saw changes from last year to this year,” says Lizanne DeStefano, a professor of educational psychology at the University of Illinois at Urbana-Champaign. Principals and teachers are now more aware of the standards, she says, and making initial efforts to match them to their curricula. But, she adds, “we have not seen a lot of indications of standards moving more deeply to influence instruction.
“I think principals and teachers are struggling with what to do next,” she says.
The Quality Counts survey found that elementary teachers were more likely than middle or high school teachers to report changes in curriculum and expectations. Teachers in states where students must pass exit tests to graduate, not surprisingly, also reported more changes in response to state standards and assessments than those in states where the stakes are lower.
Too Much Focus on Tests
Research suggests that high-stakes testing can be a powerful tool to change what happens in schools and classrooms, but not always for the better. Indeed, no aspect of the standards-based agenda has generated more debate--or stirred more dismay among teachers-than the heavy reliance on state tests to measure student learning and to dole out rewards and penalties to schools and students.
“When we talk. about standards-based reform in Chicago, and it’s actually true everywhere, don’t show me the standards documents. Show me what you test,” says Anthony S. Bryk, a professor of education and sociology at the University of Chicago, “because the load-bearing wall in all of this is not the standards documents, it’s the assessments.”
Forty-five states now compile report cards on schools, and 27 rate school performance, primarily on the basis of test scores.
Studies suggest that teacher are changing their instruction to align it with what’s tested. Teachers may increase their attention to specific topics, shift instructional time to concentrate more on the subjects that are tested, devise exercises that mirror test formats and expectations, and work with their students on such test-taking skills as filling in the bubbles on multiple-choice questions. (See related story, Page 23.)
In Quality Counts’ survey of public school teachers, about 30 percent said statewide academic standards have led to just the right amount of teaching that parallels the content of state tests. But nearly seven in 10 said instruction stresses tests “far” or “somewhat” too much. Sixty-six percent also said state assessments were forcing them to concentrate too much on what’s tested to the detriment of other important topics.
“I think that schools are changing their curriculum to teach to the test, and I think that’s unfortunate for children and teachers because it doesn’t allow for any creativity,” says Lori S. Teague, a science teacher at Parry McCluer Middle School in Buena Vista, Va. For example, this year Teague jettisoned a popular math and science unit on sharks because it didn’t match the Virginia assessments, which eventually will determine something would be amiss. But measurement experts caution that any one test samples only a narrow range of what students should be learning. If teachers concentrate on the test--rather than on the broader content undergirding the exams--it could lead to a bump in test results that does not reflect real learning gains.
Teachers and parents also worry that the strong focus on tests may be reducing students’ learning enjoyment.
“As a parent of an Indiana 2nd grader whose curriculum seems to be an almost exclusive regimen of test-taking practice ... I am becoming increasingly disgruntled with the standardized-testing explosion,” says David Ander on of Evansville, Ind. “The net effect seems to be a 1st and 2nd grade filled with math and language drilling that sends the message that learning is a chore, not something fun and exciting.”
‘Too Much To Teach, Not Enough Time’
State standards and tests have improved, but they are not as good as they should be. Though state standards have become clearer and more specific since the early 1990s, experts say there are still too many for teachers to cover them all. And, in some states, they remain so vague that teacher naturally turn to the tests for guidance.
In New York state, says Patricia A. Johnson, a math teacher at Scotia Glenville High School in Scotia, N.Y., “we thought when the new state exam came in, we’d be able to do things in more depth because there would be fewer topics. And, instead, there are just as many topics.
“There’s just too much to teach, not enough time, so you don’t get to do anything in depth anymore,” she says.
That view is widely shared. Seven in 10 of the teachers surveyed for Quality Counts said there is “somewhat too little” or “far too little” time in the school year to cover what the state academic standards require, a view that was more prominent among elementary and middle school teachers than among high school teachers.
Moreover, in many states, the tests may not adequately reflect the standards. That raises fundamental issues of fairness, because schools and students may be judged on tests that don’t mirror their curricula.
Forty states report that they have so-called criterion-referenced tests aligned with their standards in English in elementary, middle, and high school; 34 states report having such criterion-referenced tests in math. But studies by Achieve, a nonprofit group based in Cambridge, Mass., that promotes standards-based school improvement, have found that state tests do not sufficiently match their standards. For example, states may test some standards to the exclusion of others. And they tend to stress the less demanding knowledge and skills, rather than the more ambitious content spelled out in the standards. (See related story, Page 33.)
“Everyone always says that their tests are aligned with their standards,” remarks Lauren B. Resnick, the director of the Learning, Research, and Development Center at the University of Pittsburgh, which worked with Achieve on the alignment studies. “Well, enough! We now know that they’re not.”
Moreover, while most states no longer rely solely on multiple-choice questions to assess students’ knowledge and skills, relatively few ask students to write extensive essays, conduct experiments, or engage in other, more expansive projects to demonstrate what they know. Only seven states have performance assessments in subjects other than English. And only two--Kentucky and Vermont--use portfolios, or compilations of students’ classroom work, to help measure student progress.
In addition, states rarely provide teacher and students with the feedback necessary to use tests to help improve instruction.
What’s Realistic?
Meanwhile, some states may have rushed far too quickly to attach high stakes to their assessments or may be setting goals that are unrealistic. (See related story, Page 53.)
“Most states that I look at set performance goals without any reference to what students actually do, or what research says about how much improvement is feasible,” says Daniel M. Koretz, a senior social scientist with the RAND Corp.
Only 17 percent of teachers surveyed for Quality Counts said the academic standards in their states were “very much” or “somewhat” too high. Teachers in high-poverty schools were just as likely to say the standards were “about right” as teachers in other schools. But as the poverty rate in schools increases, teachers are less likely to say their students are currently meeting the standards for their grade levels.
Teachers are particularly worried that states expect schools serving large proportions of low-income children to perform just as well as other schools on state tests without ensuring a level playing field.
“I happen to be in an upper-middle-class environment, so it’s easier for me to reach the standards with those children,” says Reynold S. Forman, an English teacher at Colts Neck High School in Colts Neck, N.J. “I don’t think it’s fair to say, across the nation, no matter what socioeconomic area you’re in, we’re all going to be on the same plateau.”
Eighteen states--California, Kentucky, and North Carolina among them--have tried to address that equity issue by evaluating how much a school’s performance improves over time; 19 require schools to meet an absolute standard. Some do both.
Limited Help
The question of how high to set the bar is particularly crucial when tests are being used to make decisions about whether students can graduate or be promoted to the next grade. Eighteen states require students to pass exit exams to earn a high school diploma, and six more plan to do so in the future. Three states require students to pass state tests to be promoted to the next level in specified grades, a number that will rise to seven by 2003.
In 1999, for example, only 67 percent of 10th graders in Massachusetts passed a state English test and 44 percent passed a state math test that will be required for graduation beginning in 2003. The high failure rates led state Commissioner of Education David P. Driscoll to propose that students who passed their courses, but failed the exams, be allowed to receive a state-endorsed local certificate. In Arizona, officials delayed until 2004 a requirement that students pass a math test to earn a diploma and are considering delaying the diploma requirements for other state tests, following initial high failure rates.
We now run the risk of unfairness, in terms of this first wave of older students.”
But he worries that because policymaker underestimated how long it would take to get standards into classrooms, “we now run the risk of unfairness, in terms of this first wave of older students. I just think that’s an issue that states really need to pause and think through.”
Of the 18 states that require students to pass tests to receive their diplomas, 15 mandate that they get extra help if they fail the tests. But only nine of those states help subsidize remedial instruction for students who fail the exams the first time around. Louisiana, New Mexico, and North Carolina base student promotion in some grades on test results. But only Louisiana and North Carolina help pay for remedial study for students who fail the exams.
The issue is particularly troubling for students with special needs and those whose primary language is not English. Such students typically perform less well on state exams and are most likely to require special accommodations that may not be available or that may raise questions about the validity of test results.
In addition, little evidence exists to show that forcing students to repeat a grade pays off academically. A study in Chicago, for example, found that 3rd and 6th graders who were held back in 1997, under the city’s then-new promotion policy, did not perform any better--and sometimes did worse--than students who were “socially promoted” before the policy took effect. Even worse, the researchers found, one-third of students who failed 8th grade dropped out. Other research suggests that high-stakes testing in Texas also has contributed to increased dropout and retention rates, although those findings are under dispute.
Both education and measurement experts caution against using a single test score to make a high-stakes decision about a student, such as graduation or promotion. Yet, in many states, tests are acting as gatekeepers.
In general, surveys indicate that teachers are far less supportive of using state tests to evaluate schools or determine whether students graduate than are parents or members of the general public.
A survey conducted last year for the Business Roundtable found that more than six in 10 parents and members of the public believed that students should have to pass state tests to graduate from high school, even if they have passing grades in their classes. By comparison, fewer than four in 10 teachers surveyed for Quality Counts agreed that graduation should hinge on test scores.
But the evidence from opinion surveys suggests the public might be more comfortable with an approach that does not depend solely on test scores. More of those surveyed by the Business Roundtable thought grades and teacher evaluations were “excellent” or “good” guides to promotion than said standardized tests were. A 2000 survey by Public Agenda found that more than seven in 10 parents “strongly” or “somewhat” agreed that it’s wrong to use the results of one test to decide whether a student graduates or is promoted. Nearly eight in 10 said schools should use test scores along with teacher evaluations to make such decisions.
A Richer Picture
The solution, according to many experts, is to provide a richer picture of student and school performance by heeding other measures in addition to test scores. “I do not believe that in any foreseeable future, you can really drive reform through high-stakes testing systems,” says Bryk of the University of Chicago. “I think that the assessment systems are just too easily corruptible, and you can’t really measure fully everything you value.
“While this is a component,” he continues, “it’s got to be balanced by other components.”
Experts suggest paying more attention to the quality of teaching and learning in schools, the amount of time spent on various subjects, and the rigor of the curriculum. That might be accomplished by examining samples of student work or by sending teams of outside reviewers to visit schools. In England, such “school inspections” are the primary tool used to identify strengths and weaknesses and help schools improve.
Of the 27 states that currently rate schools, 11 do so based entirely on test scores. Sixteen include other measures in their rating systems--such as attendance and dropout rates--but they rarely carry enough weight to alter a school’s standing.
“I think where we’ve gone overboard is the notion that if you set clear targets, and you have some way of measuring them, you can then step out of the picture and somehow it will be the responsibility of local professionals to figure out how to get there,” says Brian M. Stecher, a senior social scientist with RAND. “While we were probably wrong two decades ago to be only focusing on accreditation, process-types of measures and not outcomes, I think we’ve probably gone overboard now in the other direction.”
Others suggest that states should engage in much more limited testing, in part, to leave more room for ongoing classroom assessments by teachers. They argue that individual schools and districts should make the ultimate decision about who graduates. 69ý and districts also would be responsible for providing their communities with more detailed information about performance, based on local goals and standards. Under such a scenario, the state would intervene when a school or district was not fulfilling its accountability role.
But exactly what “multiple measures” states should include in their accountability systems, or how that would work in practice, is far from clear.
Support Not There
In any case, better tests and accountability systems alone will not produce standards-based classrooms. States have yet to provide access to the professional development and high-quality teaching, curriculum, and other materials needed for schools and students to reach state standards. (See related story, Page 43.)
“My personal bias is that if you want to change day-to-day practice, you have to give people materials to work with,” says Hilda Borko, a professor of education at the University of Colorado at Boulder. “You can’t just depend on the assessments.”
David K. Cohen, a professor of education at the University of Michigan, and his research colleagues have found that when teachers participate in professional development in the subjects they teach and practice with materials they will use with students, instructional practices change and student achievement rises.
But, in general, states are only beginning to address those issues. Although 38 states, for example, require teachers to participate in professional development to renew their teaching licenses, only seven require that at least some of that training be in the teachers’ subjects or “endorsement” areas. And while 42 states pay for some professional development for teachers, only 24 provide such money for every district or school. Moreover, the quality of that training varies widely.
Of the teachers surveyed for Quality Counts, fewer than half said they had “plenty” of access to textbooks and other materials, lesson plans, and curriculum guides that match state standards. Fewer than half also reported having “plenty” of access to training in using the standards or interpreting test results. In fact, in the past year, a majority of teachers had five hours or less of training in each of those topics.
“The professional development is not there,” says Andrew Jervis, a social studies teacher at Babcock Middle School in Westerly, R.I. “School systems are not doing all that they can to make sure that all staff members have received proper training on how to use the standards.”
The survey found that teachers who had received more training in the use of state standards and assessments were more likely than colleagues with less training to report that they were using lesson plans linked to the standards, had significantly modified their curricula to align with state standards, and were using test results to diagnose individual student needs.
‘An Honest Review’
To their credit, many states are adjusting and modifying their approaches to standards-based changes as they learn more about how to make such systems work.
In Ohio, for example, the governor has appointed a commission to rethink the state’s testing and accountability system. States such as Indiana, Nevada, and New Mexico have revised their academic standards for students to make them clearer and more precise. Maryland has delayed requiring new high school exit tests for graduation until the state can ensure the supports for students are in place.
Such adjustments, observers argue, are not only desirable but essential, if standards-based reform is to live up to its potential. “It’s time for us to really do an honest review of where we are, what’s happening, identify different weaknesses and problems, and correct them,” says Bob Chase, the president of the National Education Association, “and to do it in a very open and honest way with all the stakeholders involved.”
Whether that potential will ultimately be realized is an open question.
“Those who have been pushing the standards movement, and the rhetoric of it, are now becoming aware that we’re not going to get the results,” warns John I. Goodlad, a professor of education at the University of Washington. “And so there’s a big cloud moving in.”
But others caution that it’s simply too early to tell. “I can cite you a very long list of concerns that I’ve got about almost every aspect of this set of reforms,” says Achieve’s Schwartz. “But if you ask me, ‘Has my basic belief that this strategy is the right long-term strategy been altered by the experience to date?’ the answer would be no.”
