69ý

Federal

‘Value Added’ Pioneer Says Stinging Critique of Method Is Off-Base

By Debra Viadero — May 05, 2008 3 min read
  • Save to favorites
  • Print
Email Copy URL

Includes updates and/or revisions.

The statistician who pioneered the use of “value added” research techniques for documenting students’ year-to-year academic progress is disputing a critique of his approach that was published recently in a prominent academic journal.

In the , which appeared in the March issue of Educational Researcher, Audrey Amrein-Beardsley, an assistant professor at Arizona State University in Phoenix, takes aim at the Education Value-Added Assessment System, or EVAAS. The EVAAS is an offshoot of the Tennessee Value-Added Assessment System developed by William L. Sanders, which is widely credited with helping to launch nationwide interest in value-added approaches for measuring students’ academic progress.

Ms. Amrein-Beardsley contends that the EVAAS is opaque, is possibly flawed, and has not been subjected to enough external review to merit its widespread use in school systems.

“No one is looking inside the model to see if the assumptions being made within it are true,” Ms. Amrein-Beardsley said in an interview. “The bottom line is to open this up for debate and not have these things be adopted blindly.”

‘Factual Errors’ Alleged

See Also

A recently posted response to Audrey Amrein-Beardsley’s critique . The response is written by William L. Sanders and S. Paul Wright.

But Mr. Sanders, for his part, contends that Ms. Amrein-Beardsley’s report is “rife with factual errors,” and he points to three studies in which researchers have duplicated and tested his model. Among the most recent was a 2007 study, published in the peer-reviewed Electronic Journal of Statistics, in which a team of researchers from the Santa Monica, Calif.-based RAND Corp. compared the Sanders method with other value-added techniques.

Now working in private industry as a senior manager for value-added assessment and research at the Cary, N.C.-based SAS Institute, Mr. Sanders has never shared the computational algorithms for the EVAAS. But he and a colleague described the model in a 1997 paper, which the RAND researchers and others have used to create their own software mimicking Mr. Sanders’ model.

Ms. Amrein-Beardsley also raises technical concerns with the EVAAS, some of which echo cautions that other experts are voicing about value-added modeling in general. (“Scrutiny Heightens for ‘Value Added’ Research Methods,” May 7, 2008.)

She notes, for instance, that the EVAAS does not take into account demographic and socioeconomic differences between students.

“Family income, ethnicity, ability, and other background variables unquestionably affect levels of student achievement and the progress that students make from year to year,” she writes.

Mr. Sanders, though, has long maintained that such statistical adjustments are not needed because the EVAAS compares students’ year-to-year academic growth with their own progress from previous years. “A kid is coming from the same home this year as he was the year before,” he said.

To illustrate her case, Ms. Amrein-Beardsley reanalyzed data from the appendix of a controversial 2005 study of North Carolina teachers that Mr. Sanders was recruited to do for the National Board for Professional Teaching Standards, an Arlington, Va.-based group that awards national credentials to teachers who meet the requirements of a process based on documentation of their performance.

While Mr. Sanders concluded that board-certified teachers did no better on average than those who had failed to earn certification and those who had never tried, Ms. Amrein-Beardsley said the data showed a positive student-achievement effect. She said that, on average, students of board-certified teachers outperformed those of teachers with only regular certification by three-quarters of a month’s worth of growth in mathematics and one week in reading.

Mr. Sanders said his conclusions stemmed in part from another finding in his study: The variation in student-achievement levels for teachers within the NBPTS-certified group was greater than it was for teachers across different categories.

Joseph A. Aguerrebere Jr., the board’s chief executive officer, said most studies of board-certified teachers, such as the one by Ms. Amrein-Beardsley, also find that students of those teachers outperform their peers.

Coverage of education research is sponsored in part by a grant from the Spencer Foundation.

Events

School & District Management Webinar Crafting Outcomes-Based Contracts That Work for Everyone
Discover the power of outcomes-based contracts and how they can drive student achievement.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Harnessing AI to Address Chronic Absenteeism in 69ý
Learn how AI can help your district improve student attendance and boost academic outcomes.
Content provided by 
School & District Management Webinar EdMarketer Quick Hit: What’s Trending among K-12 Leaders?
What issues are keeping K-12 leaders up at night? Join us for EdMarketer Quick Hit: What’s Trending among K-12 Leaders?

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.

Read Next

Federal What a National School Choice Program Under President Trump Might Look Like
School choice advocates—and detractors—see a second Trump term as the biggest opportunity in decades for choice at the federal level.
8 min read
President Donald Trump listens during a "National Dialogue on Safely Reopening America's 69ý," event in the East Room of the White House, on July 7, 2020, in Washington.
President Donald Trump listens during a "National Dialogue on Safely Reopening America's 69ý," event in the East Room of the White House on July 7, 2020, in Washington. He returns to power with more momentum than ever behind policies that allow public dollars to pay for private school education.
Alex Brandon/AP
Federal Trump's Education Secretary Pick Is Linda McMahon, Former WWE CEO
McMahon led the Small Business Administration in Trump's first term and is co-chair of the president-elect's transition team.
6 min read
Small Business Administration Administrator Linda McMahon speaks during a news conference with President Donald Trump in Trump's Mar-a-Lago estate in Palm Beach, Fla., Friday, March 29, 2019.
Then-SBA Administrator Linda McMahon speaks during a news conference with President Donald Trump at Trump's Mar-a-Lago estate in Palm Beach, Fla., Friday, March 29, 2019. Trump has tapped McMahon to serve as education secretary in his second term.
Manuel Balce Ceneta/AP
Federal What Could RFK Jr. as HHS Secretary Mean for School Vaccine Requirements?
The vaccine skeptic in line to lead the mammoth federal agency could influence schools' vaccine rules, even though they're set by states.
6 min read
Robert F. Kennedy, Jr., speaks before Republican presidential nominee former President Donald Trump at a campaign event, Friday, Sept. 27, 2024 in Walker, Mich.
Robert F. Kennedy, Jr., speaks before President-elect Donald Trump at a campaign event on Sept. 27, 2024 in Walker, Mich. Trump has selected Kennedy to serve as secretary of health and human services in his second term.
Carlos Osorio/AP
Federal Can Trump Force 69ý to Change Their Curricula?
Trump's bid to take money from schools that teach "critical race theory" or pass policies for transgender kids raises legal complexities.
9 min read
Republican presidential nominee former President Donald Trump speaks on crime and safety during a campaign event at the Livingston County Sheriff's Office, Tuesday, Aug. 20, 2024, in Howell, Mich.
Republican presidential nominee former President Donald Trump speaks on crime and safety during a campaign event at the Livingston County Sheriff's Office, Tuesday, Aug. 20, 2024, in Howell, Mich.
Evan Vucci/AP