When comparing the United States’ higher education system with those of other developed nations, a new report says, policymakers are misreading the data and relying on flawed statistics.
That conclusion comes in a report, “The Spaces Between Numbers: Getting International Data on Higher Education Straight,” that was scheduled to be published this week by the , a research and policy group based in Washington.
The findings are important because they call into question many of the statistics that are commonly used to make the case that the United States has lost its standing as the world leader in higher education. The drumbeat of bad news coming in recent years from reports by the Organization for Economic Cooperation and Development and others has provided the statistical underpinnings for national rallying cries to improve colleges and universities.
In unveiling his plan in July, for instance, President Barack Obama pledged that, “by 2020, America will once again have the highest proportion of college graduates in the world.”
The new report suggests, however, that, because of problems with the data that the Paris-based OECD uses, the nation’s colleges and universities may not be as bad off as they are sometimes portrayed to be.
“What I’m saying is we’re asking the wrong questions. We have the wrong indicators, and the ones we have are a mess,” said Clifford Adelman, the report’s author. “But we can still develop the ones we need.”
Experts on this side of the Atlantic Ocean say that Mr. Adelman may have a point about the quality of international data, but that the statistical problems don’t necessarily change some of the negative conclusions being drawn about American higher education.
The United States comes out on top in the proportion of the population with at least a bachelor’s degree, as of 2006, when compared with 29 other countries.
SOURCE: Organization for Economic Cooperation and Development
“He’s absolutely right that there are problems with international comparisons of college attainment,” said Terry W. Hartle, the senior vice president for government affairs and communication at the American Council on Education, the Washington-based group that coordinates the nation’s colleges and universities. “But I don’t think we’re as well off as the reader might conclude from this paper.”
Those conclusions drew a harsher assessment, though, from the OECD, which publishes the annual reports and other cross-national studies in education. “There are so many fundamental conceptual flaws in the argumentation,” Andreas Schleicher, the head of education indicators for the international group, wrote in an e-mail, “that this does not, in my view, warrant a serious response.”
Misreading the Numbers
As an analyst for 27 years with the National Center for Education Statistics, the top statistics agency for the U.S. Department of Education, Mr. Adelman long had a front-row seat to efforts in this country to gather data on U.S. colleges and universities for the international economics group. He said he spent much of the past year, however, exploring how other countries contribute to the process—even enlisting translators to decipher the statistical manuals used abroad.
As problematic as some of those international measures may be, though, Mr. Adelman says he is just as troubled by the way Americans are reading the data the OECD provides. When commentators in the United States bemoan the nation’s third- or 10th-place rank among 30 industrialized countries in terms of the proportion of the population that holds a college degree, Mr. Adelman argues, they are looking at the wrong chart in the 2008 edition of “Education at a Glance.” Those data include associate’s degrees, where the United States is weakest.
Mr. Adelman said community college graduation rates are low because “community colleges in the U.S. take on about six missions, only one of which is awarding a degree.”
“It’s not fair to compare them with two-year colleges in other countries, because that’s all those colleges do over there,” he added. Some countries also count trade-school and real estate certificates in the mix, experts noted.
If the commentators had looked at a different chart, one that compares the proportion of different age groups earning a bachelor’s degree or higher, they would have noticed that the United States ranked first in every age cohort except the 25- to 34-year-old age group, where it stands in second place.
Arthur M. Hauptman, an independent policy consultant based in Arlington, Va., said the college-attainment rates are higher among older age groups in this country because “we have a lot more second-chance degrees than other countries.” People return to school, for instance, in their 30s or 40s and earn a degree, go to school at night, or enroll in community colleges. “Cliff has been saying we have to be much more careful about these numbers, and I’m making the same point, but nobody seems to be listening.”
Defining What Counts
But the ACE’s Mr. Hartle, for one, said slipping attainment rates over the past decade among the youngest age group is still a problem that needs addressing.
“If we want people to get the most personal value from a postsecondary degree and we want the most societal benefits,” he said, “it would be better if people got a degree at 25 than 55.”
American colleges and universities also appear to trail other nations, Mr. Adelman said, because countries differ in how they are allowed to define graduation rates, degrees, and other essential information. For instance, on some indicators, the United States is the only nation for which an institution graduation rate is reported—a rate for the percentage of students who graduate within a certain period of time from the same college where they started.
The other 23 nations included in the measure that Mr. Adelman studied report the rate for the percentage of students who graduate from a college in the same system of colleges and universities. If the United States were allowed to use a system graduation rate for the 2008 report, the percentage of graduates would rise from 56 percent to 63 percent.
As an NCES employee, Mr. Adelman said, he submitted a system rate for the United States to the OECD, but the report buried the information in an appendix.
Countries also vary, the report notes, in the period of time in which students must earn those degrees in order to be counted as a graduate. The United States reports six-year graduation rates, and other countries, such as Finland, stretch the time frame to as long as 10 years.
“He’s right about the difficulties in coming up with comparable statistics,” said Mark S. Schneider, who was the NCES commissioner from 2005 to 2008. “But going from 56 percent to 63 percent? I still don’t think that’s good enough, quite frankly.”
Mr. Schneider said he agrees, nonetheless, that some OECD member nations may seek to put their countries’ education systems in a better light relative to this country.
“We have a disproportionate number of the best universities in the world, so we’re a target for many countries,” added Mr. Schneider, now a vice president at the American Institutes for Research.
The Biggest Ship
In his report, Mr. Adelman also predicts that the nation’s standing will worsen on international graduation-rate measures because of its large population and growing birth rate. The birth rates in most other industrialized nations, in contrast, have either stabilized or dropped.
“Not only are we the largest ship, but we are on track to add more people over the next 15 years than any other OECD country—nearly three times what Mexico will add, over three times what Turkey will add,” the report says.
In contrast, the report calculates that the population of the 25- to34-year-old age group is expected to shrink by 20 percent or more in 10 of the 30 member countries over the next 15 years.
“You don’t need more than 4th grade math to figure out what’s going to happen if you’re just measuring graduation rates,” Mr. Adelman said. “Countries with dramatically falling denominators (e.g. Japan, Russia, Poland) will see all their percentages rise,” the report elaborates. “Those with rising denominators (United States, Australia, Canada, Ireland) will see all their percentages fall.”
Further complicating matters, 46 countries are in the midst of restructuring their higher education systems, a process that, in many of those places, is shortening the time it takes to earn bachelor’s-level degrees from five years to three.
“When you move to a three-year cycle, you’re awarding more degrees,” Mr. Adelman said.
Mr. Schneider said other emerging changes in higher education, such as the growing numbers of universities offering online degrees, will make it even more difficult to find comparable international indicators in the years ahead. “Higher education is on the leading edge of an incredible revolution,” he said.
The report argues that some changes in statistical procedures could help portray the United States in a more accurate light. The NCES, for example, could submit only a system graduation rate, as other countries do.
The report says international comparisons could also be more meaningful if they were aimed at answering different questions, looking, for instance, at the quality of education that colleges and universities provide, the inclusiveness of the systems, and the flexibility of the paths that lead to postsecondary degrees.
“We all need more meaningful indicators than our international arbiters currently provide,” the report concludes. “Tinkering at the edge of current data collection will not provide that meaning, but the national statistical agencies are capable of providing it.”
