On Monday, I with departing Institute of Education Sciences Director Mark Schneider, who just wrapped up his six-year term. In our conversation, he argued for newer and better research centers at IES, along with a heightened commitment to producing timely and accessible reports. Well, as anyone who knows Mark well can attest, he almost always has more to say. I thought I鈥檇 reach back out and see if he had anything else he wanted to get off his mind. Here is Part Two of our conversation.
鈥搁颈肠办
Rick: On Monday, you mentioned that Marguerite Roza, Emily Oster, and Sean Reardon are doing good, serious work on education topics and releasing it in a timely fashion. Can you say a bit about what they鈥檙e doing right and why NCES isn鈥檛 meeting that need?
Mark: As noted on Monday, the most telling example is the collection of school-level finance data. Not only are these data required by law, they are also among the most important pieces of information needed to understand the distribution of resources and to relate detailed expenditure data to school outcomes. But despite years of pressure鈥攆rom Jim Blew, who headed the policy division of ED during the last few years of the DeVos era, from me, and from many others鈥擭CES has never produced timely district financial data and has yet to produce complete school-by-school financials. District financials are released more than two years after the close of the fiscal year, and school-by-school financial pilot data dating back to FY18 have not yet been released. Looking ahead, NCES has promised to release its first set of complete FY22 school-level financials at the end of 2024鈥攁gain, over two years after the close of the fiscal year. Assuming NCES meets its promise鈥攁 risky bet, given past missed deadlines鈥攖his would be 鈥渇ast鈥� by government standards鈥攂ut far too slow to meet real-world needs.
Rick: What鈥檚 responsible for this?
Mark: Much of the problem is that NCES uses an antiquated approach to data collection, issuing a uniform survey that doesn鈥檛 match up to different state systems and then waiting for all submissions before releasing datasets. In contrast, teams from Georgetown, Brown, Stanford, and others grab data directly from the source and then convert them into a more standardized format. Teams then release the state-by-state files as they are available instead of holding it all until the very last state has a clean file. The process is far faster. For instance, Georgetown鈥檚 site鈥攔un by Roza鈥攊s posting FY23 school-by-school financial files for some states in as few as six to eight months after the close of the fiscal year. With the quick turnaround, district leaders can, for example, marry the finance data with Stanford鈥檚 performance data and use it to inform decisions during the next budget season.
Rick: OK. So, what鈥檚 some of this other work that you flagged?
Mark: Similarly, Emily Oster鈥檚 work on school closures during the pandemic was released faster than NCES鈥檚 and its coverage was more comprehensive. In part because of NCES鈥檚 lagging performance, I was able to get additional money from Congress to set up the School Pulse Survey, which has finally given NCES a tool for gathering close to real-time information about conditions in schools. Fair warning, though: NCES calls it 鈥渆xperimental,鈥� which means they are lying in wait to encumber it with more and more statistical tests that will likely delay the release of the data.
Oster and Sean Reardon have also been far ahead of NCES in gathering and releasing detailed data on student performance. Reardon鈥檚 presents detailed and timely information about COVID-related learning loss and recovery. Oster is also working on releasing more detailed and extensive data on student assessment from all states. To this point, they have released data through 2023. In contrast, NCES鈥檚 EdFacts promised the 2022 data in December 2023, but it鈥檚 not been released yet. Here鈥檚 an additional wrinkle: Oster did her work with one salaried employee and a team of undergrads. EdFacts costs around $13 million per year.
Rick: I鈥檝e been struck that, during Secretary Cardona鈥檚 tenure, political appointees have gotten far more involved in the release of NAEP results and used the releases to promote administration talking points more than they have in past administrations. How concerned should we be about IES getting caught up in our partisan divides?
Mark: I was both lucky and happy to serve during both the Trump and Biden administrations. Although appointed by Trump, I have found that most of my dealings with senior leaders appointed by Biden have been quite good. But will that political distance and professionalism hold in a second administration of either major candidate? The signs are not good. For instance, IES has an advisory board called the National Board of Education Sciences, or NBES. When the Obama administration was on its way out the door, it tried to fill NBES with their appointees. However, the commissions were never fully executed, and the Trump administration refused to honor the unfinished appointments. Both administrations were within their legal rights to take these actions鈥攂ut we will still need to see what political ramifications follow.
When I assumed the IES director鈥檚 role about halfway through the Trump administration, there were few people on the board, all holdovers from the Obama administration. It was very hard to get the White House to pay attention to the open seats, and I was only able to get three people, all high quality, appointed to the board. At the end of 2020, on his way out the door, Trump appointed a whole bunch of people to the board, many of whose qualifications raised eyebrows. When Biden took office, those commissions had not been finalized, leaving the board consisting of the three people I was able to get appointed. One morning, each of them received an email saying that they had to resign by the end of the day or be fired. The administration then appointed 14 people to the 15-person board. I fear that if the Republicans win the presidency this fall, they will fire the board and replace it with people more to their liking, making what should be a nonpartisan science board highly politicized. I can鈥檛 help but wonder if that will lead to the politicization of the director鈥檚 tenure.
Rick: Looking outside of IES, it certainly feels to me like the professional education research organizations have become increasingly ideological over the years. Is that a valid criticism? If so, what might help?
Mark: I agree. In 1996, mathematics professor Alan Sokal published the paper 鈥溾�� as a hoax to show the shallowness of postmodern critical theory. When he wrote it, I鈥檓 sure he was hoping to put an end to that line of work. But, as James Meigs in a 2021 Commentary piece: 鈥淪okal meant his essay as a parodic warning. Twenty-five years later, it appears that the Sokal Hoax was actually an instruction manual.鈥� Rick, as you recall, for years, as the AERA annual conference approached, you would go through the program and identify what you considered the most bizarre takes on education. I鈥檓 not sure that you could do this today, since you鈥檇 have to publish just about the entire program.
Rick: Oh, man, that takes me back. Yeah, I eventually gave that up. I鈥檇 hoped that shame could help discourage some of the sillier stuff, but it felt, like you note, that the silly stuff won out. Why is that?
Mark: Education research is highly ideologically driven. If DEI has affected more established and rigorous sciences, which it has, then what can we expect from a far weaker scientific field such as ours? As Meigs noted about Sokal鈥檚 paper turning into an instruction manual, I guess your lists could be viewed as a guide to the future work in education.
Rick: Here鈥檚 a final question. If you could leave your successor at IES with one piece of advice, what would it be?
Mark: My daughter told me that she learned this parable while in business school.
A new executive is meeting with the executive she is replacing and asks for advice. The outgoing exec says, 鈥淚 left three envelopes in the desk drawer. Open them sequentially as crises emerge.鈥�
Sure enough, at the inevitable first crisis, the new exec opened the first envelop, which read: 鈥淏lame your predecessor.鈥� At the second crisis, the exec opened the second envelop, which read: 鈥淎pologize and swear to do better.鈥� Then the third crisis showed up, and the third envelop read: 鈥淲rite three letters.鈥�
When I pass the baton, I hope that I have gotten IES a little further down the track than people expected. But the race is long, there鈥檚 a lot of work left unfinished, and the demand for accountability and innovation won鈥檛 go away any time soon.
Oh, and finally, the next director should pray that we don鈥檛 have another pandemic. While there are many reasons we should hope never to see a pandemic again, from a narrow perspective, COVID punched a two-year hole in my tenure, limiting what I was able to accomplish.
