It usually takes Aimee Knaus, who has been teaching computer science for more than two decades, upwards of two hours to grade a classful of coding projects.
This school year, she cut that time roughly in half, with the help of an AI teaching assistant developed by Code.org, a nonprofit organization that aims to expand access to computer science courses, and the Piech Lab at Stanford University in Palo Alto, Calif.
Knaus, who teaches middle school in Kimberly, Wis., was one of twenty teachers nationwide who tested the computer science grading tool’s capability on about a dozen coding projects also designed by Code.org, as part of a limited pilot project.
Beginning today, Code.org is inviting an additional 300 teachers to give the tool a try.
In early testing by Knaus and other teachers, the tool’s assessment of student work closely tracked those of experienced computer science teachers, said Karim Meghji, the chief product officer at Code.org. If that trend holds through this larger trial, the nonprofit hopes to make the tool widely available to computer science teachers around the country, he said.
Meghji would ideally like the tool to become widely available by the end of the calendar year, but the timeline will depend on the results of the broader testing.
‘I worry AI couldn’t push my students to the next level’
Many educators see helping teachers tackle time-consuming but relatively rote tasks—like grading—as a huge potential upside of AI. Curriculum company Houghton Mifflin Harcourt’s Writable tool, Essaygrader, Gradescope, and others . And some teachers have experimented with using ChatGPT, an AI-powered writing and research tool, to grade papers.
But others are wary of outsourcing grading—especially on assignments that call for making subjective decisions about students’ writing or ideas.
“After all these years, I’ve worked to perfect my feedback and my process,” said Carly Ghantous, a humanities instructor at Davidson Academy Online, a private virtual school. “I worry that the AI wouldn’t be able to push my students to the next level of writing” as well as an experienced teacher could.
By contrast, the criteria for grading the coding projects that the computer science tool examined are cut and dry, Meghji explained.
The AI tool must determine whether a student’s coding project contains certain requirements—for instance, whether there are at least two changeable elements. That’s something the technology figures out quickly and accurately.
Megjhi predicts that, down the road, AI tools could frequently be tapped to help grade student work. Given the technical nature of computer science, it makes sense that the subject would be among the first out of the gate, he added.
“I think we have a unique situation with computer science and coding,” he said. “I do think that assessment will become an AI-assisted task for teachers across multiple subjects. I can’t say what that’s gonna look like for English or for social studies or other subjects. But I can tell you that I do think based on what I’ve seen in computer science that there is there’s definitely an opportunity for broader application.”
‘I had the answer key, but I didn’t know what I was doing’
Grading computer science tasks can be particularly tricky—and time consuming, Meghji said, which is why Code.org was interested in exploring possible AI solutions.
Many teachers who lead computer science courses don’t have a degree in the subject—or even much training on how to teach it—and might be the only educator in their school leading a computer science course.
Kevin Barry, a former social studies teacher, was tapped to teach computer science at his southern Maryland high school by a principal who noticed Barry’s facility with technology. Barry, who is one of a hundred educators who gave Code.org information to inform the development of its tool, hasn’t yet tested the product for himself.
Still, he already wishes that something like it had existed when he first stepped into the computer science teaching role nearly ten years ago.
Back then, “it took me even longer to grade because I didn’t know what I was grading,” said Barry, who teaches at La Plata High School. “I was learning it as I was doing it with the students. I had the answer key, but I didn’t know what I was doing.”
These days, the tool could help him challenge his highest flyers—giving Barry extra time to help those who may be struggling.
In his classes, “I’ve had the captain of the robotics team [and a] kid who lives on a farm and barely knows what the power buttons are,” along with 28 other students whose abilities are somewhere in between those extremes, Barry said.
“I have more advanced students that want to go on” to trickier projects, Barry said. If the tool was able to examine their work, “they can be working three assignments down the road.”
‘We should still always be the ones in control’
Knaus agrees that the tool could be a huge time-saver for teachers. She’s interested in seeing it go beyond grading, offering real-time feedback or assistance to students as they work on coding assignments.
Knaus was surprised by how in sync the tool’s assessment of student work seemed to be with her own. But if she disagreed with the tool’s estimation of an assignment, she wouldn’t hesitate to ignore the AI tool’s recommendation.
“I don’t know that it necessarily is going to always be completely accurate,” Knaus said. “When we think about AI, I think it’s really important for humans to understand that we should still always be the ones in control.
“I don’t think we want to get to the point where we trust AI” over our own knowledge, she concluded.
