The Arcadia Unified school district in California was already building a basic chatbot specifically for staff use at least four years before ChatGPT became the hottest AI tool on the market, recalled Greg Gazanian, the district鈥檚 chief strategy and innovation officer.
But the district paused the project when the pandemic hit.
Then, in 2022, OpenAI publicly released ChatGPT, and generative artificial intelligence became easily accessible to nearly everyone.
Arcadia Unified decided to pick up the development of its chatbot where it left off and experiment with applying a generative AI model to it.
For the past year and a half, Gazanian and his team have been testing and piloting the chatbot with a few staff members. Soon, he said, they鈥檒l open up its use to any employee who wants to try it out and give real-time feedback so they can make improvements.
Education Week first spoke to Gazanian about the chatbot in February and then received an update about its rollout in October. Gazanian also discussed why and how his district built the chatbot and offered advice for other districts looking into ways to leverage generative AI.
This interview has been edited for brevity and clarity.
Why did the district decide to build this AI tool internally?
What we found is developing it internally lets us do a few things: It lets us really focus on creating an AI bot that can respond in ways that are very customized to our teachers and staff. For example, it can give advice about our purpose and values, it can give advice about our curriculum. If we need it to answer slightly differently, or if we need to add new resources, it鈥檚 not something where we have to call a company, we have to negotiate with them.
It also allows us to pick the AI engines that are right for us.
We feel good about where we鈥檙e at in terms of privacy, because there is no middle layer. We don鈥檛 have another company who鈥檚 going through our data, then sending it off to a model. We鈥檙e processing the data ourselves.
Developing it in-house has really allowed us a lot of flexibility and a level of customization that a lot of other places haven鈥檛 necessarily been able to do and you don鈥檛 generally have when you鈥檙e working with a third party.
There probably aren鈥檛 a lot of districts with the expertise to build a tool like this internally. Could you talk about how you鈥檝e made it work?
At the end of the day, we鈥檙e able to do it because we have a collection of鈥攁nd I mean this in the more fun way鈥攈ackers in our organization, meaning we鈥檙e tech people. We love trying new things, we love fiddling with it, and we love trying to get it to work. We also have a board of education that鈥檚 very supportive. We have a real commitment to trying new things and tight feedback loops and being willing to stop doing things if they don鈥檛 work.
Internally, it鈥檚 myself and a few other members of the tech department who do more of the technology side. In the general staff in our district, we have people who are very comfortable trying new things and giving feedback, and we鈥檝e built this mutual trust where we鈥檙e willing to explore that.
Is this AI tool a 鈥榳alled garden'鈥攎eaning it鈥檚 trained only on vetted resources rather than all of the internet?
Currently, it鈥檚 not. We did not clip the wings of the model yet鈥攚hich means, if you ask it something broad, it鈥檒l pull information in [from other parts of the internet]. One of the benefits of that is when you鈥檙e writing letters, when you鈥檙e writing an analysis, having that broader corpus of information is really helpful. For example, if we鈥檙e writing an email to parents about the benefits of AI, if it was just looking for our data, it鈥檇 be like, 鈥淎I can be useful in this very small context.鈥� Instead, it can say, 鈥淗ere are five case studies about how AI is helpful in schools.鈥�
We encourage people to read through it, make sure it makes sense, do a little bit of checking before you use it. We do put some protection there. For example, [we鈥檝e trained it so that] if it doesn鈥檛 know the answer, it doesn鈥檛 make it up.
Los Angeles Unified bungled the rollout of its AI tool. How are you ensuring that doesn鈥檛 happen?
I鈥檓 not super knowledgeable on what happened internally. What I will say is any time you鈥檙e using a third party to do your development, there鈥檚 always a risk there. They could go out of business, there could be business disagreements, there may be a misunderstanding, there may be a change in scope. One thing that we have going for us is, because we are developing this internally, we may still run into challenges, but it鈥檚 all within our organization.
The other thing that鈥檚 really important to how we go about things is engaging our stakeholders really early. We have teachers working with us on this development internally, whereas maybe in another instance of using a third party, the third party鈥檚 never met those teachers. They鈥檙e not working with them, they鈥檙e not in the same room. In a lot of cases, in our district, we鈥檙e working with people we鈥檝e known for five, 10, 15, 20 years. We have that trust.
And we鈥檙e comfortable playing the long game and taking enough time to make sure we do this in a way that makes sense for a smooth, continuing rollout.
What has been the feedback from staff who have used the tool?
They really like having it easily available. It also saves money. They don鈥檛 need to pay for a ChatGPT license. They really like having it trained on our own data. They like how customizable it is without having to add a bunch of extra data.
They give feedback when something鈥檚 not running so well, too. Our early version of the chatbot had no memory鈥攎eaning they could ask a question, they鈥檇 get a good answer, but if they asked a follow-up question, it wasn鈥檛 working super well. There are some things that need to be a little bit more polished if it鈥檚 going to be comparable to these other services they鈥檙e paying for.
People feel so much ownership when they can give that feedback and see it change, and that鈥檚 where it really starts to feel like a community that鈥檚 developing something together.
What advice would you give to other districts?
Have a good 鈥渂uild versus buy鈥� conversation. Do you have the resources today to start this? Do you have the resources down the road to continue to support this? Having a conversation up front about whether you鈥檙e willing to commit to whatever the development and maintenance arc is over time.
Who are the people we need to have in the room for starting this? Do we have a tech person who鈥檚 excited and knowledgeable about it? Do we have someone on the curriculum side who can give us effective feedback? What is the community鈥檚 perspective?
But the other thing I鈥檇 say is reach out. I鈥檓 happy to chat. I know anyone else who鈥檚 working on these projects is happy to chat. Everyone I鈥檝e talked to who鈥檚 working on this stuff does it because they love it, they鈥檙e intrigued by it. They recognize that the world we live in is changing and is going to continue changing because of this new technology.
Recognize early on that you鈥檙e not alone in this. Let鈥檚 build a community around it. We want to do this because it helps our teachers and our staff.
