Choosing a learning management system is one of the most costly and time-consuming decisions schools or districts must make as they expand their technological infrastructures. An LMS is a robust piece of software that provides an online portal for classrooms, serving administrative functions for educators and allowing students to view assignments, grades, and learning materials. Some can be used to deliver entire courses. Once an LMS is in place, it can be difficult and expensive to switch to a different one. That’s why making sure it’s a good fit initially is crucial.
Ed-tech experts say many schools and districts don’t have a good understanding, however, of how to choose and evaluate the different systems available. Several experts offer these suggestions for how to do it right:
1. Start by figuring out what you want from your learning management system and how it fits into the overall teaching and learning structure of your school or district.
“You really need to be having a discussion about your overall program goals,” says John Watson, the founder of the Durango, Colo.-based Evergreen Education Group, which conducts research on the use of technology in schools. “The LMS decision has to be tied to your content choices and to the devices that students are going to be accessing the materials with.”
Many LMS companies boast a dizzying array of features, says Watson, so it’s important to enter the discussion with an idea of what students, teachers, and administrators need from the system. He groups options into three categories—don’t need, nice-to-have, and must-have—to narrow the choices and avoid paying for features that aren’t needed or won’t be used.
Kristy Murray, the director of the Advanced Distributed Learning Initiative, agrees. The initiative, which is run by the U.S. Department of Defense, is charged with prototyping and testing the latest learning technologies.
“Sometimes you may get more bells and whistles with more money, but you may not need more bells and whistles,” Murray says. “The most important thing you can do is sit down with your team and identify the requirements for your particular case.” In some instances, a simpler, lightweight LMS may be more functional and easier to use than a complex, sophisticated system, she says.
Peter Berking, the instructional systems designer for the Advanced Distributed Learning Initiative, and co-author of the paper “Choosing a Learning Management System,” says it’s important to evaluate whether your school or district even needs an LMS.
“You may have your heart set on it, but a Web portal or content repository ... could do just fine,” he says. Berking also suggests looking into partnering with other schools or districts to piggyback on an existing LMS, which could provide cost savings.
In fact, definitions of what an LMS is and should be able to do vary widely from district to district, says Themistocles Sparangis, the chief technology officer for the 670,000-student Los Angeles school district, which is preparing a request for proposals, or RFP, for a new system. (The district currently uses the open source LMS Moodle.)
“You have to define [what an LMS is] before you can select it,” he says. “Your vendors are going to try to define [an LMS] within their product line, but I think K-12 is still struggling to figure out what it is.”
Does the school or district really need an LMS, or would a content repository or lighter-weight Web tool work instead? How would an LMS fit into the overall structure of teaching and learning? What problems is the school or district trying to solve with an LMS? What is important to teachers? To students? To administrators?
For Los Angeles, which is moving to implement a new LMS in part because of its new mobile learning initiative, as well as the shift to common-core standards, the LMS is “the glue” that holds together a variety of functions, Sparangis says. The system will have to integrate human-resources systems, the student-information system, the curriculum, classroom-level data, and dashboards, as well as student and parent portals, he says.
“We’re basically redefining our learning environment, and the LMS is going to be a critical part of that out-of-school as well as in-school experience,” says Sparangis.
2. Include a mix of people in the decisionmaking process.
Although putting a learning management system in place and maintaining it can fall on the shoulders of the district’s IT experts, they should not be the only people involved in choosing an LMS, says Philip D. Hill, an ed-tech consultant, analyst, and co-founder of Los Gatos, Calif.-based MindWires Consulting. Hill has been following LMS companies for more than eight years and writes about developments in the market on the e-Literate blog.
“All of the groups need to be involved upfront,” he says. Try to avoid starting with the technical requirements and then forcing administrators, teachers, and students to buy into those requirements with little input about what they think of them.
Are the people who will be using the LMS at the table for the decisionmaking? Is the evaluation including the perspectives of administrators, teachers, and students, as well as IT experts? What do the end users want and need from this technology?
The 2,100-student Masconomet Regional district in Boxcord, Mass., recently spent two years evaluating and choosing an LMS. (It ultimately decided to go with Washington-based Blackboard Inc.) The district then released a paper detailing its evaluation process and what it learned. Involving everyone—from students and teachers to administrators and IT staff members—was critical to making the decision, says the district’s chief financial officer, Susan Givens, who oversaw the process. “I’ve always believed that you have to try to have representation from all user levels,” she says.
3. Play an active role when viewing and exploring LMS product demonstrations. Ask specific questions related to your school’s or district’s situation, and request to see how certain functions work.
Seeing a company representative give a demonstration of an LMS can be helpful, says Hill from MindWires Consulting, but such presentations are typically rehearsed and may not be specific to a school’s or district’s particular needs. To make sure they are relevant, “set up these demos so that people can ask questions,” he says. “Unscripted demos are very useful.”
“We always suggest that schools ask for online demos, and that the schools are really driving [those demos],” says Watson from the Evergreen Education Group. “Make sure it’s not the company saying, ‘Let me show you the five things we think are really cool,’ but it’s the school saying, ‘Here are the five things we want it to be able to do. Walk us through how to do it.' "
Is the system truly optimized to perform certain functions that are important for your organization? Was the company representative receptive to answering difficult questions during the demonstration?
Vendors may claim that the LMS can do almost anything, says Berking from the Advanced Distributed Learning Initiative, and that is why it is vital to critically evaluate those claims.
4. If possible, pilot the learning management system. (This is not a realistic option for all schools, but it can provide helpful feedback.)
In the Masconomet district, each of the companies that responded to the RFP provided the district with a login and a “sandbox” that school officials could go into to play with the system for a period of weeks. The district then assigned a number of people—students, teachers, and administrators—to go in and use the system. “You really get some terrific feedback,” says Givens, the district’s CFO.
While users should explore the system, it’s important to have a well-defined process for evaluating the LMS, Givens says. “Make sure you have a protocol for people to follow if you do a sandbox,” she says. “If you have people testing, don’t just go in and test. Everybody has to go in and do the same thing for each product; otherwise you’re open to subjectivity.” For example, one task each pilot user had to complete was uploading a video from YouTube into the system.
How user-friendly are the features? Was the system hard to navigate? Were the users able to complete the tasks given to them? What feedback did they have about their experiences?
Watson recognizes that “most schools don’t have the time or resources” to pilot an entire system. But he emphasizes that the first time the system is used should not be after it has been bought and installed. If a full pilot is not feasible, he says, be sure to set aside time for different user groups to try out the LMS in a smaller way before making a decision.
5. Talk to people from schools already using the product you’re evaluating. Look for schools or districts that are similar to your own. Vendors can suggest references, although some ed-tech experts advise educators to find examples independently.
Doing site visits with similar school districts that have the system in place can be a rich and informative part of the evaluation process, says Jhone Ebert, the chief technology officer for the 311,000-student Clark County district in Nevada. “You learn so much through collaboration with people who are already in this space,” she says. “They tell you what to look for and the things that went well in the rollout, as well as things they wish they would have improved upon.”
References can also provide feedback about their experiences working with the vendor, she says, which can provide additional insights.
Ebert’s district will be transitioning from Blackboard’s LMS to Instructure’s Canvas in the fall.
What has been your biggest frustration with your current LMS? If you had to do this all over again, what would you do differently?
While many vendors will provide references of schools to contact, Hill of MindWires Consulting recommends pursuing references found independently of vendor suggestions. “Make sure that not all the schools you talk to come from the vendors,” he says. “What’s more useful is for you to contact people who weren’t contacted by the vendors.”
6. Evaluate the total cost of ownership, not just the price of the product.
“The number your vendor is leading with might not be the total cost,” says Evergreen Education’s Watson. Be sure to evaluate the total cost of ownership, which includes not only the licensing fee, but also professional development, support, repair, maintenance, hosting fees, and any network upgrades or hardware. Make sure you are comparing apples to apples when looking at different vendors; some companies bundle different types of features in different ways.
What is the total cost of ownership for this system? How much will this LMS cost the district over five years? What kind of technical support and professional development are included in the pricing?
Ebert, from the Clark County schools, also recommends considering the cost of transitioning from one system to another. “There’s a cost to move your content from one tool to the other,” she says. Depending on how the content has been built—directly into the system or into a third-party system that will easily export the data—the transition costs could mount quickly, she says.
7. Remember that this is not just a transaction, but a relationship that will potentially continue for years. Make sure the company is a good fit. Examine its track record. Explore what markets it has worked in.
“If you don’t have the partnership, then it’s really hard to fully utilize the product,” says Givens of the Masconomet district in Massachusetts. Making sure that the company has had consistent “up” time—which means the LMS had few outages—and that the district would receive significant technical support and guidance were key considerations in the decisionmaking process, she says.
- Agilix Inc.
- Blackboard Inc.
- Desire2Learn Inc.
- Epsilen LLC
- Moodle
- Pearson Learning Solutions
“You can tell a lot,” Givens says, “throughout the process: How did they treat you during the evaluation? Did they get back to you in a timely way? Did they answer all of your questions? Did they work with you to really understand what your concerns were and try to provide you with whatever resources you needed?”
Ask very specific questions about the kind of support and professional development the company will be providing, says Murray of the Advanced Distributed Learning Initiative. Especially in large organizations, the system needs to function smoothly and be up all the time, she says. There will be growing pains as teachers and other staff members begin to adopt the system, and if it’s not functional or crashes—even for five minutes—you will lose your audience, she says.
In addition, the stability of the vendor can come into play, says Hill of MindWires Consulting. Find out what will happen to your contract if the vendor gets bought out, he suggests. Look at the track record of the vendor to see where the company is pushing the envelope and how well it stays on top of market trends.
Watson says he pushes back against the word “partnership” when it comes to vendor and district relationships. “It’s not a partnership when one person is writing all the checks to the other,” he says. But he agrees that cultivating a good fit between the vendor and the district is crucial. And looking into the clients the vendor generally works with can be enlightening, he says.
How quickly will the vendor respond to questions? What level of support will be available, and who will be able to access that support? Will there be a dedicated technical-support person, or will school officials be calling into a help desk when there are problems? How quickly will systems go back up if there is an outage? What happens if those expectations aren’t met? What percentage of the company’s clients is in the K-12 sector?
“Some [LMS] companies are more specialized at the postsecondary level, and every software company wants to tell you that the feature you want is in development or it’s going to happen, … but you’re going to have a much higher level of confidence if that is a provider that’s focusing on [the K-12 sector],” says Watson. “Figure out how many clients are similar to you, because then you’ve got a sense of whether they’re going to prioritize [what you need].”
