A Multi-Perspective Look at LMS References

When you invest in an LMS, you are sticking your neck out professionally by committing $100,000 or $1,000,000 to a system that touches every employee, channel partner and/or customer in your business ecosystem. It’s your responsibility to confirm that whatever you are buying is what you THINK you are buying that’s why you need to pursue LMS references.

Before you sign on the dotted line, it is wise to speak first with the LMS vendor’s customer references.  References are usually your last chance to validate your assumptions and avoid making a bad decision. However, the reference step in a sales cycle is not as simple or straightforward as it might seem at first glance.  Vendors, references and buyers are all involved.  All three have shared interests, but also have different priorities and perspectives that deserve to be considered.

This post looks at all three points of view.

The LMS Vendor Perspective

For obvious reasons, customer references are tough for average and poor LMS vendors.  A typical vendor might work 50 sales opportunities in a given year.  If every prospective buyer asked for three references, that would total 150 references calls to field a year.

That requires you to line-up a lot of satisfied customers whose business needs align with the exact reason buyers are asking for references.  If a vendor is not doing a consistently fantastic job in serving customers, the demand for references will always outweigh the supply.

On the other hand, great LMS vendors run their business in a way that leads to a steady, ample supply of strong customer references.  Even so, every vendor can tell you stories of losing an opportunity in the 11^th hour due to a rogue reference response.

Vendors never know what a reference will actually say.  Will they focus on the bumps in the road or the way those issues were resolved or the quality of the results?  Are the references genuinely satisfied with your solution, or have they simply avoided telling you they will soon be switching vendors?  Is the contact having a bad day?

All of these factors conspire to create a reasonable amount of risk for vendors to consider.  To help navigate that risk, vendors are understandably stingy with references and hold them until the last possible moment in the sales cycle.

You’ll hear excuses like, “We don’t want to overwork our references” or “We only provide references upon being named a finalist in your selection process.”  This response actually means vendors are concerned about their limited pool of available, willing customers and they want to minimize the number of times each reference is approached.

If you are a great LMS vendor and run your business in a way that continuously fills the pipeline with a diverse selection of available references, you can use that as a sales advantage to win new accounts.  For example, when a selection process narrows to the final two vendors, offer your prospect five or more relevant references and challenge the buyer to ask the other vendor to do the same.  Many can’t.

The LMS Customer Reference Perspective

If you’re on the other side of a successful LMS implementation or you have inherited your LMS, a vendor has probably asked you at some point to serve as a reference.

Chances are you and your team have long-forgotten the headaches and stumbling blocks you experienced during the implementation, testing and deployment process.  Great.  What should you do?

Are you thrilled with your LMS vendor?  Then be honest about it, but remember to limit your commitment.  When approached by your LMS provider, say “Yes, but I don’t want to be contacted more than a few times a year.”  More than 4-5 calls a year is probably too much — unless you’re a saint, a sadist or you’re on the payroll.

Not-so-thrilled with your learning platform?  Then use this opportunity to get your grievances on the table and ask the vendor to resolve these issues before you serve as a reference.

Ecstatic or not, you need to be crystal clear with your vendor about what they should expect.  Tell your contact that you will tell the truth, good or bad, and do exactly that on every reference call.  Of course, vendors hope you will emphasize the upside, or at least do no harm.

However, if your assessment isn’t fair and accurate, you are ultimately hurting yourself, your vendor and the potential buyer.  Remember, someday you will be the one interviewing a reference, and you will appreciate the same honest courtesy.

To prepare for the reference call, ask the vendor for a written executive brief that outlines the interviewing company, business need, scope of LMS project, critical functional requirements and the proposed solution.  You also want to know the names and titles of anyone likely to interview you and their reference research goals.  Also, prepare yourself to answer the questions in the next section.

The Interviewer Perspective

The goal of any reference call is to gain an understanding of how a vendor actually conducts business, implements, supports and treats its customers – especially when things are not going as planned.  It’s also important to see how the LMS product and services matched the reference’s expectations.

Since most references will throw out fluff answers to softball questions, your job as an interviewer is to ask penetrating questions and then probe to learn more — much more than the reference planned on sharing.

Here are some questions I like to ask LMS references when helping organizations buy an LMS:

BUSINESS QUESTIONS

1. What is your role in the organization and with the LMS vendor?
2. How long have you been a client? How would you describe your experience?
3. Were you part of the original selection team? If yes, why did you select them? Have they met your expectations?  If no, how long have you been working with the LMS?
4. How often are you asked to be a reference?
5. Do you view your LMS vendor as a strategic partner or more of a software product?
6. Is your vendor proactively helping you evaluate your progress, plan for future?
7. What do you like best about the LMS vendor? What do you like the least?
8. If you could go back, would you select this vendor again? Why or why not?
9. Do you know the name of your main support contact?
10. How does this vendor compare to others you have worked with in the past?

IMPLEMENTATION QUESTIONS

1. How did the implementation process go? Was it the same as explained in the sales process?  If no, what was different?
2. Did the vendor propose the right amount of services or were you plagued with change orders?
3. How long did it take to be assigned a project team and kick-off your implementation after you signed?
4. Was your assigned project team up to speed on your company and LMS mission when they met you or did you need to educate them?
5. Did the project team have a deep or cursory knowledge of learning technology and similar implementations?
6. Did the implementation process feel like checking off steps or focus on short and long-term success?
7. What kind of personnel turnover have you experienced with the vendor’s sales, support, implementation team or executive contacts since you became a customer?
8. Did the vendor meet agreed-upon implementation project milestones and deadlines? Why or why not?
9. Is there anything you would recommend we do to get the best possible implementation from the LMS vendor?

PRODUCT QUESTIONS

1. How long has the product been in a production environment?
2. What is the feedback from learners and administrators? Positive? Difficult?
3. Are you having any problems with functionality?
4. We plan to use the LMS primarily for _________ and ________. How easy, intuitive or clunky is it to do those types of tasks?
5. Were there functionality gaps you didn’t expect? What?
6. What doesn’t work in the product the way you think it should?
7. Do you feel the application is responsive and fast?  Does it crash?

ONGOING SUPPORT

1. Does the vendor meet their uptime and available service level agreement?
2. We plan on using the vendor for both admin and/or end-user support, can you describe your experiences with both?
3. How long does it take to get product issues resolved? Is it what you expected?  What would you improve if you could?
4. Do you have the same team supporting you or is it new contacts with every call?
5. How many support tickets do you currently have open?
6. Describe your experience with upgrades, updates and fixes? Easy?  Amount of testing required?  Cost vs. expected cost?
7. How much input do you have or not have into new LMS features?
8. Do you participate in any of the vendor’s customer user groups, online communities or conferences? How useful are they?

Conclusion

There is not necessarily a perfect set of answers to the above questions and every buying organization has different needs, priorities and risk tolerance.  However, without asking the questions, a buyer’s only choice is to assume the vendor has everything they want the way they want it.  LMS references are a good reality check.

Vendors, buyers and customer references all share a joint interest in successful reference calls:

• It is the vendor’s responsibility to develop a deep pool of genuine references because it means they are providing customer value.
• It is the reference’s responsibility to speak only the truth and offer a fair and balanced view of their experiences.
• It is the buyer’s responsibility to interview at least 2-3 references and ask the right questions.

LMS buyers seldom get a chance to make a second LMS selection after a first failure – at least not for the same company.   Use LMS references to learn and confirm.  If nothing else, any surmountable hiccups buyers discover can be addressed in the final contracts.  Caveat emptor.

Thanks for reading!

Need Proven LMS Selection Guidance?

Looking for a learning platform that truly fits your organization’s needs?  We’re here to help!  Submit the form below to schedule a free preliminary consultation at your convenience.