Learning Analytics: Big Demand, But Bigger Challenges?

When I stepped into the extended enterprise learning world 25 years ago, the concept of data-based decision-making was vastly different. The learning analytics process was usually ad-hoc and reactive. And all too often, data was inaccurate, incomplete or lagged behind real-time by 24-hours or more. Now data is plentiful – sometimes even overwhelming – and demand for insight is stronger than ever. This shift makes sense. After all, to compete in today’s dynamic digital economy, organizations must continuously make forward-looking decisions. The faster and more informed those decisions are, the better.

However, success depends on three factors:

• The right data
• The right tools
• The ability to apply both data and tools effectively.

Insight isn’t just about collecting data and making it accessible. It’s also about identifying relevant, useful data and applying it consistently to improve business practices and results, over time.

That’s a tall order for any organization. So how are learning practitioners responding to this need?

Learning Analytics: Top of The “Hot” Trend List

We see mounting evidence that data-driven insights matter – perhaps more than ever. For example, recent results from the 2020 L&D Global Sentiment Survey suggest that learning professionals recognize the strategic importance of measurement capabilities.

This annual poll asks one question:  “What will be hot in L&D?” This year, 2278 respondents from 86 countries cast nearly 6600 votes. And interestingly, the top choice in 2020 is learning analytics (up from 3rd place in 2019).

According to the survey’s source, Learning & Performance Institute Chairman, Donald H. Taylor, these results reflect a continued shift towards the improvement of L&D business practices.

He explained, “This is the first time that the table has been topped by an option focused on improving the work of the department, rather than a delivery mechanism or an approach to learning.”

It’s good to see analytics in the spotlight as a priority. But there’s a huge difference between understanding the importance of data and actually making it part of daily training operations.

So what will it take to put data into practice?

Learning Analytics Maturity: Tough Truths

Despite strong interest in analytics and an awareness of its strategic importance, many learning professionals are struggling to make it work.

In fact, at one of our 2019 webinars, we asked participants to indicate their level of analytics effectiveness. Not one participant claimed to be highly effective. Zero. Yet, a whopping 70% described their capabilities as either somewhat or completely ineffective.

Why are so many learning organizations lagging? The answer seems at least partially linked to a lack of sophisticated technology.

At the same webinar, we also asked participants, “How do you analyze learning behaviors and business impact?” More than half (52%) use only basic tools (standard LMS reports). Nearly a quarter (24%) rely on limited data integration between systems (LMS, CRM, AMS, HRIS, etc.). And another 19% leverage advanced data integration and/or custom dashboards. However, no one has a highly advanced toolset. (In other words, 0% use real-time dashboards or predictive models.)

Together, these polls suggest just how far learning organizations still must go to deliver on the promise of data analytics. But success requires more than advanced tools, alone. Imagine putting a Ferrari in the hands of someone who hasn’t even completed a driver’s education course!

We’re not the only ones who’ve uncovered these issues. In fact, several years ago, a Bersin by Deloitte survey found that only 10% of learning professionals were “very effectively able” to address time-sensitive business requirements with their analytics capabilities. Why? In part, Bersin conclud­ed that learning management systems are “useful in tracking, monitoring and delivering content, but less useful at reporting and analyzing.”

What will it take to raise the standard level of analytics maturity? Let’s look closer at what’s holding learning professionals back…

3 Critical Learning Analytics Challenges

Organizations everywhere seem to be swimming in data, but they just don’t have the time, expertise or tools to analyze it all effectively. When we tested this assumption with our 2019 webinar participants, 44% said that they’re actually facing all three obstacles.

To better understand how these issues affect learning organizations, we separately interviewed multiple training managers and industry experts.

Many told us that they want to demonstrate how training initiatives contribute to business performance. However, most acknowledged that they don’t have sufficient data or skills (or both) to make a credible, compelling case. What else did these interviews confirm?

1) Lack of Expertise

Too few learning professionals have sufficient knowledge and skill to extract relevant data from multiple sources, ana­lyze it and then develop actionable reports.

Organizations need individuals who know how to use learning management systems (LMS), learning record stores (LRS) and Experience API (xAPI), in concert with busi­ness intelligence (BI) and data visualization tools such as Tableau, Domo, Google Analytics, Microsoft Power BI, SAP Analytics Cloud or Sisense. But in today’s tight talent market, finding specialists with the right skill set can be time-consuming and costly.

I often hear from individuals who are baffled about exactly what to measure and how. It’s imperative for learning specialists to develop stronger data fluency, so they can effectively frame metrics, manage data measurement and help interpret results.

Without basic data competency, the business practice of learning analytics can’t take hold. As a McKinsey report says, “The real power comes from adoption…when organizations embed analytics in the operating models…and day-to-day workflows.”

2) Lack of Bandwidth

Training managers often turn to IT departments for upfront guidance from a data-savvy analyst. However, learning performance is usually a low priority for IT staff and their availability is typically limited. Nevertheless, strategic input is essential throughout the analytics process.

As one training manager noted, the value of analytics comes from more than just compiling data and generating reports. What matters most is the insight that an expert interpreter brings when taking time to uncover true objectives and map them to pertinent data.

For example, learning leaders may ask for statistics about course re-enrollments over time. However, they would actually gain more value from understanding why course promotions consistently draw a stronger response during specific timeframes.

A data-savvy analyst can pinpoint the most useful metrics and determine their business impact relatively efficiently. But scheduling time with this kind of resource can be difficult at any price.

3) High Cost of Analytics Tools

Many learning organizations say analytics software is cost-prohibitive because the upfront licensing expense is too high, or they must invest too heavily in user training – or both. This issue is significant, even for relatively intuitive “lightweight” analytics tools.

For example, in 2018, nearly half of 100 global learning professionals told the Corporate Learning Network that their analytics budget was less than $10,000. Yet the cost of necessary tools can easily exceed that amount.

In fact, effective data analytics involves much more functionality than standalone LMS or LRS reporting tools can provide. To fully understand what’s happening with an organization’s learning initiatives, data must be integrated from across the entire ecosystem − including CRM and ecommerce platforms, finance systems and more − working in conjunction with business intelligence software and data visualization tools.

Various “next-generation” analytics platforms (such as Tableau, Domo and Sisense) have been available for about 15 years. These solutions make it possible to discover business insights and recommend next-steps based on comprehensive data analysis. However, the cost of licens­ing and implementing these tools can quickly spiral – especially when organizations need access at scale.

For instance, although some BI dashboard vendors offer free trials, set-up and configuration costs have been known to reach $20,000 or more. Plus, for many of these platforms, monthly licensing costs reportedly average $70-$80 per user, even for individuals who only view and distribute completed reports.

How Can Learning Analytics Innovation Help?

In today’s data-driven business world, the need for organizations to analyze, understand, anticipate and respond to learning needs is stronger than ever. So training professionals want more than a tool that just spits out activity reports.

They want a simple but powerful solution that helps them easily discover useful insights in a timely fashion and guides them in the right direction without bogging them down. What’s more, they prefer to receive information that is aggregated and packaged in ways that are immediately useful and visually compelling.

Not surprisingly, other business professionals share these preferences. This is illustrated in a 2019 BI market report from Dresner Advisory Services, which ranks the strategic significance of 37 different BI-related capabilities.

Specifically, users attribute the highest strategic significance to reporting, dashboards, data integration, advanced visualization and end-user self-service, in that order:

Fortunately, market demand is spawning a variety of innovative solutions, including embedded analytics technologies, as well as hybrid solutions that blend specialized data visualization tools with expert services.

These advances aren’t lost on business software analysts at Gartner, whose strategic planning assumptions for analytics in 2020 include these highlights:

• The number of data analytics experts in business units will grow 3x faster than in IT departments, causing companies to rethink organizational models and skill sets.
• Organizations that offer users access to a curated catalog of internal and external data will derive 2x more business value from analytics investments than those that do not.
• Augmented analytics will be a dominant driver of new analytics and BI platform purchases.
• 50% of analytical queries either will be generated via search, natural language processing or voice, or will be automatically generated.

Interesting developments, ahead! I look forward to seeing how these improvements are integrated into new and existing solutions for training organizations.

Conclusion

Data analytics is a brave new world, especially for training professionals. But learning organizations can no longer afford to sit back and wait for their analytics maturity level to improve. It’s time to lead the way forward or be left behind.

If your organization is one of many making analytics a top priority this year, I recommend that you let your unique business objectives guide your decisions. Don’t chase new tools simply because they can produce impressive-looking visual reports. Don’t sign-up for a cool new methodology if you don’t know how it will actually move your agenda forward. And don’t assume that whatever works for another organization will also work for yours.

Above all, remember that the only truly important data is the data you actually use. Everything else is just an expensive exercise in number-crunching.

Thanks for reading!

Need Proven LMS Selection Guidance?

Looking for a learning platform that truly fits your organization’s needs?  We’re here to help!  Submit the form below to schedule a free preliminary consultation at your convenience.