Maximizing ROI with Microlearning

Watch Video


Good morning, good afternoon, and welcome to today’s Chief Learning Officer Magazine Webinar. My name is Alec, and as always, I’m going to be here today to help answer any of your general all type of questions that may come up. But before we get started, here’s a couple of things you need to know. There’s no dial in number for today’s webinar. All audio will be streamed through your computer speakers or headphones, so adjust your volume there accordingly.

You can submit questions and any technical issues you may encounter by clicking on the Q&A icon located on the right-hand side of your screen, labeled here with this top arrow. Click that icon, type in your question, click submit. If it’s content related, we may pull it during the Q&A session if we have time. If it’s a technical question, I’ll have an answer for you that should appear on your screen shortly after you submit.

You can download a PDF to follow along with today’s session by clicking on the handout icon located also on the right-hand side of your screen, labeled here with this bottom arrow. Click there and you should be able to pull down a PDF to follow along with the presentation today.

Today’s webinar is being recorded. You’re going to receive a link to that recording in a follow-up email, so please allow at least a business day before that info is sent. In that same email, you’re going to have your HRCI and SHRM certification information for today’s event, and another link to the PDF.

All right, so in today’s webinar, this is part 2 of a microlearning learning series with Axonify. This was titled Maximizing ROI with Microlearning. It’s our pleasure to have with us Carol Leaman, the CEO of Axonify.

Carol is a disruptor. She’s the CEO of Axonify, a disruptor in the corporate learning space and an innovator behind the Axonify Microlearning Platform, proven to increase employee knowledge and performance necessary for achieving targeted business results. Prior to Axonify, Carol was the CEO of PostRank, a social engagement analytics company that she sold to Google in June of 2011. So, it is our pleasure to have Carol with us. So, Carol, let’s get started.


Carol Leaman: 

Thanks very much, Alec. We are thrilled to be back doing this second part of this webinar. For those of you who missed the first part, JD Dillon, our Chief Learning Strategist gave that and I’d encourage you to go back and listen to part one of this Maximizing ROI with Microlearning Webinar series, just to get the benefit of that if you didn’t attend or get the recording from that one also.

So, what I’m going to do first is recap just a little bit about microlearning, what it is, to refresh everybody’s memories or for those of you who are new, give you a little bit of foundational information. Then we’re going to talk about microlearning and measurements, what that means, and how it requires the different mindset and approach. We’re going to delve into some of the details of that.

Finally, we’re going to end with some real life customer stories and in different markets, just to give you a really specific view of what certain organizations are doing to measure return on investment and tie that to microlearning programs that they’ve implemented in a variety of different ways. With that, we’ll get started.

So, I first wanted to just go back and let everybody know that there’s something they should keep in mind, and that’s really where are people on this spectrum of what matters most? In particular, what is the difference between what matters to learning and development and certainly historically, and what matters to the business? So, the line of business leaders in your organization.

What you see here is really just that evolution of where people have spent their time caring when it comes to learning and the effectiveness of learning. So, L&D has really traditionally been focused on what we call traditional learning data completions, task scores, how long it took people to complete specific courses, and just the reaction, the various smile sheets. That’s really all people historically have been able to gather.

Today, we’re going to talk about this some more, microlearning and the data associated with it has allowed learning and development to start to think about things like engagements, measuring baselines of knowledge, what that growth in knowledge is, levels of confidence in the knowledge that the individual learners have, and then behavior measurement, and what people are actually doing on the job. Those are sort of the things people are turning their attention to.

What the business is really interested in is the outcome piece. What’s the business data that they’re measuring and they’re accountable to that really often ties back to learning? It’s things like top line revenue growth, in the retail sector basket size, what are customers leaving stores with and how can we increase that?

In other organizations, safety incidents can be a massive cost, so how do we reduce those and the costs associated with those? In some cases, customer satisfaction. Those are the business outcomes that the business is often most interested in.

So, there is a little bit of a disconnect and we’ve actually done a survey in terms of what learning and development is focused on versus what the business needs. What we’re trying to do is draw that connection between the two so that everybody in the organization can get what they need.

The first thing we’re going to do is start with the Kirkpatrick model. Those of you in learning and development will be very familiar with this. The Kirkpatrick model is a decades old model that really gets to the heart of the effectiveness of learning and there are four levels to that model.

The first one being reaction. When I mentioned a moment ago smile sheets, basically what learning and development has had at their disposal for years is just how did the learner react? Did they enjoy the training? Did they think it was effective? Taking a survey more often than not and measuring that.

The second thing that learning and development has had at their disposal is actual learning. I just popped up there the 69.8% on level one. That’s actually the percent that this was a study done in fact by Brandon Hall Group of organizations who are measuring level one, 69.8%.

Level two in the Kirkpatrick model is learning. That’s really the measurement of the increase in knowledge both before and after the training event. As you can see, 41% of organizations do some level of measurement at level two.

The third level in Kirkpatrick is behavior. This is where we start to get a little bit more difficulty in terms of the measurement of behavior change that results from applied learning back on the job. So, what have employees done to implement the learning? Have they changed behavior based on that knowledge, and how do we know? As you can see, only 16.7% of organizations have reached this level of measurement and know in fact that the learning is being effective in the workplace.

The fourth level is what we call the Holy Grail of learning, it’s results. It is really the effect on the business, those top line revenue goals, or bottom line profitability or in the expenses lines reduction, what are the results of those things that really result from behavior change and knowledge and learning on the part of the employees. Unfortunately, only 11.6% of organizations surveyed by Brandon Hall Group are doing anything with respect to measurement of results tied back to learning.

It is the case that we’re still very much in the early stages of moving towards tying knowledge to behavior change to the measurement of the business outcome. But I’m happy to say that the future is here and we’re now able to move in that direction.

I’ll also say that for those of you interested, Bersin did a study very similar to this and came up with another acronym called HILO. It is, again, the trends in measurement and which companies are bucketed in their use of data to tie learning to business outcomes. I’d encourage you to look at that Bersin paper as well if you have an interest.

What we know is that with the advent of microlearning and the availability of tools at our disposal, we are, in fact, able to do so much better today than was possible even as recently as five years ago, and certainly 10 years ago. The game has totally changed and we are, in fact, able to move the needle in so many ways.

What microlearning has done specifically is given organizations the ability to truly start to measure those things going on in levels two through four, and in fact beyond, when you start to think about things like predictive analytics with the Kirkpatrick model. We’re going to talk a little bit about that next.

So, as a quick little recap from part one of this series, we’re going to just delve very quickly into microlearning and what it is and what it isn’t, because there are quite a few myths out there and we’re hoping to dispel them. In a nutshell, microlearning is simply an approach to training that delivers content in very short, focused bites.

It was called prior to the term microlearning becoming more prevalent, bite-sized learning, chunking down learning. It was that whole idea that’s kind of been out there for a few years but is now very commonly known as microlearning. The better part about microlearning is because it is short bites.

It’s much more easily woven into the workday than previously possible, where you’ve had to pull people into classroom or get them focused on long videos and carve a significant portion of the day away from their job in order to deliver learning. That woven into the workday is one huge benefit of microlearning.

The other big benefit is that microlearning, as it turns out, is very much mapped to how the brain is best equipped to learn and retain knowledge long-term. There are lots of scientific studies out there that show now that the brain is very, very good at receiving, digesting, remembering, and then actioning four to five bits of new information at one time.

If you fire hose a learner with many, many new concepts in a longer sort of setting, as we’ve always done historically, the brain’s ability to retain those pieces of information and then to turn them into action long-term is virtually nonexistent.

The forgetting curve kicks in and people simply forget and then start to guess, use judgments or don’t act, in fact, in the workplace doing the things that you need them to do. That’s really what microlearning is. A couple of the benefits of microlearning. It isn’t just a strategy though. It is, in fact, a shift in mindset.

What we certainly find is it’s sometimes hard for the business and for learning and development professionals to wrap their heads around how do we transition from those big, heavy learning sessions that we’ve been doing for years to moving to an environment where it’s fast, it’s fun, it’s short, it’s highly targeted to the individual. Just simply making that shift in mindset is so critical, believing that, in fact, it’s possible and starting down that journey.

How do you do that? Well, there are six things on this slide that I’m going to walk through and really they’re the essence and the principles of what we call microlearning done well. So, these six fundamental components really have to be included in an effective microlearning strategy. I’m going to briefly touch on each of them.

The first one is having a really clearly defined business goal. For microlearning to work, the first thing you need to do is identify a really specific clear measurable business result that you want to achieve. Microlearning has to be focused on solving a single problem, not just a huge unmanageable pile of interconnected issues, which is historically what we’ve done.

We’ve tried to pound lots of information into people’s heads across a variety of topics while we have their attention. Microlearning flies in the face of that. Microlearning when it’s done against these really measurable business goals allow you to validate their impact and the organization’s ability to reach those results. It starts with having that goal.

The second thing is desired employee or learner behavior. Once you’ve identified the business goal, you have to clarify the actions that employees are expected to take in order to achieve those business goals. This starts with really understanding the behaviors that your employees and learners have to demonstrate. Once those ideal behaviors are identified, then you can outline the knowledge that they need in order to execute on those behaviors.

This will help you separate the need to know information from nice to know information, which tends to consume an awful lot of training these days, just a lot of nice to know stuff that isn’t critical in fact to business objectives often. It allows you to really focus your microlearning efforts properly, and I’m going to go through this process in a little more detail a little bit later in the webinar.

The third thing that you need to be cognizant of is some proven learning science principles. The reality is learning is a science. It really does have a lot to do with the brain. While we don’t understand everything about how the brain works, we do possess a set of evidence-based principles that can help employees retain knowledge for the long-term.

Some of you will have heard about these, they’re things like spaced repetition, which is practicing a new topic repeatedly over increased periods of times to deepen memory, or a retrieval practice, which is using questions as a form of study to strengthen memory because it forces the brain to recall information and actively pull it out, or confidence-based assessment, so measuring an employees expressed confidence in a topic to improve memory and self-awareness.

By combining microlearning with these proven science techniques, it not only maximizes your investment in time and effort in employee training, but it ensure employees have the confidence they need to make the right decisions at work in that moment of need.

The fourth thing is, going back to the left side of the slide, is that anywhere, anytime access. Most employees, as we know, are completely overburdened and under-supported just like all of us, in fact. We don’t have time to stop working for hours to sit in sessions, let alone days or weeks, doing what we know is often considered to be essential training.

Microlearning sets learning opportunities into the workflow, as I mentioned earlier, using the time the learners have available during their regular shift. It also takes advantage of familiar technology used at work like mobile devices, point of sale devices, things like that that really ensure employees don’t have to move and leave their individual workspaces to learn. It often simplifies the training experience for employees. It reduces complexity for managers and provides valuable development in just minutes per day.

The fifth thing is the right fit content formats. Microlearning is focused in the moment of need, and it helps employees target most critical information needed to their jobs. This is how people learn and solve problems in everyday life, which means the microlearning experience is more familiar and comfortable for employees, that’s why it’s critical that microlearning incorporates the right content format for the message. For example, a short video is likely better for demonstrating how to climb a ladder as compared to a text only job aid. You really have to design an entire eLearning module in a way that is going to best suit the need.

Finally, the sixth thing is meaningful training and performance data. The right fit nature of microlearning increases the number of touchpoints that your employees have with learning resources that you can collect millions and millions of data points very, very quickly. Because you have those data points about your employees’ knowledge and performance, things like the types of content they consume, what they know and don’t know, and in some cases if they’re able to apply that knowledge correctly on the job, you have then the ability to use the data to determine how your training efforts are impacting your business, and you can make proactive adjustments accordingly.

These six principles are absolutely essential to that microlearning experience done well. Here’s another little bit of data which we thought was really, really interesting. It is, we’re seeing a shift from learning and development and HR people in organizations being the key buyers or the key people interested in learning solutions and learning programs.

Two, business leaders, so line of business leaders, actively seeking solutions to solve the business problems that they have. Happily, we’re also seeing a shift to learning and development getting embedded in business units and then partnering very closely with the business to find those tools, tips, and techniques like microlearning to drive the business forward. It’s a bit of a myth that only learning and development and HR are actively seeking ways to improve performance and tie knowledge to business outcomes.

What we know is what I said earlier, microlearning really has become the Holy Grail of learning measurement and that is why, part of the reason why, businesses have become so interested in microlearning and business leaders is because now they can see evidence of that key tie.

The other thing is that … Sorry, the question is, why and how? What we’re going to do is start with the why. Why is the business so incredibly interested and why microlearning? This ties back to something really that I articulated in those six principles. The first thing is that microlearning now allows us to gather knowledge at the very individual level, so person by person, what every single individual knows or does not know, not only in a subject area or a topic area, but right down to the key learning point that is very, very granular and what we often call surgical in terms of knowledge.

The other thing that we can do is then link that knowledge to on the job behaviors. There are organizations tying knowledge to behavior observations, and I’ll talk a little bit more about that in just a few minutes. If you take the knowledge, the measurement of knowledge, and behavior observations and all of those data points that resolved, and you feed it into advanced statistical models and machine learning, and machine learning for those of you unfamiliar with it, really has risen. I’m sure many of you have heard the term big data, it’s become quite prevalent the last few years.

Essentially, we now have the ability to use business intelligence tools to feed lots of data into those machines and learn a whole bunch of things about that data and what correlates. That’s essentially what statistical models and machine learning do. When you do that, you can ultimately tie your business outcomes back to what that machine learning and all of those data points expose to you. You can do that at the individual level. You can do it at the team level. You can roll it up in many, many ways.

So, I’m going to summarize everything I just said by saying this, if you take individual knowledge, so the profile of the individual learner, what they know, what they don’t know, and many other things that you can gather from their learning experience, tie that to what they’re actually doing on the job and then ingest or look at what your historical business outcomes are, and I’m going to give you actually a live example of this in just a moment, you take that, you generate billions of data points, you apply machine learning and those statistical models.

What you get are often very deep insights into how learning at that individual level is having a very direct impact on performance of the individual and the ultimate business results or the business outcomes that you’re looking to achieve. In summary, for that section, really for microlearning to make learning measurement a reality, it does take that different mindset and an approach to doing that is fundamentally different than the tools that business leaders and learning and development leaders have had over decades.

Here’s that very specific example I’m going to run through, so that you can get an idea of how this works in action. We often prescribe when we’re talking to organizations, it goes back to one of those principles I mentioned, what are your key business outcomes that you’re looking to achieve through learning?

Let’s just say, as an organization, that one of the key things you’re looking to achieve is zero recordable ladder incidents, because in your organization, they happen to be costing you a significant amount of money. As we know from organizations that we work with where safety is a concern, ladder incidents can be an incredibly costly type of incident.

So, that’s what you’re looking to achieve. That’s your business outcome at a very, very granular level. Now, what do people need to do? Let’s decompose the behaviors that your learners or employees need to actually exhibit in order to achieve that business outcome. This would be one of those dos, and in some cases, you could have 10 things that the learner or employee would need to do in order to achieve the outcome.

In this case, an example of a do is, was the ladder climbed in the proper way? That is a behavior. So, here is how some organizations are capturing that. They’ve decomposed those specific behaviors that are required to achieve ladder safety incident reduction and simply toggling correct or incorrect by observing those behaviors being performed in the workplace. Each one of those observations, correct or incorrect, is another one of those billions of data points correlated as part of the entire process.

When you aggregate enough of them across behaviors, you can get a very, very rich data set that measures the effectiveness of the behavior in terms of whether it’s being done correct or incorrect in volume. Once you have those behaviors, now, and when I say behaviors, behaviors decompose into those group of things that the employees or learners need to do, now you need to think about okay, if they’re going to exhibit those behaviors, what do they actually need to know? What’s the knowledge that the employee needs to have in order to give them a shot at doing it right the first and every time?

Here’s an example, again. One of the things they might need to know is, did they gain knowledge on the proper technique for climbing a ladder? We need to convey to the learner how to do it properly. That’s one piece of knowledge that they need to have. So, how do you do that? Back to microlearning.

Well, you can measure very specific things with respect to ladder safety by asking questions of the learner around the specific procedures to follow, what makes ladder climbing safe or ladder climbing dangerous? Again, as each employee answers these questions, you’re gathering very specific data on what they know and don’t know in that topic area, and whether here they’re getting things right or getting things wrong. Again, at the very granular level, person by person.

When you have enough of those data points, you can very quickly see topic area by topic area what the baseline level of knowledge is and what knowledge is growing to, again, comprised of millions of data points based on the actual knowledge demonstrated by your learners. So, what that allows you to do is create a behavior and knowledge profile of each of the individuals. What have they been observed doing? What do they actually know in those topic areas? Then that gives you a very, very robust initial data set to be able to tie to measurement.

So, what’s the beginning of this? Well, the beginning of it or I should say the end of it, but it looks like it’s the beginning, on the left side, of these series of things that tie together is the actual content. As has been true mostly in the past, and because these tools and techniques and this whole advent of microlearning didn’t exist, essentially what learning and development had to do for years was start simply with creating content. We need content in the area of ladder safety. We’re just going to create as much as we think we need to have and then push that out.

What we know now is content should actually be the last thing that you create. So, once you’ve decomposed your behaviors, you know what business results you need to get and you can measure what people know and don’t know, it allows you to really think about what’s the most effective content, the surgical content I need to create to impact behavior.

Something really interesting that we’ve discovered many times now is that the content that learning and development leaders and business leaders often tend to think is the most impactful in fact isn’t. What you can do when you gather this level of data is figure out what content is good and what content is an issue. As an example, if you know what your business results need to be, you are observing behaviors and recording them, if the behaviors are poor and you know that they don’t know, then you know you’ve got to deliver more knowledge into those individuals because they don’t know and the behaviors are poor.

But we’ve had situations where the behaviors are poor, so just going back to, I’m just going to go back to this slide, the behaviors that do are poor, the employee, the learner is not exhibiting the right behavior. They actually know a lot, so their scores in answering questions or going through content is quite high. So what that allows you to do is discover that if the behaviors are poor but the knowledge is high, you’ve actually created the wrong content.

We’ve seen a number of organizations have to go back through and recreate content because it’s not delivering the kind of knowledge. It’s delivering lots of knowledge and knowledge that the employees successfully complete, but it’s not the knowledge needed to drive the right behaviors and business outcomes. So, getting super granular using microlearning allows you to gather insights exactly like that, that make your efforts going into content creation much more highly effective.

So, the next thing I’m going to turn to are three microlearning measurement stories that I think are super interesting, and demonstrate different applications of microlearning and how these kinds of organizations are getting some pretty significant results across thousands of people and really how it’s changed the game for each of these organizations.

The first one is Bloomingdale’s. Many of you recognize it’s a very, very well respected storied brand in the retail space in the U.S., Bloomingdale’s is a subsidiary of the retail giant Macy’s. It has more than 35 upscale departments and home stores in about a dozen U.S. states. Bloomingdale’s was really frustrated that its safety incident rate, and for those of you in retail sector know safety is a huge concern in retail in fact, it was far above the industry average and what was acceptable by their executive team.

In fact, seven stores were costing millions of dollars a year in workers’ compensation and general loss. They had a particular issue with certain stores. They decided a few years ago to apply microlearning across their brand, across more than 10,000 employees and really focus their content on the loss prevention awareness program.

What are the results? What we’ve done here is we’ve tied their specific results, and again, as you’ll see with the other examples, to that Kirkpatrick model, the levels one through four and how Bloomingdale’s has been very successful getting through all four levels. Level one, reaction, what Bloomingdale’s has seen is greater than 90% voluntary participation of their learners, their employees, logging on three or more times a week to voluntarily learn something new. 72% of them when surveyed stated that they preferred microlearning to other forms of learning that they had been doing previously.

Level two in the Kirkpatrick model, Bloomingdale’s saw over double digit knowledge growth. 86% of their associates said that they knew that microlearning had increased their knowledge and their confidence in their knowledge, which really then ties to level three, which is action. So, 83% of them said that microlearning had helped reduce shortage and had prevented accidents by changing their behavior. There was a real recognition on the part of the individual learners that this type of learning had had a very measurable impact on what they were doing day in and day out on the job.

Finally, the best part of all, that level four measurement, which has been so elusive up until recently. Bloomingdale’s has seen $2.2 million a year savings since the introduction of microlearning in 2012. That really comes from a 41% reduction in safety incidents. They’ve had a great deal of success and continue to have employee microlearning in what is sometimes difficult environment where you have high turnover, learners who don’t necessarily stay around and reap the benefit of the investment that Bloomingdale’s has made in them over the long-term.

The second example I’m going to talk about is a completely different organization. This is British Telecom. So, many of you are from overseas would know that BT is a division of the British Telecommunications Group, BT Consumers specifically. They’re the largest provider of fixed voice broadband, TV, mobile services in the UK. One of their top business priorities was to improve customer satisfaction while onboarding more than 2,000 advisors in 12 months. They had taken a survey of their customers on what they needed to do in order to improve customer service.

There were very specific actions, like for example, we do not want to speak to an advisor who’s overseas. We want our call resolved by the first person we speak to not the second or third person. BT had that challenge to really, how do we very, very quickly train lots of new advisors and move the needle on all of the things that matter to our customers as far as customer service goes?

With the introduction of microlearning, what they saw at Kirkpatrick level one was voluntary participation, more than two times a week, and again, these are often hourly paid, high turnover sorts of employees, and 96% agreed that their knowledge growth began from the very first day that they started to use microlearning. Level two Kirkpatrick, they saw knowledge boost to 86% from a baseline of 69%. That’s in the BT connections group which was one particular are of the business that they had some customer service issues around.

They saw knowledge go from 76 to 91% in the sales and retention group. This is people selling product on the phone to existing customers and retaining customers when they call in with an issue. Their individual rap or advisor confidence increased dramatically as they self-assessed their confidence in the knowledge that they had to address those customers.

Level three, they can now answer customer questions much more quickly and accurately because the microlearning they recognize it and the BT leadership is recognizing that. Level four, those very measurable ROI items or business results, they have seen a 2.3%, 10% reduction in repeat customer calls, which is huge in terms of thousands of calls that they have now not had to take because they are repeat.

14 second reduction in call handling time, that’s allowed them to also answer 8,000 more calls per year. They’ve seen a 5% improvement in customer value and a 2.7% improvement in connection rate. These are all very common measurements in the telecom industry that are absolutely essential to customer service in the telecom space.

The third example that we have is Walmart Logistics. As I’m sure most of you know, Walmart is an American multinational retail corporation that operates a chain of hypermarkets, discount department stores, grocery stores, in fact, it’s the largest retailer in the world. The third largest public company in the world and the biggest private employer. They generate $444 billion worldwide and have 2.2 million associates, so absolutely massive.

Walmart’s key issue was that it was frustrated with the very high cost of safety incidents in their distribution centers. They have over 80,000 employees in about 150 locations costing hundreds of millions of dollars a year. Distribution centers are extremely dangerous places to work, and Walmart has an extremely high interest and culture of safety and are always looking to be superlative in terms of keeping their employees safe, not just at work but also at home.

So, what was their experience in using microlearning? Well, again, you can see stellar results here. They, in terms of level one reaction, got greater than 90% voluntary participation right out of the gate and continue to have that today, so six years later. When you think about that, these are employees in a distribution center who don’t necessarily have access, ready access to learning, but they have, through some very creative and strategic ideas, made learning accessible to those employees.

In fact, it’s so prevalent that employees frequently ask whether they’ve Axonify today, which is their learning platform. The second thing is on level two of Kirkpatrick, they’ve seen knowledge growth in safety topic areas that are critical of 7 to 10% and those continue to be sustained today, so a great degree of knowledge lift.

They also observe, in terms of level three, action behavior, a million behavior observation amounts, very specific observations to things like ladder safety, for example, are employees demonstrating the correct behavior and actions that tie back to that knowledge? They have a very high knowledge behavior correlation. The end result is that they’ve seen a 12% decrease in medical incidents and a 30% reduction in lost time injuries, which equates to a significant amount of money for Walmart.

So, as you can see, all three organizations have had fantastic results employing microlearning. Going back to the beginning, what I showed in one of the first slides, is really traditional learning data that has been the only thing available to learning and development professionals and the business has now been able to be tied to what matters to the business, which is that growth in sales, decrease in expenses, and increase in bottom line profits.

So really everybody now has the ability to be on the same page trying to achieve the same things to measure everything along the way, and in fact, get to that Holy Grail, which is Kirkpatrick level four. It’s a new frontier and if you’d like to learn more, we’d be happy to help you.

With that, I am going to turn it back over to Alec. Excuse me. I’m just going to cough for a second and if you would like to chat with us, we are going to be at ATD, and we’d be happy to be show you in the booth, give you a demo and all of that good stuff, and talk to you about tying learning to measurement very specifically. Back to you, Alec.



All right, Carol. Thank you for that. If you have any questions for Carol, please click on the Q&A icon, type it in, and click submit. We do have some questions here in already though, so we’ll get started with these. Let me find them here. Okay, this one is going to come from Becky, Carol. Becky’s question is, “Can you share any examples of measuring success for leadership behaviors?”


Carol Leaman:

We have not actually had an organization yet give us the data that I can say with absolute certainty has changed behavior from a leadership perspective. What I can tell you is this. It is used for leadership change management sorts of topics, effectively managing people kinds of topics. Anecdotally, we have had a number of organizations tell us that it is being effective, not just the organizations themselves but the leaders or new leaders going through some of those programs.

So specific, it’s a little bit harder where you have leadership concepts to identify those specific business outcomes. It’s easier where you have more of a procedural related thing, but certainly from a behavior observation standpoint, it’s entirely possible to identify and capture the behaviors associated with demonstrated leadership qualities. So, again, people are using microlearning for that. It’s just a little bit difficult to tie it to the business outcome and we have yet to have a customer provide us certainly the business outcome data, but anecdotally, absolutely.



All right, perfect. We had actually a few come in about this next that are right, they’re all similar, they’re all in the same vein as this question here, how do you typically consider microlearning in terms of length?


Carol Leaman:

We consider microlearning, if it’s a video, for example, or a microlearning score module of some kind, we typically prescribe no more than about three minutes. In three minutes, you can effectively convey those four to five pieces of information that really do map to how the brain best works to remember anything. Beyond about three minutes, you start to lose the effectiveness of driving those key learning points.

If you’re using a question based reinforcement engine like Axonify afterwards to reinforce those key learning points, you really only want to be asking about three to five questions, reinforcement questions a day. So, three-ish minute video, three to five questions, reinforcement questions, and those work extremely well to drive knowledge very quickly.



All right, perfect. We had a few come in asking where they can locate part one of this webinar. I know we still have it on our website on\webinar then you just have to click on the previous events tab. But Carol I don’t know if you know, do you have it posted anywhere that they can find on your website?


Carol Leaman: 

Absolutely. If you go on the website under the resources tab, you can access part one.



Okay, perfect. All right, this one here comes from James. James’s question is, “Do you know how Walmart provides accessibility to its employees to access all this microlearning?”


Carol Leaman: 

Yeah. There was a general manager in one of their distribution centers who had an absolutely genius idea, and it was that he recognized that each of the employees needed to charge a battery in their handheld device once a shift. So, they would go down to the battery charging station, put the device in it, and then stand there for three minutes waiting for the battery to charge.

So, what he came up with was, what if we put a fixed kiosk or terminal right beside the battery charging station so instead of just standing there, they can actually do their training? That became a very, very effective access point. At the end of the day, it really is about thinking about your environment and what can be easily woven into the workflow of the learner to be the least disruptive and we can certainly help with that. We’ve got many, many ideas that various organizations have used.



All right, perfect. This one kind of ties in to exactly what you were just saying of knowing the environment and how you can work it in, but this is about higher education. Judy’s question is, “How could this microlearning be used in higher education that is content and lecture heavy?”


Carol Leaman: 

That’s a great question. It can be used really as part of a blended learning strategy. It’s simply the case. This is true of corporate organizations as well. Having events or sessions that are longer absolutely is essential in some cases to lay foundational information. Where microlearning can be highly effective is reinforcing the key learning points that come out of it.

So, what are the five typically an hour’s worth of learning in a classroom based setting, there can be extracted no more than about 10 key learning points. So, if you extract those 10 key learning points, create questions around them and then use those questions to reinforce, in short bites, three to five questions a day, you can very effectively reinforce that classroom based instruction.



All right, thank you, Carol. This one is going to come from Sarah. Sarah’s question is, “Sometimes the level three and four data seems kind of similar, for example, behaviors versus results. How would you recommend clearly differentiating between the two?”


Carol Leaman: 

Yeah, sometimes they can get a little bit fuzzy. So, the business results are really the final leg in that journey. What money did we save as a result of, and I’ll take that simple example, ladder safety incidents? If in, let’s just say, 2017 we spent $5 million on insurance claims, medical visits, time off work, et cetera that were associated with ladder incidents and we had 300 of them. Now, in 2018, we’re on track to have 100 and much less in terms of cost. That’s really the business outcome. What are you looking to achieve there, is it dollars, measurement by dollars or incident rates and those sorts of things?

The behaviors are more, so in order to achieve incident rate reduction, not have as many or spend as much money on them, what is the behavior I need somebody to do? In the case of a ladder safety incident, I need the individual using the ladder to number one, use three points of contact when climbing a ladder, so two hands and a foot, or two feet and a hand. I need them to put the brake on. That’s another behavior. I need them to lean the ladder at the appropriate angle onto the shelf, that’s a third behavior.

So, that’s really what the behaviors are, are the specific things you need the individual to do, the actions you need them to take, and the business outcomes are the resulting financial metrics, incident rate metrics, customer calls, reduction in customer calls, those sorts of things that result from somebody doing things the right way.



All right, great. We have time for maybe one, maybe two, but this one is going to come from Benoit, hopefully I pronounced your name correctly. If I didn’t, I apologize. The question is, “How do you handle mobile access to learning when you have hourly employees and say your HR department is worried about making it available outside of work hours?”


Carol Leaman: 

That is a very, very common question that we get. So, organizations that do have that issue usually prescribe that the employees do not use, so as a matter of policy, we don’t want you using personal mobile devices and/or doing anything outside of working hours. So, that’s how they deal with it is policy.

I can tell you at a practical level, all of those organizations experience something that they don’t expect, and that is employees love the experience of the learning so much that they find the app on the mobile device, download it anyway, and do it when they’re riding the bus home or the train in, or standing in line at the grocery store.

So, you can prescribe it, and have that as policy. At a practical level, we’ve seen many employees actually want to do it so much that they do it outside anyway. It has, I can tell you, never been an issue for any of them, and we’re talking millions of learners.



All right, perfect. We did have a few more people coming in and asking where they can locate part one again. It will be on\webinars, then you just have to click on the previous events tab, or as Carol said, you can find it on in the resources tab. Is that correct, Carol?


Carol Leaman:

That’s right.



All right, perfect. That’s going to be all the time. We’re going to have to cut it here. Carol, thank you so much. It’s always a pleasure to have you on. Thank you to everybody in the audience. We had a lot of questions rolling through. Thank you for your participation. I want to thank Axonify for sponsoring today’s event. We work a lot with Axonify here, and it’s always a pleasure to work with them. So, thanks again to everybody on the line.

If you enjoyed the webinar, please take the time to fill out this post event survey, which is going to appear right as this webinar ends. Your feedback is very important to us here at Chief Learning Officer Magazine. Thanks again to everybody on the line. We’ll see everybody back here for our next CLO webinar, which will take place this coming Tuesday, April 17th at 2:00 p.m. Eastern, 11:00 a.m. Pacific. That is titled The Language of Employee Retention. Thanks again, everybody, and have a great day.