Transcript

5 Reasons to Rethink the LMS

Watch Video

Nissa

Hello, everyone, and thank you for joining us today on our complementary webinar, Five Reasons to Rethink the LMS. My name is Nissa, and I will be your moderator for this session. Today’s presenters are David Wentworth, principal learning analyst for Brandon Hall Group, and JD Dillon, principal learning strategist for Axonify. At this moment, I’d like to extend a thank you to Axonify for sponsoring today’s webinar. Axonify is the world’s first employee knowledge platform. It combines and award-winning approach to micro learning, with innovative knowledge on-demand capabilities, and the entire experience is gamified, driving high levels of participation. For more information, please visit their website at www.axonify.com.

And now, a little bit about Brandon Hall Group. Brandon Hall Group is a research and analyst firm that empowers excellence in organizations around the world through our research and tools. Our vision is to inspire a better workplace experience. We have world class research, data, and expertise across all areas in human capital management. Currently, we have open surveys in extended enterprise learning, and HCM technology. We would love to hear from you. In exchange, you could receive free gift cards, research reports, and more. To participate, please visit our website www.brandonhall.com, and click on open surveys.

This presentation is being recorded. Access to the playback and slides will be provided to everyone within 48 hours of the webinar in a follow up email. If you would like to ask a question, please do so by typing it into the question box for our presenters. If you would like to download a copy of the presentation now, you can find that in the handout section. And now, I’ll turn it over to David so we could get started with the agenda.

 

David Wentworth

Great. Thanks, Nissa. I do want to leave this screen up just for another second or so, if you do have any questions, please feel free to type them in the question box there. You don’t have to wait to the end. I like to make sure you can ask your questions if it’s specific to something on the page, and we’ll try to answer them as we go. We will leave some time at the end to answer anything we may have missed, or any new questions as well. But, please feel free to ask questions, or submit comments, or what have you through that question box as we go, so thanks for that.

So, basically as we get started here, what JD and I wanted to talk a little bit about is what’s happening right now in the learning technology space, and what one of the big challenges organizations face is, is that when it comes time to make a decision about learning technology, it used to be relatively simple. It was, “I need an LMS, and now I’m going to go out and I’m just going to figure out which one I want.” That was the hard part, it was there were so many. Well, we still have many, many LMSs to choose from, but the technology question has changed quite a bit, and so, the learning environment is different in which we’re making these selections.

So, if we think about what the LMS is, right? Which is the learning management system, I’m hoping most folks are pretty familiar with the LMS is, but essentially, just so we’re all on the same page, it is, at its core, it’s a system that’s designed to host, track and deliver online, and in person training courses. That’s really what the LMS does. It manages this type of learning. Now, as time has gone by, various different solutions have branched out to incorporate other features and functionality, and so forth, but at the end of the day, this is what it’s designed to do, this is what it’s core functionality is, and because of that, most LMSs tend to be very heavily focused on the 10 of 70:20:10 framework.

For those of you that might not be familiar with that, that’s essentially some research that’s been around for 20 years or so, that indicates that when people learn, about 10% of what they learn at work comes from formal experiences like a class, or a course. 20% comes from informal learning experiences, and 70% comes from experiential, or on the job type of learning, trial and error learning by doing, that sort of thing. So, with all the things that people learn, only about 10% comes in a formal experience, yet that is what’s the biggest learning technology expense is dedicated to, is delivering on that 10%.

So, we find ourselves in a situation where the LMS tends to be falling short when you look at the bigger picture of the learning. We start to see that in our research. We ask organizations every year about their experiences with the LMS, and we’ve been recently asking companies to start thinking in terms of formal, informal and experiential when they think about their technology. And, for those organizations that are using an LMS to deliver learning, we find that the informal and experiential functionality of these tools is not so great. On a scale of one to four, one being not at all satisfied, and four being very satisfied, we asked organizations … We asked them about across a wide array of characteristics that are LMS, but we asked them, “How satisfied are you with the informal learning experience, and how satisfied are you with the experiential learning experience?”

You could see here, you have a 2.23 out of four for informal, and a 2.20 for experiential. So, they’re falling quite short here, on the ability to deliver on those types of learning, and not just because of this reason, but because of a whole merited of reasons, one of the main ones being challenges with ease of use. 44% of organizations say that they are currently considering replacing their LMS, and for those of you who have done any sort of technology replacement, you know what a big challenge that can be, and it’s not something that’s taken lightly. So, the fact that nearly half of organizations are considering making that change, based on the dissatisfaction with what they’re able to deliver, is pretty telling.

We did have a question come in, I’ll answer this one right here, just because I’m sure other people have this one. It’s about the handouts and someone is unable to actually download the handout from the control panel. If that’s the case, of if you just don’t want to do that today, an email will go out with a link to a recording of this session, as well as a PDF of the slides.

 

Nissa

David, if you give me a minute or two, I’ll delete it and try to re-upload it again, hopefully that’ll fix the problem.

 

David Wentworth

Well, I don’t know if you read it, it’s an employer firewall challenge. I don’t know if you need to do that. So, what JD and I wanted to talk about then, is I sort of set the stage here with that research showing the dissatisfaction, that the LMS is sort of failing organizations, in terms of a current modern learning experience. So, what’s going wrong? So, JD and I want to walk through some of the things that organizations are running into with their LMS. So, for those of you who are thinking about looking into getting an LMS, or some other learning technology, or maybe replacing what you have, start to think in terms of these things and what other organizations are seeing as challenges, so that you can think differently, and start to look in other directions, as you’re trying to find your own solutions.

So, the first thing that we see, is that simply that the traditional LMS can’t scale learning to keep up with the pace of business, right? So, going back to that fact that the LMS is so focused on delivering that 10% of formal learning, delivering courses and classes. You know, if you think about learning, what is a class? It’s, typically, and hour, and hour and a half, two hours, maybe it’s a two day class, in person. A course, how long do you think a course should take to click through? Is it 45 minutes to two hours? Four hours?

So, not only just in terms of the time it takes to actually consume that learning, but actually to create it, and develop it, and deploy it. Learning is supposed to be solving business challenges, and if you think of the amount of time it takes from conception to delivery of one of these big learning programs, most organizations can’t move fast enough to solve a business challenge by going through this process and delivering it in a traditional way, because everything in work is changing, right? We no longer operate on hierarchy, or so focused on process. Things just don’t occur in the top down, homogenous way. Everything’s changing.

The organization is connected completely differently. It’s now a wirearchy, and a lot of the things that occur within an organization, whether it’s technology or otherwise, they seem to happen from the ground up, grassroots things catch on, and work and have success, and so they build from there, as opposed to be driven from the top down. It’s all about being agile, and being able to adapt quickly to change. The old methods of doing things simply don’t allow organizations to do that, and it’s not any different for learning, right? The idea that everything focused on classes, courses, the ADDIE model, I don’t know if everybody’s familiar with that, it’s an instructional design method, which is actually based on a software development model called Waterfall, which software developers have been abandoning for 20 years because it takes too long.

Organizations need to get more agile with the way that they develop their learning, and more iterative so that they can adapt over time to the changing needs, and so, we’re looking for learning experiences that are more social, and collaborative, involve more video, available on mobile devices. There’s a heavy focus on context and personalization, right? “What does this learning mean for me? How does it fit into my life? How does it related to my role?” And again, traditional methods, typically, can’t deliver on that. You can’t give that type of experience. And so, JD, I want you to come in and say, “Hi,” to everyone. Talk a little bit about how learning is making this shift from the scheduled learning to continuous experiences.

 

JD Dillon

Thanks, David. As I think you alluded to in our first point, is while we’re having a conversation that’s based in technology, we’re talking about system and the LMS, and what not, what we’re really talking about, and what I think is essential in maintaining pace with the business, is shifting from a technology focused conversation to an experience based conversation. Creating an experience, and enabling an experience that provides LND in the business with the opportunity to insert the right content, and the right inputs, and the right experiences in order to solve problems at the moment of need, and remain agile with the needs of a modern organization.

So, the image on screen is a simple break down of what could a continuous learning experience look like, and how do you potentially facilitate that with a mixture of tactics. So, in this case, I think it’s not about replacing everything that we’ve done, one of the reasons we’re talking about rethinking the LMS, not throwing away traditional tactics, is that we’re still going to leverage a lot of the things we talked about, in terms of events and online content, and what not, but I think there’s an opportunity to make them a lot stronger if they’re not the everything of LND strategy.

If they’re just a component of an experience that we drive that really wraps around the individual, and drives concepts like continued motivation, and reinforcement in behavioral assessment, and data, and feedback, that is a constant, perhaps even a day to day learning opportunity, and then those inputs things that are more scheduled, and structured natured get inserted into that flow when there’s a larger, kind of more complex, or compliant potentially, oriented need. But, it’s really about driving continuous learning to achieve the results you see on the right hand side, rather than trying to schedule, and control, and own that experience, because the reality is learning is continuous, so how do we leverage the right tactics, and resources, and technology in order to support that, and as you alluded to earlier in alignment with the 70:20:10, reality of work place learning.

 

David Wentworth

Mm-hmm (affirmative). Yeah, and I think that’s a key point to remember whenever we talk about these things, I think, for folks like me who do a lot of this research, and do a lot of presentations on this, I tend to focus on the emerging technologies, and the new, and the future, and sometimes it gets interpreted as, there will no longer be classes, or courses, and that’s clearly not the case.

I mean, instructional classroom training had been the number method for learning delivery, as far back as we’ve been doing research, and it probably will be for the foreseeable future. It’s the way in which we’re able to most efficiently deliver large amounts of knowledge, and as JD mentioned, it’s just a question of rethinking how we approach it, right? We’re still going to do these things, we’re still going to have courses, we’re still going to have classes, but how do we make the most of them? How do we change them to fit into this type of learning ecosystem where all this other type of learning is happening in and around those events.

Make no mistake, the technology is not what’s making these other learning experiences happen, all of this learning is occurring regardless of what you do, right? That’s the nature of the 70:20:10. People are taking these courses, and these classes and they’re learning things, but that’s only 10% of what they take in. Everything else they take in is from talking to colleagues, just doing their job, experiencing life, so those things are happening. The challenge is, how do we facilitate those things? How do we bring them into the learning ecosystem? How do we enable them with technology that allows us to match that? And so, we can make the most of it, we can help people do these things better, we’re able to actually guide them so that they’re learning the things that the organization needs them to, and that we’re able to measure them and see if it’s having a positive impact.

So, it isn’t about completely forgetting how we deliver learning. It’s about just changing it, and adapting to fit the way people learn on an everyday basis. This is the next thing that leads into that, how people learn, and how people just operate in their daily lives. The traditional LMS just isn’t able to meet the needs of the deskless workforce. If you think about it, just by definition, the LMS has been something that you either get scheduled to go into a classroom, or you are given a course that you sit at your desk, and have to click through the course. Here’s actually a question that speaks to this, and JD, maybe I’ll let you hit this question once we get into this point a little bit, but the question is, “We have many employees who are not technology oriented, or don’t have regular access to computers. How do we reach those employees?”

They fall into this category of the deskless workforce. Now, there’s a couple of ways of thinking of the deskless workforces, and the first thing that usually comes to mind is the fact that, okay, mobile devices, right? We’re deskless, we’re moving around, and everyone’s got their smartphone in their pocket, that’s one part of it, and we do see that millennials are now, the largest generational group within the workforce, and it’s not as though they have the sole rights to new technology. I think if you ask just about anybody across this generational span, most, all of them would have probably a smartphone in their pocket, and know how to use it. So, it applies to everybody, but it isn’t so much that we’re trying to cater to millennials.

They are just the new generation that has come up with this stuff natively, and just used to operating this way, and so, whereas, the other generations, were able to adapt as the workplace adapted. There is no adaptation for the millennials, this is just how it is, and so, they’re coming into an environment where, maybe, that change hasn’t already occurred. So, that’s one part of it, is the fact that you’ve got these millennials, and other different types of technology that … And people are working from home, they’re working from the road, and so, there’s all these different opportunities to deliver learning that don’t necessarily mean someone has to sit down at a desk in front of a computer.

So, the idea is that trying to bring learning into the workflow, whatever that workflow may be, for that particular group, that particular role, whatever it happens to be. So, some of the characteristics of embedded learning, being able to bring it to that point, is making it contextual, so the learning that you’re delivering isn’t just generic. It speaks to the learner, whether it’s because of their role in the organization, their literal location. Maybe it’s their geographical location specific to them, their own development path, essentially connecting learning so that the learners say, “Well, I have this learning in front of me, and I’m taking it, I’m looking at it because it means this to me, and my role within the organization.”

Being able to make learning small, right? So, short videos, quick updates, things like that. That’s key, so you can’t ask people to take a class, or an e-learning course once a week, it’s just too much. So how else are we going to get them access to learning resources? Making it short, videos is a great way to do it, making it more informal, allowing more just everyday learning more AdHawk without as much scheduling, giving a place for people to learn from one another, the mobile piece, which I touched on, the ability for people to learn when and where they feel most comfortable doing it, as well as the social elements, right? Learning from one another, discussions, recommendations, getting a new perspective on learning.

It’s a pretty powerful tool when you have two learners taking, say, the same course, and then they have an opportunity, whether in person or in an online forum of some sort to discuss the learning they just experienced. They now get to see each other’s point of view, and they’re actually adding different levels to the learning that they’re taking it. So, that’s another key piece. And so, as we talk a little bit more about the continuous learning, JD, I’ll let you talk about, continuing on this idea of fitting it into context of work, and maybe, if you have some thoughts on that question that came in about, what about these sort of, maybe non tech workers who aren’t regularly sitting in front of the computer.

 

JD Dillon

Sure, a ton of great points there in terms of ideas around how to better support a deskless workforce, for those folks who are not regularly connected, or easily accessible by learning, and the operations teams. I try to distill this large point down to three key ideas in the context of that continuous experience that we’re looking to provide. So, as you see on screen is one. Making the right resources accessible at the right time, and place for the individual. I think every organization, even potentially roles, this is going to vary a little bit.

How do you get the resources to the individual, rather than forcing the individual to remove themself from the work, and then go somewhere in order to learn? I use my background in call centers as a great experience at this. Call center management does not want people to leave the phone. That’s where the business takes place, that’s where their goals are going to be achieved. So how do you go to them, rather than have them come to you? Then the second point, in terms of simplicity of use. I think this is where a lot of conversation comes in, in terms of talking about people who, maybe aren’t, “Tech savvy,” in the workplace.

As you said, a majority of individuals in enabled organizations, especially first world environments, are going to have access to certain types of technology, they own smartphones, things like that. But, is the technology they’re being asked to use at work simple enough to use, and familiar enough to overcome any limitations that may be there? I can speak for myself in saying, I still don’t know how to navigate a variety of HRS systems and Internets from past employers. Just so it wasn’t that it was my personal lack of technology savviness, it was the simplicity and the design wasn’t there to allow me to take advantage of a resource.

So I think that’s a big consideration when you’re looking at that as well, and there’s also the idea of delivering the right fit formate. So, in most cases, hopefully you can find a way to enable people through technology in order to be able to access them quickly, and in a variety of ways, but what’s the right formate going to be? So, as an example, obviously if you have a retail audience, you’re not going to probably want to build 30 minute podcasts for them to listen to, to deliver learning content, because they’re likely not going to have that time to remove themselves from the operation in order to consume that type of a message, even if it’s a great formate for your message.

So, what’s the right way to deliver that message? And, we’re talking about things like deskless workers. I think there’s an opportunity to start exploring new ways to access these individuals. We’re working with Axonify, with a variety of companies, we’re exploring, or maybe leveraging bring your own device policies. I’ve met with several chief information officers, and IT teams who have come to the reality to understand that if we want to advance, if we want to make our people better, if we want to make them safer, we have to start thinking about these types of ideas that may have been off the table, or maybe challenging conversations.

But, I think it’s essential to think about the way technology is designed, the way content is enabled through the right fit, simplified technology experience, and then how we leverage various types of technology, whether it be, we use a lot of break room kiosk, mobile devices, POS units inside of stores. So where are those access points that people have? And maybe it’s in their own pocket, in certain cases.

 

David Wentworth

Right, and I think you make a good point about the simple to use, we see that so much in enterprise technology, right? When you look at something on the consumer side, and when you signed up for Facebook, they didn’t ship a user’s manual to your house. You just sort of started using it, and it kind of made sense, and you made your way through. We need to get technology to be more, and more like that in the organizations, so that we don’t spend time with training or dissuade people from using it just because it’s too difficult.

It used to be relatively intuitive, just like most of the other things they use. To talk a little bit more about the question about non technology, or plant workers, we’ve seen, in my research, I’ve seen organizations do things like have a handful of tablets available in a location that allow people to access the training that they need in certain times, and they can go in shifts and rotations, as to when they need to get on there. In a retail environment, a lot of times you can deploy the learning on the point of sale machine, so that you can use one of the registers so someone could take the training while they’re there, and as the other registers are conducting regular business.

So, there’s a lot of opportunities there. Another good question, as we’re still talking about mobile, before I stepped in, JD has a good case study of an organization and how they attacked this. We have a question from Dennis about how do you fit adimensional into a cellphone? What’s the barrier of delivery? So, adimensional, and if I’m misinterpreting, Dennis, let me know, but it’s sort of like the arbitrary dimension, right? To sort of the other, beyond other dimensions. The idea really, what the barrier is to mobile is, the barrier has moved quite a bit, very rapidly recently, right?

And so, we move more into HTML5, and adaptive and responsive web design, most of the content people are building, and stuff that you’re looking at on a mobile phone, you know, you build it once, it displays and behaves on a computer, you look at it on a tablet, it behaves specific to that tablet, it behaves correctly, specific to your smartphone, regardless of what you’re on. But, there is a line of, “Listen, is it right to deliver this particular thing on a mobile device?” That’s the first question that organizations have to ask.

It isn’t just, “Well, everybody’s got a mobile phone, so everything has to be mobile, mobile first.” And there are a lot of organizations that say, “Well things have to be mobile first,” and if that’s right for your organization, then that’s the way you should think about it, but not everything by default has to be mobile ready, not everything has to be delivered in all these different ways. So, you have to take into account the audience, you have to take into account the content itself. What are you trying to get across? As well as things like the time frame. When do you need people to take this or learn this? So, all that goes into whether or not you’re actually going to use some of these solutions.

So, there are barriers for that. But, I want you to talk a little bit, JD, about the At Home case study.

 

JD Dillon

Sure. So, the work we’ve done with At Home through Axonify is a great example of this conversation around deskless workers in a retail context. So, as you see, the organization is broken apart across more than 100 stores with 3000 plus employees, and one big piece of information to think about, if you start doing the math on that is it’s a very lightly staffed operation. So, there are only a certain number of employees in the store, and they’re very focused on customer needs, product placement, and various things you need to run a retail environment.

So, the challenge is when you have such light staffing, and you don’t necessarily have the resources to deploy massive amounts of trainers out into the field, how do you reach individuals in this context, with the right information to help them deliver on your business goals? So, in this case, we leveraged the Axonify platform within an existing touch point in the break room, in terms of things like break room kiosk, in order to get access to information that way, but also to break down content through a micro wearing delivery that really, again, fit into the flow of the work day.

So people didn’t have to step away from the retail floor to go do their training for 30 minutes or an hour, or even potentially longer than that. So, it was breaking the formate and the messaging down to fit into the context of the work, and then finding the access point within the flow of work, within the store so that people could access the training on a consistent, and often on a daily basis. So, you see the weekly average participation number there of 94%, means 94% of At Home retail associates are logging in to their platform and getting micro learning bursts of content, at least once a week, and then potentially more often than that.

The great story around the reduction and onboarding times comes from simply the fact that, in this environment, they didn’t have the ability to reach their employees during an onboarding setting very consistently. So often it would take months to get trainer to the store, to get a regional manger to visit the store, to cover certain content, or to get certain messages out there. But, because they established this consistent experience, this continuous learning idea that we talked about earlier, and they leveraged the reality in the context of the disconnected, or deskless retail environment, they were able to get that information to the employee faster, and ramp them up into their basic skill set to get them on the floor, and productive as an employee much quicker, because they adopted this type of an approach to technology and able learning.

 

David Wentworth

Great. So, the next thing that we run into with the LMS is we get right into business results. At the end of the day, that’s really what learning is supposed to be driving, right? I mean, the only reason why any piece of learning exists is to drive some sort of business result, and that is a true statement all across the learning spectrum. I mean, there shouldn’t be any aspect of learning that is done solely for the sake of learning, even if it’s compliance. There’s a business reason behind the compliance training, whether it’s keeping the organization from being audited, or just making it a safer place to work, all the way up through leadership development.

It’s all about making the organization better, and time, and time again, we find the more traditional technology, the more traditional approach has a hard time moving the needle there, and what you end up with is organizations that get very focused on the basic outcomes of learning, like completions and smile sheets, rather than is learning moving the needle forward on business. So, one of the things I want to point out there, and the next couple bits of research I’m going to show you, in our studies at Brandon Hall Group, we segment out our data in various ways. Sometimes we look at different company sizes, to see if there’re differences. We look at different industries, to see if there’re differences, and we also look at organizational performance.

We take a group of organizations whose key performance indicators, like profitability, customer satisfaction, as you see here, those things are moving in a positive direction every year, and then we separate them out from everybody else who, maybe not all of those things are moving in that positive direction, and try to see if they’re thinking differently, doing things differently. And so, what you’ll see is, in terms of organizations saying, “How effective are you at improving performance with your learning programs?”

Now, overall, these numbers are somewhat disheartening. You could see that the green line is high performers. So those are those organizations that, like I said, have those key performance indicators moving in that positive direction, and only 52%, just over half say that we are very effective, or very effective at improving organizational performance with our learning programs. 64% say, “We are effective or very effective at improving individual performance. When you look at the rest of the organizations, it’s much worse, right?

39% say that they can improve the organizational performance, and 45% improving individual performance. If less than half of organizations are able to improve, effectively, improve individual performance via learning, I’m not really sure what’s going on in their learning programs. At the end of the day, that really has to be the results of learning, and not simply getting 95% of the people to complete the course. You have to be accomplishing something with that.

So, we do see that as a big challenge, but as I said, high performers, there’s a correlation with being able to drive that performance through learning compared to the other organizations. So, when you think about how do we get learning to really drive those results, JD, you talk a bit about thinking about what those outcomes are to start with.

 

JD Dillon

Yes, and like we said, it’s about adopting an approach and a strategy for enabling outcomes first learning, and then selecting the right technology and resourceful that are going to bring that to life. So, one of the things that we work with, all of our partners through is really starting with the end in mind. I think in a lot of cases, organizations have learning management systems that just have insanely huge catalogs of content, and they think that’s a great resource. There could be some great stuff in there, but they lean on a content centric approach to learning, and then a lot of cases were inundating people with a lot nice to know information.

Unfortunately, I think everyone here has had that experience where you’re working with a subject [inaudible 00:31:31] expert to build a piece of content, and they want everything plus seven kitchen sinks included in the module. So, you end up with this world where all of the need to know is buried under the nice to know information. So, one of the things that I work with organizations to do, is again, start with that defined, measurable outcome in mind. What needle are we trying to move within the business, and where do we want the needle to go? So, do we want to increase sales? Add a certain product line? Do you want to increase safety incidents in a certain area of the business? What are we trying to accomplish? And, how are we going to know it worked?

Because, if you can’t measure the outcome of what you’re doing, how do you know if it made any type of a difference? So, what’s the outcome we’re trying to achieve, and then work our way backwards to what we need to do as LND, through what do people have to do? So, what’s the key behavior that people have to demonstrate in order for us to reach that outcome. So to sell more, what do sales people have to do? What’s the proven behavior that leads to increased sales? What knowledge enables that behavior?

 

JD Dillon

So what foundational behavior does the sales person have to know in order to execute on that behavior? And then, as a result of having connected that value chain, what do we have to do as learning and development? So in order to drive that chain, do I need to provide job aids? Do I need to provide a video? Interactive module? Or a mixture of somewhere in between that’s going to help me lead to that outcome, and then help me measure the impact of what we did, on what the business achieved, and then validate the impact and the ROI of learning that way.

 

David Wentworth

Yeah, and this is precisely why so many organizations have such a hard time with measurement, is exactly what you said. They will develop this really great program, put a lot of time, money, and energy into developing the program, they roll it out, people take it, and then they stop and go, “Okay. Now let’s measure it.” Well, measure what? They’re starting to measure at the end without thinking about it at the beginning.

So they say, “Well, what can we measure now?” Well, I can tell you how many people took it, I can tell you what grades they got, and I can tell you whether they liked it or not, and while those are important pieces of information, on their own, they don’t really tell you much about what the learning is doing for the organization. So, tell me a little bit about Toyota.

 

JD Dillon

Sure. So Toyota’s a great example of work we’ve done that’s outcome based, and enabling an experience like that we’ve already talked about earlier, especially with a disconnected deskless workforce. So, we’re talking about a dealer network with 25,000 dealer representatives and a centralized training team trying to help them move the needle on sales. I’ll just speak for myself, I am not a car person, I don’t know a ton about cars, but when I bought my car, I knew everything about that car, right?

I did a ton of research, and walked into the dealership armed to have a conversation, or to get what I was trying to achieve. I think that’s the expectation we now have, is that customer are coming through the door, in every environment, armed with information so that it challenges your staff, and your sales team to be, not just up to speed with them, but to be to deliver the right information, the differentiators for your product versus another, that’s going to drive the sales decision, right? That was the story that wraps around this Toyota case study, and that’s leads to the bottom line increase you see there, is why are sales reps that engage in this form of learning, and this type of experience able to sell up to two more vehicles per month, than those who didn’t?

It’s because the targeted outcome base approach that Toyota took, in focusing on what is that that critical behavior, that need to know information, that’s really going to drive the sales result that we’re looking for, and that’s how they’re able to measure that they’re work, as a training team, is what heavily contributed to this type of a bump in the outcome for the organization.

 

David Wentworth

Right. Absolutely. So, the other big piece, and I think we’ve touched on this in one form or another as we’ve gone through, but it really comes down to the engagement piece, and that traditional LMS experience really hasn’t done a very good job of engaging employees in the learning. It goes back to that embedded learning, putting it in the workflow. Learning has always felt very separate from the job. It’s, “I’ve got my job. I do this work, and now I need to stop, and I need to go do this other thing, and then I can come back and do my job.”

And so, we asked organizations about whether or not they were ready to take action on certain things, and these are all items that are related to engagement in learning, how to make learning more engagement. What you see is the percentage of companies that say, in the blue, that we’re not at all prepared to do anything about this, and then the other end of the spectrum is the green, which are organizations that say, “We’re ready to take action on this.” So, at the bottom, what you see here is the increasing experiential learning, trying to get more of that 70, of the 70:20:10 going.

17.5% of organizations say, “This is where we’re going to go with this. We’re going do more of this.” Only 12% say that they’re not prepared at all, but as we move up that list, things like games and gamification, more than half of companies say, “It’s not even on our radar,” basically. Learner contributed content, another huge engagement piece. The idea that learners are sharing with one another, things that they already know, how tos, resources, big engagement piece. 40% of companies aren’t ready to touch that either.

So, there’s a lot of elements here that can contribute to engagement with learning, but not a lot of organizations are ready to do anything about that yet. And so, the other piece that we ask is, “Here’s some more engagement pieces, what are the top learning initiatives for achieving business goals?” Now, these are specific initiatives from learning that are designed to help achieve business goals. How important are these as initiatives? You can see it’s a combination of important in blue, and critical in green. You see things like redesigning the classroom experience, like JD and I talked earlier, the classroom’s not going away, but let’s rethink it, make it more engaging.

Trying to create those smaller content objects, so we can drop learning in when and where we need to. Looking at the old content to see if there’re new ways to deliver it, can we break it down, and rebuild it in new ways, so we can make it more engaging? Analytics is another piece that helps up figure out how well the learning is working, where can we tweak it? How could we change it to make it better? More informal, that 20 of the 70:20:10. How do we do more of that? Again, back to the experiential, let’s do more of that, and overall, obviously, the biggest thing is companies are looking to align a learning strategy with the business, but in terms of engagement, all these other items are clearly very important to organizations for achieving the business goals.

So, engagement in learning is very closely tied to actually achieving the business goals. So, just talk a little bit more about some of the engagement tactics that you put in.

 

JD Dillon

Sure. So, engagement is an interesting and challenging conversation, right? I think we fall short in a lot of ways with engagement, because we attempt to over simplify, and think that we can find one answer, one formula that’s going to work for everyone in our organization. The reality is what engages one person, doesn’t engage another, both in workplace and at home. But, even within that, I think there’s some consistencies, or some principals to be identified that can help us overcome the inherently challenging question that is how to engage people in today’s workplace?

Here are four key ideas on screen, I think the first, and the clear differentiator is value. I know there are a lot of people out there who have been disappointed by the way they’ve been supported at work, over the course of their life. So, maybe it’s no one here’s fault, of course, because we’re all doing good things, but maybe somewhere in a past job, some type of training initiative, or something about their manager disappointing them, so they didn’t get the value when they logged into an LMS, or they completed a training course, and that’s hurt their overall opinion of what future offerings may be like.

So, I think first, it’s focusing on making sure that every time we ask for someone’s time and attention, when they’re going to log in to complete a course, or they’re going to complete a micro learning module, or watch a video, that there is a clear value to them, as an individual, and what they’re trying to accomplish in their job. And for me, it’s going beyond personalization of learning, and targeting effectively to the right people. It’s about driving adaptive capability, so leveraging, again, an approach, and technology that can use data to adapt to what this person needs right now, in order to achieve what they’re trying to do on the job. So, it’s going a step further to drive value, number one.

Number two is back to that simplicity consideration, making sure that it’s a familiar experience that doesn’t put undue burden on the individual, to get to what you’re trying to do to support them. I once did a, how many clicks does test, in a former job, and in one of our systems, it took 7 clicks to play a video, from log in to getting to the video, seven clicks. How many clicks does it take you to place a video on YouTube? Not seven is the answer. So, how do you make sure that you’re using familiar, comfortable designs, so that people don’t get arbitrarily frustrated, or have, like you said, technology problems that’s going to stop them from getting the experience you’re trying to provide.

Next layer down is motivation. Again, another thing that’s inherently personal. Motivation is different intrinsically and extrinsically for different people, but how do you leverage a variety of mechanics, and we at Axonify do a lot of work around gay mechanics, leveraging a mixture of mechanics so that you’re potentially hitting different individuals with different motivators that are going to be exciting to them. Some people care about leader boards, and getting points and achievement. Some people don’t. So, how do you balance to make sure you’re not trying to drive one idea, you’re providing multiple opportunities for people to engage and get motivated, so they come back for that continuous learning experience.

And finally, how do you make it fun? I think a lot of cases, we’ve potentially given up, or maybe we misdefine what fun can be in learning. So how do you leverage the concept of play? How do you make it an enjoyable experience? We do a lot of work with the idea of casual game play, coming in and playing a game, not just because it relates to the learning experience, or is made to make learning look prettier, but it’s something that’s actually fun, enjoyable, competitive, and feed that natural instinct a lot of people have, that makes games so fun, and dominate in the real world. So, finding ways to also make learning fun.

 

David Wentworth

Right, and that’s a big challenge too. It’s a tight rope sometimes for organizations, the line of games between fun and corny, so you have to be careful how well you play that out, but you’ve got an example of a company that puts some of this stuff into practice.

 

JD Dillon

Sure, our work with Bloomingdale’s is a great summation case study of a lot of the things we talked about today. If you look at the stat lines there, you’re going to see the outcomes of a lot of things we talked about in terms of continuous learning, the right form of delivery, focusing on an outcome based approach, that need to know information, and that’s what leads to the results you see on screen, in terms of our work with Bloomingdale’s and producing safety instances in their stores, and the annual savings number that you see there.

But, I think the other two points to highlight, ’cause those are great numbers, but let’s highlight, first of all, the fact that offering personal training, something that is a value add experience, not only resulted in these business outcomes, but associated stating that they have increased confidence on the job, because they feel supported, they see the value, they’re getting what they need in order to get better at their job, and as a result, they’re confidence was shown to increase when we asked them about how that related to the work they were seeing from a training perspective.

And then there’s the statement around gamification. 86% says that gamification increased their participation. People wanted to come back in a voluntary experience because we leveraged a variety of game mechanics, and a variety of motivators, and found what made it exciting and interesting for the individual. So I think the Bloomingdale’s case study shows a great example of measurable business results that could be correlated back to the work that their training team did, and their safety team did. But also, the idea that a variety of motivators and concepts can be used to, not just make it more interesting and engaging, but also show clear value so that people want to come back, and want to engage. You don’t have to chase people, or force people to want to learn and get better at their job.

 

David Wentworth

And so, the last point we wanted to walk through, and we touched a little bit on this, about measurement, and the fact that organization tend to be measuring the wrong things when it comes to learning. We’ll take a look a little deeper into why the traditional LMS is letting folks down in this area. So, talking about looking beyond learning outcomes, and looking at organizational outcomes to measure, we asked companies, “Are you using these outcomes to measure?” Let’s forget about this transactional tactical learning outcomes, let’s look at business outcomes.

“Are you using these to measure?” And if you look at the difference between the green line, which is high performers, and the blue line, which is low performers, it’s a very, very stark difference, huge gap between what the high performers are doing, and the low performers are doing. High performers are far more likely, in every single case, to use all of these different outcomes to measure the effectiveness of their learning. Things like, obviously, individual performance, right? Again, 38% of low performers say that they’re using individual performance to measure their learning.

How do you know if your learning is having any impact what so ever? Forget about on the business, what on your individual performers? How do you know what they’re doing if you’re not using individual performance as a measurement? Employee engagement, this is beyond what JD was just talking about, like being engaged in the learning, this is engagement in the organization, engagement in the job, right? That holy grail of business measurements, are your employees engaged? Learning has a lot to do with that. It isn’t the thing that’s going to win the engagement battle, but it certainly can lose it for you.

And so, it’s a critical piece to measure employee engagement, and see how well your learning is effecting that, all down the list. As you go to other things, things like revenue growth, and profitability, yes, obviously it is much more difficult to draw a direct line from learning program to revenue growth. There’s so many other variables in place, but if you take the approaches that we’re talking about, and you start at the beginning with, “What’s an outcome? What is a behavior change that should impact something like revenue growth? What is that behavior change, and how can we address that via learning?”

If you start with that thought process, you can at least draw a line of sight from one to the other knowing that there are other variables in place. So, actually there’s a good question that came in, and I want get to, JD, before I turn it back over to you on the measurement piece, and it is about measurement. The idea is, we talked earlier about a sales as an example, and sales is easy to measure and that’s calling and saying, and that’s obvious the easiest one because it’s so numbers driven, how do you measure other things like knowledge gains, application of a tool, or of an external client, which you can’t collect data? Pre or post test is the question.

There are many ways to do this. Obviously, yes. Pre and post test, what do you know now? What do you know then? That’s one way. We find that a lot of organizations rely on managerial observations, or supervisor observations, like are people doing things differently? Are they doing it the way that we asked them to do it? Simulations are great to, because not only is simulation is a learning tool to help you learn, but you’re also measuring in real time, right? Are you actually following the steps? Did you do the right things? Did you react the right way? Whatever it is. Anything that can allow somebody to put that knowledge into practice, and if it’s simulation they’re doing it in a way that doesn’t impact the business yet, because it’s in the simulation, or you put ’em out there, and they learn by doing, and again, monitoring and measuring those behaviors.

So, there’s a mix of things like observations and assessments. If there is a business metric involved, speed to productivity, right? It used to take 90 days for this type of role to be fully autonomous and ready to go. Can we get that down to 60 days? And, can we do that via learning? How do we get people up to speed faster? Multiple different ways of measuring that go beyond the easy numbers in sales. So with that, talking about some of the actionable data that you can use, JD maybe you can talk a little about this.

 

JD Dillon

Sure. So, I think where does this measurement conversation go? I know that I spent a lot of time as a corporate learning leader, exporting a lot of data, building a lot of spreadsheets, and sending a lot of spreadsheets to a lot of different senior level stakeholders in our organization to basically say, “We are, or not compliant. Your team did, or did not do this,” but ultimately, what did that get us from a validation of the effort the learning team put into the work, and then what the business was able to achieve.

I think the essential element of this conversation, the key point to takeaway is, how do we leverage data to activate the right intervention, the right behavior as close to the work as possible? And, the people who are as close to that work as possible is front line managers. So, in my opinion, front line managers are the most important people when it comes to workplace learning, because they’re the people who understand and control the day-to-day for the employees who are doing the work. So, who do we leverage the right types of data, and provide it in the right ways to managers to make an act on it, in new and meaningful ways?

I think, one, it takes us past the types of data that I think a lot of us are most familiar with, and a lot of us, I think, are still held accountable to, which is things like completions, hours of training provided per employee, test scores, these types of ideas, and breaking down into new types of data that can come out of people’s engagement with learning activities. So, things like how many people are engaging? So we talked earlier about those 90 plus percent number at the At Home stores, what does it mean when someone is engaged in a continuous learning experience? And, what does it mean for people who are not as engaged?

We talked about the Toyota sales people, how some folks were not engaged in those voluntary learning exercises, so making sure that managers are aware where the engagement’s taking place, so they can step in and provide the right motivation, and the right feedback, and value proposition so that people engage. Where is knowledge growth, and potentially, decline taking place? That’s where I think it’s essential to measure knowledge continuously, that’s where the idea of space repetition and reinforcement comes into play, because you may know it today, you may know it on the post test six months from now, but shortly thereafter, something happens. You don’t use that information very often, something changes and your knowledge waivers.

How do you consistently measure knowledge, so you can, again, report to managers where people are seeing growth, and where people are seeing decline? Then, how does that correlate to what people are doing? So, as you alluded to, how do you get that data about what’s actually happening in the real world to show that knowledge transfer? So in some cases, we work with organizations so that they’re literally out there observing behavior and reporting that data back into the system.

In some cases, it’s being pulled off of systems that can observe what a person is doing. There’s no one in a big rig truck watching the person drive a big rig truck, but there are a lot of safety concerns in a big rig truck, so how? Maybe you’re pulling the data off of the systems inside of the truck, if anyone here is in trucking, there’s a lot of data coming off the trucks nowadays, how do you use that data to show how that learning is transferring, and maybe influencing what people will get in the future from an adaptive learning perspective?

And then other things like achievement. I think a lot of times we talk about data in a coachable sense, how do we improve coaching and feedback? But, how do you also celebrate when people are achieving things? When people are learning new things, breaking new barriers, achieving new levels of performance that can be celebrated, recognized, and people can be pointed to as great models of performance by their managers? And also, things like compliance. It’s much better to be proactive about where you’re seeing compliance and regulatory considerations, as opposed to waiting for that report to come out of the training team once a quarter, once every six months, whatever that may be.

And then also, like I said, this leads to a much more productive employee-manager experience when they’re provided with this type of data on top of the KPI data they always use as part of their job, and helps close some of those gaps in coaching, right? People won’t naturally assume you’re not reaching your goal, you must need training. If we provide them with other points of data, that can help them understand, “Well, no you have the knowledge. I can see that. Your behavior, executions isn’t where it needs to be. What else is going on?” So they can have a more meaningful conversation than just throwing darts, which I think a lot of people are doing because they don’t have the right information to have a productive conversation.

 

David Wentworth

Precisely. So, those are the five key points that we wanted to walk through, and I’ll just put this up here. This is just a reiteration of our thinking, making sure that you’re thinking in terms of agile development, and continuous learning engagement so that learning can keep up with the organization, even performance management is … We’ve seen a dramatic shift recently to the more continuous performance conversation, as opposed to the annual performance review. And so, learning has to be able to keep up with that if we’re going to use performance as an indicator of learning, and as learning as an intervention for performance, you know, they work together.

They both need to happen in a continuous environment. So, embedding learning into the workflow so it doesn’t feel like some other thing, it feels like it is part of my job, this is what I do. Linking learning to business results, that’s obviously the key, that’s part of the measurement. Making that learning engaging, I put in here even compliance, I think compliance gets forgotten a lot, there’s no reason why compliance training needs to be as dry, and as boring as its historically been. It’s part of the learning family, and it’s typically the first thing people experience when they come into an organization, is some sort of compliance training. So why should that experience be so dismal?

Setting them up for what they fear as a lifetime of dismal learning yet to come. And the last piece is measure and analyze that actionable data, as JD was just talking about, making sure we do that. So, JD, we did have a couple questions come in, so I’m going to ask … the first one I had, and it speaks to measurement, it’s specific to you, one of the case studies, JD. Someone was asking about the measurement around confidence on the job, how did you collect that data? Are you familiar enough with that to know what the methodology was, and for your use and frequency?

 

JD Dillon

Sure, there’s two answers, generally speaking around work we do related to confidence in terms of people’s knowledge and ability to execute. So one, I think the number we were looking at was likely related to the survey data. So, we enable our customers, and our partners to provide surveys as part of the continuous experience so they can gage and get some anecdotal feedback about how they’re being supported. But a second piece that we play when is the idea of confidence based assessment. So every time we ask someone a question, as part of that continuous learning, and micro learning, and question based learning experiences is that we do, we don’t just ask, we answer and submit the correct answer.

We ask the people to gage their confidence in that particular answer. So one, it makes you think twice, just like every time you close a word document, and it says, “You’re sure?” Everyone of you thinks, “Did I save?” Or, “Do I want to do this?” Same type of an idea. So if you ask people, “Are you sure?” Or “How confident are you in this answer?” One, it makes you think twice, and two, it grabs a hold of that data point for us. So we can see how are people self assessing their confidence in their knowledge continuously, and how does that change over time? So, that’s something that we can provide as one of those actionable reporting data pieces so that we can show people where people maybe have great knowledge, but low confidence. So maybe they know, but they’re not going to execute, they’re second guessing themselves. Or, they may have high confidence, and low knowledge which means they think they know, but they really don’t, and those people-

 

David Wentworth

See, that’s probably the most critical piece of data.

 

JD Dillon

Yep, yep. So, we’re grabbing all of that data point, in addition to the other things we talked about when it comes to measurement, so that we can leverage it, and kind of grit it alongside knowledge and capabilities.

 

David Wentworth

Gotcha. Another pretty specific question for you, in the past Axonify had an engine to do follow up emails, but this is completely different. Does that still exist? Does that ring a bell to you?

 

JD Dillon

I’m not completely sure what follow up email means. I can speak to Axonify’s ability to trigger notifications, reminders, a variety of different types of messaging. So I think the soft answer is, yes. I don’t know exactly what that refers to, but we can notify and message people in a variety of ways.

 

David Wentworth

Okay. And then Christopher has brought this up, as we were talking about managers, he says that activating first line managers seems to be the most challenging. Even the best technology systems can’t accomplish this, unless learning is embedded into the organization’s culture. Any company stand out as the best examples of this?

For me, I unfortunately don’t have any companies at my fingertips to name, that I could say that are good examples of this, but you really hit on a very key element, is that technology is not going to fix anything. Technology has never really fixed anything. If you’ve got a bad process, or a poor culture, or whatever it is, technology only allows you to do it much worse, faster and on a larger scale. The idea is, you’re right, it has to be about the culture, and some organization’s first line managers, are they really managers? How well do they manage people? Or, had they just been promoted, and it’s the end of the line, and there was nowhere else to put them, but in a management position. So, we see that a lot.

So, in the context of this webinar, and the last few minutes we have left, I don’t think we can really address how you make that cultural shift, if it doesn’t already exists. But, one thing I see time, and time again when I do look at our case studies, across the board of any type of learning program that’s successful and wins awards through our awards program, one of the key elements is, there’s been this talk down buy-in, right?

Whether the program comes bottom up, or wherever it starts from, but the idea that executives get on board and get involved, and the CEO makes a video talking about how great this program is, or this initiative, and how important it is, and doing road trips and going in, and walking through and having events, getting people excited about it, when you talk about it, it seems like it’s kind of a dumb thing, right? “Let’s have the CEO tell everyone to do it.”

But in practice, when people do see that, “Listen, this is important, and this is how important it is,” it tends to trickle down, and it tends to have an effect. So, JD, I’ve looked through this next question, maybe you can, if you have any companies you can think of that have that type of a culture, or if you just want to talk about it more generally.

 

JD Dillon

Yeah, the one thing that I would add on to that, because I don’t have a great example that I can reference right now, but from my own background, I can also speak to the value of pushing that messaging up in the opposite direction. So often see in some organizations, where maybe you’ve got senior level stakeholder buy-in is huge, for anything you do in a company, but in some cases, depending on your culture, and politics at play, it may not reach all the way down to the frontline, something stops somewhere in middle management because someone has power, control, things like that.

So I’ve exercised tactics in a variety of ways where I find who are some of those early adopters, or who are those managers that are just great at their jobs, who are going to grab a hold of an opportunity, like using actionable data, and learning data in order to drive performance, and how do you use them as a great case study to push that information back up through the chain? Because one, if you’re able to celebrate a manager who’s using these ideas, and realizing real business results, people moving up through the hierarchy are going to be very excited about the business results side.

 

JD Dillon

And two, this manger is going to love being celebrated in a lot of cases, and then you’re able to turn around to the other managers, who maybe you’re experiencing with that cultural challenge, who aren’t quite as engaged, and be able to say, “Someone like you, who does your job, and is held accountable for the same things you are, is seeing these results,” and without actually saying it, you can kind of say, “Why aren’t you?” So you’re providing not just a great example, but also potentially, some tactics around, “How is this person doing the job?” To help maybe, start shifting some of that cultural issue.

 

David Wentworth

And the last question I have is pretty long, but I’m going to summarize it. I think I get the point here. It’s going back to your Toyota case study, and the question was how can you attribute the addition two car sales a month to Axonify, with all the other variables that might be in play?

And as I mentioned, in this type of measure, of course, there are always variables in play, but I think in this specific instance, JD’s referring to the fact that they were measuring between the group that had completed the training versus the group that had not, and the ones that had finished the training were the ones that were doing two cars more, compared to those that did not do the training. Is that correct, JD?

 

JD Dillon

Yes, and you have to make sure in any measurement, an example like that, you have to mitigate a variety of other factors. I think it’s clear that there are a variety of things that impact performance, it’s never just training. But, one, when you have a sample like that case, and two, you make sure that you’re not comparing your top performing stores or sales reps versus your bottom performing [inaudible 01:00:47] and things like that. So you make sure to make sure you have the best sample size as possible, and that you’re making a clean comparison, like we did in that case study.

 

David Wentworth

And so, to follow up on that, [inaudible 01:01:00] is asking why? How is it the LMS responsible for this. It’s the content that drives that. What’s the role of the LMS? And, if the same content was rolled out in some other LMS, the result would be the same.

That’s not necessarily true. It’s a question of what is that experience like? The LMS is the environment in which the content lives, and if the LMS is limited by how it displays, how it delivers, it doesn’t allow you to deliver anything but a bulk SCORM course, then you’re unable to have that type of experience. So the LMS plays a huge role. It’s not simply the content. Now, that’s not saying Axonify is the only LMS that would have been able to do that. I’m just saying that not all LMSs are created equal, and not are all able to deliver the same types of experiences, and not all are able to deliver the same type of measurement and analytics that would allow you to know that those results are even being achieved.

So, yes, it does rely heavily on the content, but it also relies heavily on the experience learners are having, which comes back to the is it relevant? Is it contextual? Can I take it where and when I want? Does it relate to me? And all of that. So, I do think that’s an important piece of it. And then, I guess we’re over the time, and just to let William know that within the next 48 hours or so, an email will go out with a link to the recording, as well as a PDF of the slides. So, with that, I want to thank you, JD, so much for your insights, and sharing your examples and thank everybody for joining us, and asking the questions, and being part of the conversation.

 

JD Dillon

Thank you very much, David.

 

David Wentworth

All right. Thanks everybody. We’ll talk to you next time.