well color me tickled pink. we jumped headfirst today into a brave new world, what we call participatory TV, with our inaugeral episode of the Truth in e-Learning Web TV series.
Episode One was called “The Truth About e-Learning ROI”. we had a good audience of e-Learning, HR and training & development professionals from companies large and small across many industries such as 3M, Lowes, Pearson, McKee Foods, HIP, Rollins, Reynolds & Reynolds, AirTran, Nortel, Johnson & Johnson, Nestle, Sacramento Municipal Utility District, YMCA, FM Global, Target, KLA Tencor, Dorman Products, SRP, Christus Health, The Solid Waste Association of America, and more.
the intensity of live production is a bit daunting. if you think about the million things that can go wrong, whether your fault, someone else’s fault, an act of god, an end user’s internal network and computer set-up, you can imagine why my stress levels rise until the show finally starts and the sailing gets smoothe.
remember, if someone complains that they couldn’t see your production and you ask them if their computer is turned on (a pre-requisite for watching WebTV), it’s still your fault. fortunately, things went relatively well. and everything that didn’t we learned from so we can improve in upcoming episodes. an example was the video frame rate. for this episode it was more like dancing at a club with strobe lights rather than smooth TV-like video. that will be fixed.
so what did we talk about today in Episode One, The Truth About e-Learning ROI?
the e-learning industry has become much more about technology and technical solutions (the means) than teaching, learning and job performance improvement (the end). technology often becomes the driving force behind e-learning initiatives. while technology is important, we need to keep in mind that it is a means to get to an end. too often, e-learning initiatives get lost in the means and the end is not achieved. as an industry, we’ve underperformed in this respect. but it’s not too late! and the Truth in e-Learning WebTV series aims to explore what we’ve done well, where we’ve gone wrong, and how we can improve going forward.
show format: once per month on a Wednesday @ noon ET we’ll get together to talk through a key concept in e-learning with a special guest or two.
we also have a viewer participation session about 45 minutes into the show. TeL is an interactive environment for all of us to get together, and not a one way monologue. it’s participatory TV. we’ll also have special segments from on the road called Truth on the Road. musical guests? coming soon…
our host: david guralnick has been in the corporate e-learning world for about 17 years, mainly working with Fortune 500 companies, but also with some hospitals and non-profits. his work has focused on the design of e-learning, simulations, performance support tools and specialized authoring tools. he is the president of Kaleidoscope Learning, an adjunct professor at Columbia University Teachers College, chair of ASTD NY Chapter e-Learning Special Interest Group, and chair of the International Conference in e-Learning.
in this episode, The Truth About e-Learning
ROI, we focused on four questions:
- what do we mean by ROI
- how important is ROI
- how do you calculate ROI
- what’s wrong with the way ROI is generally calculated in e-Learning today?
our guest: to help us answer these questions, we were joined by special guest Hal Christensen, partner with Christensen/Roberts Solutions, and a performance improvement expert. CRS received the 2005 Performance Center Design and EPS Award for the Salesforce.com interactive coach.
before we get started, let’s make sure we’re all on the same page with a definition of ROI:
what you’re spending and what you’re getting back from your e-learning investment.
ROI is a terrific concept. it’s great that training departments are looking at the return they are getting on their investment and how it affects the overall corporate bottom line and asking questions such as do we have an impact on helping to solve the actual problems that the organization has? impact, however, should not be measured by “we produced a certain number of courses this year” or “we got better test scores” or “we were able to enroll a large number of people”.
what we should be focused on is what impact you have on the bottom line. what is the change in performance on the job? what is the change in productivity and proficiency of people doing the job? or intangibles such as reducing turnover rates and other things that can be measured. training departments are often not given the time or the budget to measure such things, which is a problem.
investment: what is the cost of the investment being made? many corporations look at the cost to design, develop and deliver the training. what we need to look at is the cost for each employee to stop working and learn (either online or in the classroom). that is a cost to the organization. the total all-in cost is much larger than most people realize and should be calculated into the return on investment. you should also factor in the opportunity cost of such things as a sales representative in class (online or in the classroom) and not selling.
if organizations start to factor these things into the ROI equation, they will have a much clearer sense of how they should deliver training and the value of their investment.
as an aside, the idea of training standing on it’s own as a compartmentalized area of the company should be passe because it is an outdated idea. the purpose of training is to improve job performance. it’s to help the business. the prescription to improve job performance might be training, it might be performance support. either way, training should be more integrated into the business. maybe the training department should be renamed the performance improvement group.
several factors are involved in determining cost for ROI calculations. an example of cost in the training versus e-learning equation is travel time and cost. but travel is just one of many costs and has probably received too much attention. yes, e-learning helps reduce training travel costs, but does it help reduce organizational cost any better than its classroom equivalent?
travel is just one of many costs that should be factored into the ROI equation. e-learning can reduce costs 40-60% over traditional training due to the lack of travel requirements. and the flexibility of e-learning that allows you to chunk content and do small segments at a time instead of all at once (like when you suddenly have 100 people all flown in from different parts of the country). people can spend their time more efficiently. and that’s all good.
but even if you reduce the size of the training modules (chunk them) so people can train while on a coffee or cigarrette break, on the toilet (ok, if we’re talking about e-learning maybe they have a laptop with wireless connectivity), or while driving to work, you may still have a significant amount of time that employees are in the training environment and not on the job.
so e-learning can reduce time and cost relative to classroom training, but e-learning still takes employees, such as sales people, away from the job. what is the cost of 100 minutes per person that sales reps are spending in the learning environment. training is an investment, but you have to add opportunity cost into the total cost side of the equation. with e-learning as a form of training, you are still asking people to take time away from their job. sales groups typically resist training because it is time away from selling. part of their compensation package is, after all, commission based on sales. anything that stands between them and their commission is not their friend.
return: with the return, we should measure something other than the total number of people registered for training programs, the “butts in seats” syndrome. you are doing a training program because there is a particular problem within the organization and the company wants to find a way to improve performance and productivity. as an example, a call center might want to see how they can reduce the number of calls coming in, the number of abandoned calls, the amount of time agents are spending on each phone call. that is the problem to look at and solve. recommended solutions should be designed to solve those problems.
with training we tend to focus on cost. and we just compare e-learning to other training methods when thinking about ROI. we really should look at finding what solution might be the best to solve the problem, rather than training as the only solution. are there solutions outside “training” that might solve the problem better than training? we are too locked into a mindset that nobody can do a job until they are trained first and there is no way to improve productivity and how a department functions if you don’t first train people. we tend to aim at the performer rather than the performance. is there a way to directly affect the performance rather than going through the training? if so, you can turn your ROI equation upside down.
training is absolutely necessary, but we need to look at the effectiveness of the training not by whether it cut training time, but by whether it improved job performance. and if so, are there other ways to improve job performance more effectively than training (classroom and e-learning). if training is put online and it was not improving performance as a classroom course, it will not suddenly make anyone do anything better. it will cost less to deliver, but it won’t be more effective.
technology, like training, can make an impact on improving job performance. but technology in and of itself is not sufficient either. the art of teaching and learning, laid on top of that technology, is what improves job performance.
Hal, in his work, has focused on improving performance by delivering training at the point in time where employees are trying to do something. for example, when call center reps are using a software tool to do their job, they do not necessarily need training on how to use that tool. not all learning needs to be done before someone does the job. can learning be a by-product of doing the work? when someone does the work, can we provide training wheels to make sure the learning comes as they are pedalling.
again, in the call center example, imagine if you could reduce a 30 day training program for new representatives on how to use the software tools used in the office to 4 days and not lose on important performance metrics such as serving customers better and giving more accurate answers. in a project Hal worked on, he designed a program where the call center reps didn’t have as much up-front training, but as they used the tool, they internalized the information so after a few weeks they knew far more than a person who spent 30 days in training because they learned as a by-product of using the tool. think about how many man-days of training they reduced by eliminating 25 days of training per person.
often, the solution is not to present classroom knowledge online (recreating the classroom course online), but to provide employees with the feedback they need to tell them whether they are doing things correctly or not as they are doing it. rather than re-training people, you can provide them with feedback.
as an example, Hal worked on a project for a hospital to help make sure that physicians properly code procedures they do. if they code incorrectly, they either underbill or overbill customers. neither is a desired outcome.
one solution would have been a day-long course for the physicians. instead, Hal built an auditing tool that produced feedback while the physicians were actually doing the coding, like training wheels. a physican could take the information he entered into the coding tool and put it into the auditing tool, and the tool would produce feedback that would give him specific results on what he was doing wrong. since each doctor was making a few mistakes, but not many, they didn’t need a whole training course. the auditing tool gave them exactly what they needed. they could see what they did wrong and they could correct their behavior and they didn’t need a training solution to achieve the desired outcome. everyone was happy with the outcome and the ROI was better than what could have been achieved with a training program.
Hal recently won EPSScentral’s Performance Centered Design award for a project he did for Salesforce.com, a Web-based Customer Relationship Management and Salesforce Automation toolkit used by salespeople to add new prospects and track them through the sales cycle. with business productivity applications, user adoption is difficult – to get sales reps to start to use the tool once the decision has been made by the company to use the tool.
the tool will be a failure if sales people don’t use it. even though the software is intuitive, it does take sales reps a while to get up and running on it. training takes people away from selling.
Hal’s challenge: is there a way to enable the people to do the work immediately with Salesforce without having to go to a training program? can we augment the interface of Salesforce.com so that the interface goes from a neutral “i dare you to figure out what data to put into me” to a proactive coach that guides users through exactly what they need to do and ensures that they do it correctly and guides them through best practices and business rules. the solution should help them perform their tasks better?
the program Hal developed, called QuickSuccess, includes a toolbar that is layed on top of the SalesForce.com application. the QuickSuccess application is a layer on top of the actual Salesforce.com application.
the QuickTask toolbar enables users to go through the key tasks they need to perform to get started.
the key part of the QuickSuccess application is step by step guidance to a particular process such as converting leads to opportunities. in the center of the page is a red box that highlights the particular field a user should be working on. on the left side is a best practices box to help people understand what they are doing and why they should be doing it.
the goal is not just to provide step-by-step instructions but to also provide best practices and the best way to perform a task to reduce the opportunities for making errors or to get users in trouble at a later point in using the application.
in Salesforce.com, when you are on a page, there are certain required fields. so the QuickSuccess application makes sure users fill out the required fields and do it in the most efficient way (for example, encouraging users to use the drop down calendar to enter a date instead of manually entering text, which leads to consistency errors in formatting – july 20, 2006 versus 07/20/06).
QS also explains why users should be doing something. at the same time, all other non-required fields on the screen are shut off. the training wheels are on. eventually, the training wheels go to “light” mode once a user has internalized what needs to be done. the light version provides support for fields (context sensitive help). notice the question marks in the following image:
there are also 3 productivity tools for helping users to create reports and search for information. QS encourage sales people to USE the data without fumbling around. adoption increases when users realize that they can actually use the data.
in practice, QS has eliminated the need for training. that is a great cost saving. but the key cost that has been eliminated is salespeople NOT selling because they are training. training is generally innefficient because people forget 70 percent of what they learned. so when they actually start using a tool they were trained on, they end up fumbling through the tool (trial and error) and need to typically ask other people for help, etc. obviously not as much as if they had never had any training, but there is still a fair amount of feeling their way through a relatively unfamiliar task. all of this translates to opportunity cost – sales reps being able to sell or not.
organizations don’t usually have an appropriate way to measure organizational improvement. we all need to identify the metrics we should measure ourselves by and tie training and performance improvement to those metrics (e.g. reduce the amount of time it takes a call center rep to help a customer find an answer to their question). training is typically evaluated on whether it was completed, or whether people enjoyed it. but it is often not evaluated on whether it helped people do their job better. everything should be tied to a performance goal.
Audience Question: Can you speak specifically about some of the ways and methods of measuring improved performance and productivity and performance in sales. Sales numbers is one metric, but that might not be attributable directly to the training.
Hal: companies need to measure the effectiveness of sales people. the problem is how do you identify something you have done as a trainer that impacts effectiveness. training is indirect, it is trying to change the behavior of someone. if you begin to focus on the actual environment, you can make ties between the changes you have made, like the speed people perform their tasks, you can measure the savings. it’s hard to link training to more product sold because that could be tied to market conditions, but it can be tied to increased speed making the product.
trainers should focus on what the problem is and the metrics the organization measures and wants to improve, and then build a program to make sure the training has a positive impact on those metrics that are measured. most training starts with the premise that we have to teach someone something, rather than what the impact is at the end. we’re not starting with the problem. you need to start by looking at the metrics that tell you that something isn’t right and needs to be improved.
look at the ten most common tasks salespeople perform and look at how they perform them before and after the training. build a program that will affect the metrics. look at the problem first. a training department should look at what kind of performance they want to improve and then build a program that helps solve the problem. science helps identify the problem. then art and science are the solution to improve things.
look for metrics or evidence of a problem that would lead to the need for a training program. define what you want to correct and then build a program that addresses the problem. a training program is not always the only solution. the problem is thinking that if we give people education the problem will be solved. what is not right and what can we do to make it right? training may be a part of the solution.
are we doing better after the initiative than before the initiative. if you are looking for funding for a training program, you will be more successful if you can show what the problem is and your recommended prescription to fix it and what should be measured to show success.
To find out more about Truth in e-Learning, and the schedule of future live Web TV episodes, visit the Truth in e-Learning Web site.
e-Learning Designers and Developers 
This show brings together e-learning experts with strong points of view to discuss key e-learning issues in an engaging, entertaining, and interactive format.
I was asked to give a brief recap of the inaugural episode of 
