Stop Checking the Box

stop checking the box

I recently attended an industry conference and was enjoying some spirits with my peers when the topic of measuring training was put on the table.  I swear it was only minutes before everyone was sharing awesome percentage of completion numbers and giving each other high fives, even talking about winning awards.  Everyone seemed pleased by how many employees had checked the box that they completed or participated in a program.

Then I sucked all the life out of the room with one question – but was the training effective?

There was no response; just a large chorus of crickets.  Seriously, no one was curious to take a deeper look and see if the training intervention delivered the desired outcome, addressing why the training was needed to begin with.  Even more frightening, many present these “percentage of completion” results to their organizational leadership and CEOs to demonstrate success or maybe to justify their existence.  We wonder why Talent and Learning leaders struggle to get the proverbial seat at the table.  We have even gone as far as replacing ROI with ROE (Return on Expectations).  As an industry, it sounds like we are making an excuse for how we measure success.

I’m pretty confident my CEO could care less about my expectations for the training versus a favorable return on the dollars he gave me to spend.  Honestly, I’m too much of a chicken to even consider pitching ROE to find out the response.  I don’t develop or roll out any learning programs unless it solves a business problem.  So the question I am asked is very straightforward: Did it work?

So back to the conference, drinks and the stimulating conversation which now has evolved to discussing the best metrics model for measuring true effectiveness of training interventions.  Most people would be discussing the British Open or the latest scandals in Washington, D.C.,  but not us.  We were discussing all levels of Kirkpatrick, as well as some really fancy software programs that generate charts, graphs, spreadsheets and more.  Very cool, right?  It depends on what is being measured and what should be measured.

New jobs have been created in the learning field simply to measure the effectiveness of the training interventions.  It’s almost like we implemented our own stimulus program for Learning & Development. I’m not saying that measurement is not important.  I’m saying let’s cut to the chase and measure the one thing that is important to everyone at the leadership level in the organization – business impact (did it work?).  Understand the business problem, the role the learning solution plays with solving it and what success looks like at the end.

As an example, if consultants do not have the skills or competencies to be competitive resulting in a loss of new business and lower billable rates, we know the problem as well as the desired outcome to work towards.  Win more business and increase the average billable rate.  If you can demonstrate this – then it worked.  Very simple, yes?  Then why do we make it so damn hard?  Probably because it’s much easier to report on total percentage of completion and course evaluations versus the real business impact.

There are countless articles on why Talent and Learning leaders need to be more strategic, partner with the CEO and help execute the corporate strategy.  Good gracious, I think I have read the majority of them.  However, for this to happen we need to cut through knee deep crap and start measuring business impact.  Stop looking at how many people checked the box by completing or participating in a program.  Start looking at how it will help solve a business problem and support the larger corporate strategy.  Put yourself in the position to answer yes when your CEO asks the question, did it work?

FOT Background Check

Dan Carusi
As current Vice President & Chief Learning Officer for Deltek, Dan Carusi doesn’t know which he likes more – teaching or learning.  A father of two, Scout leader and coach, Dan is often learning as much from the kids as they are learning from him (or possibly more).  With more than 20 years of experience, Dan is responsible for overseeing Deltek University and the Talent & Learning organization, where he oversees all aspects of talent management, curriculum development, operations & delivery, global employee & customer education and Human Capital consulting – often using what he learns from the kids as tools for teaching, with the end goal of making life-long learners out of everyone.  Teach Dan something at “email”, LinkedIn or @DanielCarusi.

3 Comments

  1. broc.edwards says:

    Dan, thank you for saying this. It’s amazing how often we confuse what is easy to measure with what is important to measure. Training for the sake of training is stupid. There is one – and only one – reason for training: to increase performance. That’s it.

    Your question “did it work?” is a great question and really gets at: “Did the learner increase their knowledge or skills in a way that caused them to perform better?” That’s a very different question than “Did they pass the test?” or “Did they show up and pretend to care?” or “Were they engaged enough to laugh at the trainer’s jokes?”

    Reply
  2. David Berke says:

    I agree: “Did it work?” is a great question. But it’s not an ROI question. I’m probably being too much of a stickler here. But your CEO might too. Here’s why: typically training and development focuses on a change in knowledge and/or skill. You can prove whether someone knows more about something or can do something they couldn’t do before. But that’s different than claiming a direct causal relationship. There can be many reasons for loss of business and lower billable rates – certainly having consultants who do not have the skills or competencies to be competitive (whatever that means) could be among them. So could outdated products, poor fulfillment, overly complex support processes, etc. etc.

    My point: Absolutely ask and assess whether it worked. Just don’t claim what you can’t prove. Because one of these days someone – like the CEO – is going to call you on it.

    Reply
  3. Joel Kimball says:

    Yeah buddy. It’s relatively easier in the mfg world – “can you now run the machine at rate?” If yes – SUCCESS! If not – what do we need to do so that you CAN run the machine at rate. Other things are trickier, of course – how do we ensure that maintenance actually knows how to fix the machine when it goes down? Cause they know today, since we just trained them…but the machine won’t go down for a year cause it’s brand new, doncha know. So…single point lessons? Video tape the training and upload it to the plant’s training site? Notes on 3×5 cards?

    Maybe not as sexy as ROI, ROE, PDQ or whatever – although I find it much more stimulating, cause – real life!

    Great topic well – presented. Thanks, Dan and FoT’istas!

    Reply

Leave a Comment