SAVE | EMAIL | PRINT | MOST POPULAR |  RSS | | REPRINT
|
Demonstrating Value: How Do They Do That?
December 24, 2008
We must design, develop, and evaluate training programs in a way that means something to the client and demonstrates success—leading to inherent trust and a reputation for delivering performance and bottom-line impact. (From Training's 2009 Preview)
By Allison A.S. Wimms
Some departments have mastered the art of demonstrating their value. It's as though they have an intrinsic reputation that oozes a sense of worth-every-penny. Further, they sometimes fail to meet expectations—yet their reputation for providing value and "bottom-line impact" remains intact. How do they do that?
Think about this. When employees are asked to identify departments that provide the most value and positive impact to their organization, the training department is rarely the first that comes to mind. That's reality. Here is a real-life example experienced with a former employer:
During a meeting, someone from the IS department touted that the new screen he was developing within existing software would save the company about $1.2 million. Effectively, he was saying $1.2 million would be added to the bottom line. Everyone nodded with approval, and no one questioned the presenter—it was as though participants inherently trusted that IS would deliver on that bottom-line estimation.
Except me. Here's what I was thinking:
1. How does a screen deliver $1.2 million?
2. Can I have that same screen installed on my home computer?
3. Does anyone realize the screen won't deliver a penny if our training department doesn’t teach employees how to use it correctly?
The third point, although a bit sarcastic, is true. Yet no one in the room spun around to see what a representative from the training department thought.
Of course, software may provide hard or soft dollar savings, but that’s not the point. The point is this: Most organizations do not easily see that training is required to make most things happen—employees need to learn how to do something before they actually can do it. Yet other departments receive the kudos for performance, changes, and bottom-line impact.
We need a shift in thinking. We as an industry are not about our deliverables—job aids, classes, material—we are about improving performance and helping organizations to function, but that's not the way most describe our work. We need a new reputation.
However, before we can enjoy such a reputation, we first must earn trust. This is where we could learn something from IS. To start, we must play our game right: We must develop programs that truly meet clients’ needs; ensure such programs are effectively designed; and evaluate and report findings in a way that means something to the client and demonstrates success. Because the reality is this: Success and value are in the eyes of our clients, and we need to see things the way they do—we also need to show them where to look.
In most cases what a client desires is twofold: 1) a program or solution that addresses a need, and 2) proof the need was met. This is where we need direction.
When it comes to demonstrating success, we have Kirkpatrick's Four Levels of Evaluation; and when it comes to overall design, we have the ADDIE (Analysis, Design, Development, Implement, Evaluation) model. We must blend the two more effectively if we are going to focus on success and value.
As noted in "Crossing the Great Divide: Program Development and the Four Levels" (2007), Jim Kirkpatrick and I describe how we must incorporate evaluation throughout each phase of the ADDIE model, in order to create programs that mean something to the client. We state that evaluation is not a standalone phase to be thought of as "last." It must be an inherent part of all aspects of the instructional design process—it cannot be an afterthought—otherwise, we may miss something critical that could put the design of a solution on the wrong track.
Think in terms of an IS department, which would never wait until a new screen has "gone live" to see if it meets expectations. In the same manner, we must not wait until implementation to solicit feedback. We must focus on how to measure success long before the program is developed. To do this, we must blend evaluation throughout each ADDIE phase. We cannot wait until after implementation.
Here are just a few things we must consider during the design process:
Analysis
Have I accurately diagnosed the client's need? Do I know how the client will measure training "success?" What evidence will I need to prove this success? In other words, we should be thinking about evaluation (i.e., how to gather evidence of success) from the very beginning.
Design
When it comes to measuring program success, how will I do so? Will the evaluation I design measure what the client is seeking (i.e., proof of success)? In other words, what level of evaluation will be most valuable to the client? Have I documented the proposed evaluation approach in the Design Document to obtain client buy-in?
NOTE: IS departments are great at design. They write Scopes of Work, Critical Success Factors, Client Responsibilities, and they obtain signatures of approval. They say, "Mr. Client, your signature indicates your agreement with this Scope of Work. If we deliver xyz solution, you will deem our work successful."
Development
Evaluation and "proof of success" are more involved in our world. We measure in terms of reaction, learning, application, and results, and this takes skill. It requires the right tools, right questions, right timing, etc. Do the evaluation questions match the program content? Is the evaluation easy to understand? Is the overall evaluation approach (i.e., materials, process, how-to instructions) included in the pilot program?
NOTE: Compare this to "Acceptance Testing" in the IS world. The client approves the deliverable long before cutover.
Implement/Evaluate
In the IS world, implementation typically indicates the end of the project—not so in the training world. Have I sought feedback from the facilitator (i.e., reactions as witnessed during classroom training)? Do my results provide the evidence the client is seeking? What needs to change, if anything, to improve the overall program and/or evaluation approach?
Post-Implementation
Have I developed a report that will mean something to the client? Does the report show how the program improved performance or helped the organization to function? If not, what are my next steps? If so, how will I ensure the clients "gets it," that it was not a "program" we delivered; rather, we delivered performance and results?
It's about a change in thinking and improving our technique. We need to think constantly in terms of success and how we will prove such success. We need to approach ADDIE in a different way. We must have evaluation (i.e., proof of success, value, and impact) on our minds throughout the entire process.
We must design, develop, and evaluate programs in a way that means something to the client and demonstrates success—leading to inherent trust and a reputation for delivering performance and bottom-line impact. If we keep this in mind and act accordingly, we, too, will have mastered the art of demonstrating our value. That's reality.
Allison A.S. Wimms is a senior training and development specialist at Johns Hopkins HealthCare LLC in Maryland. She is co-author of "Crossing the Great
Divide: Program Development and the Four Levels," written with James D. Kirkpatrick, and she is working on an instructional designer's handbook specific to evaluation. She will speak on "Program Development and Evaluation: What are we missing?" at Training 2009.
|
SAVE | EMAIL | PRINT | MOST POPULAR | RSS |
|
|
| Back to Training Index |
|
|
