Don’t rubber stamp regulatory courses — establish KPI for training success

When it comes to training programs, most financial services organizations focus too singularly on checking off course completions to determine the success of the program. For regulators, that’s a potential red flag. As far as they’re concerned, finishing the course achieves the bare minimum.

The goal of a compliance training program should be ensuring employees have the tools, knowledge, and skillset to succeed. To achieve these goals, financial services organizations need to establish key performance indicators (KPI) to measure the success of their training programs.

KPIs should be set up to measure how quickly and easily employees complete their training, along with how they utilize their training during daily activities to mitigate incidents of noncompliance and achieve better performance.

Yes, start with monitoring training completions. But timeliness matters more.

The biggest indicator of relative success for training programs most financial services institutions prioritize are course completions. And it is a number to track. This metric shows that employees are taking the time to complete their training, which should mean they are prepared to handle a variety of responsibilities related to their job and develop the skills to progress in their career.

Yet for regulators, it’s typically not enough. In addition to course completions, these governing bodies want to see if learners completed their training on time. Whether it’s new regulatory requirements, changing market conditions, or internal policy changes, completion dates are set so that employees are ready to go before new requirements go into effect. If there are some employees who don’t complete their course-completion acknowledgements in a timely manner, the company could be at risk for noncompliant behavior.

How many do-overs? Attempts-per-completion should also be tracked.

Attempts-per-completion is a compliance training metric that provides more insight into how learners succeed with their training. When organizations test comprehension, they should look at how many attempts it took each employee to pass their assessments. If most employees are passing on the first attempt, that should indicate the content is easy to understand and is likely doing its job in thoroughly educating staff.

If, on average, staff require multiple attempts to pass training assessments, testing administrators need to figure out why. They should first check how long employees are reviewing the materials before testing. If employees aren’t spending an adequate amount of time studying or at least reviewing the materials, they won’t fully understand what is being covered and won’t likely pass. Organizations may need to change the incentives or mandatory requirements attached to preparation time.

If employees are investing time in reviewing materials but still aren’t passing in a reasonable number of attempts, training administrators might need to update, reformat, or otherwise adjust the materials to make sure the learning objectives are clear before and during training.

Training administrators should also routinely survey learners to see where they can improve compliance courses. If the learners feel the material isn’t relevant or engaging, that could affect how well they retain knowledge and should be addressed when courses are updated.

Don’t ignore incident mitigation and performance improvement.

Determining the success of organizational compliance training programs doesn’t just stop after employees complete their training. Organizations need to set and monitor success metrics for the parts of an employee’s responsibilities that are impacted by the training they completed. This training should either help eliminate incidents or improve performance.

With that in mind, organizations should set up KPIs, such as number of incidents/complaints, customer response time, application approval time, or anything else that measures performance for specific activities covered in compliance training courses. Training administrators and department heads need to work together to set up these KPIs and determine performance benchmarks to see if the training program is doing its job. If after training, the employees are close to or outperforming their set goals, you know the training provided is having a positive effect.

If, however, the number of incidents increase or the desired performance improvements are not being achieved, something needs to change.

Training administrators and department managers need to reassess the training materials and make changes to help achieve desired outcomes. There are several variables for them to look at including: changing environmental factors in the workplace; disconnects between learning objectives and how employees are supposed to achieve results; lack of real-world scenarios in training materials and more.

In addition to looking at these factors, organizations should also review post-training survey results to see where learners felt they could have used more support to help improve future compliance training. All these metrics will not only help the organization improve but will also show regulators everything the organization is doing to create successful training programs that help build a culture of compliance.

Christopher Boersma, CRCM, CAMS, CISA, CC, is Product Manager at BAI.

Explore the BAI Learning Manager.