Cybersecurity managers and CISOs often ask why they should run regular phishing exercises and not just having their users following a yearly computer-based training (CBT), as they do for most of the other topics.
Beside the commercial sales pitch a security firm could deliver (full disclosure, besides my research activities, I also work as a security consultant), what does sciences says about that?
Not so much, unfortunately.
One of the first attempt to answer this question was made by Aaron Ferguson in 2005. Ferguson is an NSA visiting Professor at the famous West Point US Military Academy. He sent phishing emails to 512 West Point cadets after they received 4 hours of computer security instructions. The phishing email, called the West Point Carronade by Ferguson, tricked 80% of the cadets. While the scenario was quite targeted and the context highly favourable to make this scenario successful, it was quite a success, despite the training.
In 2010, Davinson & Sillence trained users using “Phishing Phil”, an online game about email phishing. Their goal was to evaluate the impact of the level of risk communicated on the users’ behaviour. By their own words “There was no effect of the training programme on secure behaviour in general”. Unfortunately, they did not measure the actual behaviour of the users before and after the training.
In 2013, Jansson & von Solms conducted a series of phishing exercises on an academic institution in South-Africa. He ran four scenarios in parallel on seven different groups, in two waves. The subjects who clicked during the first waves received and embedded training (meaning, the link they clicked or the attachment they opened displayed a warning about their insecure behaviour) and a warning email. They also had the opportunity to follow online training by clicking on a link displayed in the warning page. The next week, the same users received either the same email, either a different one. There was 42.63% less click during the second wave than the first. This seems to indicate that simple feedback and a short training right after having clicked (embedded training) can reduce phishing susceptibility.
In 2019, Gordon et al. conducted a series of phishing exercises on 5416 employees of a US healthcare institution. After the 15th exercise, they identified the “offenders” (those who clicked 5 times and more on the previous exercises). They provided computer-based training to these offenders a continued to measure their results to the 5 next exercises. While the phishing exercises reduced the click ratio for both the offender and the non-offender groups, the CBT provided to the offenders did not decrease the click rates than the non-offenders (low-risk) group.
That is not a lot of data to form an opinion. Still, it seems that a simple message embedded in the message received by people clicking in a phishing exercise’s link is more efficient than a CBT to reduce phishing susceptibility.
References:
- Davinson, N., & Sillence, E. (2010). It won’t happen to me: Promoting secure behaviour among internet users. Computers in Human Behavior, 26(6), 1739–1747. Scopus. https://doi.org/10.1016/j.chb.2010.06.023
- Ferguson, A. J. (n.d.). Fostering E-Mail Security Awareness: The West Point Carronade. Retrieved 3 December 2019, from https://er.educause.edu/articles/2005/1/fostering-email-security-awareness-the-west-point-carronade
- Gordon, W. J., Wright, A., Glynn, R. J., Kadakia, J., Mazzone, C., Leinbach, E., & Landman, A. (2019). Evaluation of a mandatory phishing training program for high-risk employees at a US healthcare system. Journal of the American Medical Informatics Association, 26(6), 547–552. Scopus. https://doi.org/10.1093/jamia/ocz005
- Jansson, K., & Solms, R. von. (2013). Phishing for phishing awareness. Behaviour & Information Technology. https://www.tandfonline.com/doi/abs/10.1080/0144929X.2011.632650