SF - 1.03.0 - metrics
Metrics
I need to talk, at least a little bit, about metrics. Firstly because an awful lot of security frameworks will either demand or provide you with metrics. Secondly because of the close tie between security and management. And particularly the statement that what you can't measure you can't manage. I'm not really sure that I entirely agree with that statement, but it has a lot of merit to it. An awful lot of people will want metrics to indicate that the security efforts that you make with regard to a certain security framework will in fact improve the security posture in some measurable way.
Of course, as soon as we talk about metrics we start to talk about KPI, or key performance indicators. This is just really metrics by another name, but management types tend to really appreciate key performance indicators.
In regard to the key part of key performance indicators, I should recommend a book by the name of "PRAGMATIC Security Metrics," by Brotby & Hinson. Pragmatic is not just a description in the title of this work, but an acronym, pointing out that the security metrics that you choose should be predictive, relevant, actionable, genuine, meaningful, accurate, timely, independent, and cheap. I highly recommend this work as it points out that not everything that you can measure really gives you any information about how you should manage. The book itself will provide more details on all of the terms that I have just listed, and I highly recommend it for anyone in really any field of management, but particularly security.
I really enjoy the game of curling. I appreciate the complexity and strategy of the game. I tend to tell people that it's like playing chess, if the chess pieces are forty pounds each, and you put them on the board by throwing them down a sheet of ice to a position over a hundred and forty feet away. No, I am not changing the subject. If you watch curling on television, the commentator will give you statistics for the players. But what does it mean if someone has a hit rate of 67%? What does it mean if a player has a draw rate of 73%? Operationally, you should either place the stone where it's supposed to be, or not. That's either a one or a zero. But, I suppose tactically, have you hit the other stone at precisely the right spot to push it out of the way, or did you get it 67% close to the precise spot? When you draw down the ice, have you placed the rock perfectly, or is it 73% likely, strategically, that your opponent will not be able to draw around your stone and mess up a subsequent activity?
In terms of management and communication of extremely complex technical information, in an extremely complex and difficult situation, I always recommend that the master class in regard to communication of this sort was the Dr Bonnie show during the pandemic. The information was delivered, more or less on a daily basis during the high point of the pandemic, not just in terms of the numbers, but in terms of what they meant. One example was the effectiveness of the vaccines, as they started to come along. The Pfizer and Moderna vaccines were recommended, because they both had an effectiveness rate of 90%. AstraZeneca was to be used only as a kind of a last resort, since it only had an effectiveness of 60%. Presumably this meant that 60% of those who got the AstraZeneca vaccine did not contract the disease during the testing period. However, AstraZeneca could, quite reasonably, have claimed an effectiveness of 89%, since 89% of those who got the vaccine, whether or not they got covid or not, did not become very ill and did not require hospitalization. In fact, AstraZeneca could, also reasonably, have claimed an effectiveness of 100%, because no one who got the AstraZeneca vaccine, during the testing regime, actually died of covid, and therefore 100% of those who got the vaccine survived.
Hopefully this goes some way to pointing out that metrics alone, in isolation, are not necessarily the final word on the effectiveness of security. Security metrics are indicators, and generally very valuable indicators of what is going on. But you have to understand the implications of the particular metric. Not everything that can be counted counts.
Security frameworks (SF) series:
Introduction and ToC: https://fibrecookery.blogspot.com/2026/06/security-frameworks-sf-0000-intro-and.html
Next: TBA
No comments:
Post a Comment