Cloud security fundamentals part 5: Measure what matters

Many security engineers have woken up to dozens of Slack messages and emails telling them the day they dreaded is here: a vulnerability has been deployed, and now it must be fixed. Meetings and plans are abandoned while security engineers rush to fix the problem.

It’s often a process failure that has led to the now-urgent issue. And these emergency issues can appear across a spectrum that includes all types of remediation efforts. In these cases, cloud security can feel like a game of whack-a-mole, whether security engineers are tackling urgent vulnerabilities or the next swath of misconfigurations.

The solution is to make a shift to proactive systems and processes that quantify and measure security efforts.

What matters and why?

People often repeat the phrase “what gets measured gets managed” — a saying originated by management consultant Peter Drucker — because it points toward a core truth: if you’re not measuring a given effort, you’re likely not prioritizing the work you’re putting into that effort.

That’s why figuring out what matters and why you should measure it is critical for operationalization.

For cloud security, the reason to measure is simple: an insecure environment can lead to a breach, which causes monetary, reputational, and regulatory losses. Figuring out what matters is where it gets complex. At the scale of the cloud, there are many things that potentially matter, and the sheer effort of identifying and quantifying all of them is a journey unto itself:

• There are rates to measure, such as the rate of misconfiguration and the mean time to remediation (MTTR).

• There are systems to measure, such as how well different tools work together and how much time is spent remediating vulnerabilities, repairing infrastructure, and designing systems securely.

• There are automation and allocation efforts to measure, such as the ongoing process of automating simple tasks so people can focus on higher-value work.

The methodology remains the same: establish a baseline, set goals, and work toward achieving those goals. But each effort offers a different context, as well as a different set of variables to measure.

Take MTTR, for example. Many companies measure MTTR in terms of hours or days. But attackers move quickly after discovering a vulnerability, meaning fixes have to be quick too. Especially when it comes to critical resource configurations, teams need to:

1. Measure MTTR.

2. Measure MTTR such that they can work toward being able to remediate issues in minutes rather than days.

How do you measure what matters?

Precise measurement is too complex and too context-dependent to specify and dictate from the outside-in. What companies can do, however, is work from core questions that will guide their answers.

Three core questions will help structure your measurement efforts:

1. How secure are you? 

2. How productive are you?

3. What’s your ROI?

Each question results in a series of additional questions.

How secure are you?

To answer, consider:

• What’s your visibility coverage for your environment?

• How out of compliance is your cloud environment?

• How many vulnerabilities are you finding and eliminating?

• What is your MTTR?

• How many vulnerabilities are you preventing before deployment?

How productive are you?

To answer, consider:

• How quickly are you approving cloud deployments?

• How many cloud engineering hours are you investing?

• How many cloud security hours are you investing?

What’s your ROI?

To answer, consider:

• Are developers delivering new products and features faster? 

• Are cloud engineers focusing more time on building value?

• Are security teams doing more with what they have?

With these questions guiding you, you have what you need to start measuring what matters.

What enables measurement and progress?

Eventually, security teams should be able to answer the above questions on the spot. Being able to do this, however, requires not just the measurement efforts described above, but also the foundational ability of different teams to self-organize and work together.

Cloud security is inherently complex and no one team,much less one person, can handle it alone. Operational excellence requires equal parts organization (knowing what’s running in your cloud environments and maintaining them) and collaboration (DevSecOps), followed by measurement that shows how development and security teams could be using their time more efficiently.

Effective cloud security not only relies on sheer operationalization but on operational excellence across teams. Security teams need to work with cloud engineers to quantify all resources, configuration attributes, resource relationships, and application code. Teams need to map out the entire software development life cycle (SDLC) and toolchain, and involve everyone who contributes to both.

In the effort to measure what matters, companies need to build measurement muscle. Doing so requires that everyone works together. 

Make the abstract tangible so you can measure and improve it

There’s no objective difference, nor black and white line, between secure and insecure. Many companies are confident they’re secure — and then a breach happens.

By operationalizing cloud security, you can turn your intuition about how secure your environment is into a tangible, measurable series of processes and metrics. Improvement is only possible when you can measure where you are now and where you want to be.

Operationalization reveals what matters, shows you how to measure what matters, and lays the foundation for the sustainable work of measurement and improvement. When security becomes an operation, incidents are less likely to occur, and security engineers can sleep more soundly.