Bruce Schneier on Security Metrics that Matter

“I like to measure the performance of the team,” said Bruce Schneier (@schneierblog), CTO of Resilient Systems, Inc., in our conversation at the 2015 Black Hat Conference in Las Vegas. “I like to see metrics about people, about process, about technology. There isn’t one metric that works since it’s such a complicated and moving target … Right now companies have to use the data that they have to figure out if their teams are effective.”

Schneier feels that certain metrics, such as blocked attacks, don’t really provide a gauge of how secure you are.

“Metrics can tell any story. Question is what story do you pull out of the data? Right now my worry is there’s too much data, too many metrics. You can say anything,” said Schneier. “But really, you’re making up a story with the data.”

What you really need to do is measure the team’s performance. How are they doing?

“If you can measure when and if your team responds and how they respond and how they react and how fast they close serious incidents then maybe you have something,” said Schneier who realizes that a lot of current metrics aren’t that useful. “How do you measure how effective you’d be against a Sony-like attack? You kinda can’t.”

Here at Tenable we try to help security teams explain to the business their current state of security. It’s a hard thing to define, and it’s even harder to communicate.

“There aren’t any really good pithy ways for the CISO to tell the board we’re doing OK, we’re not doing OK. It’s going to be gut. You’re going to stories instead of data,” said Schneier. “But there is this disconnect because it’s such a technical topic and the board really wants a soundbite.”

Assurance Report Cards™ (ARCs) are Tenable’s answer to bridging the communication gap. ARCs are available in SecurityCenter™ 5.0 and display an enterprise’s security posture in the familiar format of a report card. They measure and visually communicate the status of an organization's most critical security controls.

Future of endpoint security

I shifted the conversation with Schneier to talk about endpoint security, and asked him how vigilant he felt we are with the proliferation of devices.

“I worry less about computers and more about the cheaper devices – phones and the embedded devices, the Internet of Things. The endpoint security there is really terrible,” said Schneier.

For all the unknown devices that perpetuate our networks, Schneier pointed out two competing visions. The first is requiring minimum standards for devices on the Internet.

Any such requirements seem difficult if not impossible to enforce.

The other, which seems more plausible, said Schneier, is that the network needs to be smarter. The reality is there are always going to be unknown devices on every network. The goal is to get security in spite of that.