IT Compliance Institute, April 10, 2007:

Best Practices

Threats, Compliance, and the Human Condition

Blame human psychology: when it comes to information security, we’re simply not built to intuitively rank actual risks. Learn how building threat models can help companies rationalize the biggest security and compliance risks they face.

By Mathew Schwartz

Human psychology: can’t live with it, can’t live without it.

When it comes to judging security risks, we humans do great—provided we’re responding to an obvious and immediate threat. Thousands of years of evolution have prepared us for the moment we get charged by a rhinoceros: the adrenaline kicks in, and without a second thought we literally run for our lives.

Information security threats, of course, are not two-ton, plant-eating mammals with horns; they’re rarely so obvious and frequently much less imminent. As a result, we typically overreact to less risky threats while ignoring bigger, quieter, more long-term hazards. Thus we obsess about laptop encryption, try to automatically monitor for information leaks, while ignoring the threat of insiders or social engineering attacks, and wait for some impending governance, risk, and compliance platform silver bullet to solve all future problems.

Lack of information is frequently not the cause of our inability to identify our biggest information security and compliance-related threats. Rather, it’s a more fundamental problem. "We are not adept at making rational security trade-offs, especially in the context of a lot of ancillary information designed to persuade us one way or another," alleges BT Counterpane chief technology officer Bruce Schneier in a recent essay titled "The Psychology of Security." In particular, he identifies five areas "where perception can diverge from reality" when it comes to evaluating security trade-offs: risk severity, risk probability, cost magnitude, countermeasure effectiveness, and the actual trade-off itself.

The causes are simple: evolution isn’t done yet. "Our ability to duck that which is not yet coming is one of the brain’s most stunning innovations, and we wouldn’t have dental floss or 401(k) plans without it," noted Harvard psychology professor Daniel Gilbert last year in a Los Angeles Times op-ed. "But this innovation is in the early stages of development. The application that allows us to respond to visible baseballs is ancient and reliable, but the add-on utility that allows us to respond to threats that loom in an unseen future is still in beta testing."

So how can security and compliance managers overcome these basic human-condition stumbling blocks and better rationalize actual threats? The answer: compliance officer, know thyself. Perhaps by understanding our psychological predispositions, suggests Schneier, "we can learn how to override our natural tendencies and make better security trade-offs."

Beyond Darwin: Security Survival in the Information Age

What you don’t know can kill you—or at least seriously damage the business, as you apply resources to mitigating one risk, while potentially missing or ignoring an even greater one. Accordingly, many organizations are adopting risk management frameworks to help executives build a complete picture of all risks to the business, whether they relate to business models, competitors, compliance, outsourcing, or technology.

Yet as noted, human psychology and objective risk assessment have an imperfect relationship. One predominant psychological predisposition—affecting everyone from executives down to IT managers—is what Ed Adams, president and CEO of Security Innovation, and author of the forthcoming Information Security Management: Survival Guide, calls the "recency" trap. "That trap is reacting and often overreacting to a recent or current event that causes you to make security investments—usually in the wrong place, and for the wrong threat."

One example: the trend to encrypt all data stored on a laptop, especially since organizations including Boeing, Ernst & Young, ING, and the Veteran’s Administration lost laptops storing people’s personal information. "The reaction is, all of a sudden we must encrypt all data on any machine that could possibly leave the building, and that’s a reaction that’s going to be very costly in terms of time and productivity," says Adams, and possibly not all that useful a security countermeasure at many organizations.

Indeed, encryption is notoriously difficult to implement; many organizations try but don’t get it right. Thus laptop encryption may create the feeling of security, while actually leaving organizations less secure—a double-whammy. "You only have a certain amount of resources, and time, and attention, and when you mandate something like ‘all data on laptops must be encrypted,’ you’re taking your eye off of other problems that are much more real and much more risky," he says.

Toward Rationality: Threat Modeling

Instead of just reacting to current events, Adams recommends a different approach: threat modeling. Namely, identify the most likely threats to high-value applications, development processes, business logic, or just the business itself. Then see whether these threats are currently mitigated. If not, then organizations can create an action plan to prioritize risks and assign resources appropriately, starting with "low-hanging fruit, areas prone to attack, or at high risk," he says.

How can organizations build threat models? Useful starting points vary by domain, but include the National Institute of Standards and Technology’s Special Publication 800-30 ("Risk Management Guide for Information Technology Systems"), the Microsoft Threat Modeling Process, and the forthcoming, version 3 of the Information Technology Infrastructure Library (ITIL), which advocates threat modeling as part of best practices for IT service delivery. "All ITIL is saying is, as a good IT department, you’re delivering high-quality service, and this is one of the activities you should be conducting so you can best prioritize how to spend your time when a new threat becomes apparent," says Adams.

Threat models, once generated, also persist, and this speeds ongoing risk assessment. "When a new risk is produced, you can pump it into that model and see if you’ve already mitigated against it, or if you need to address it," he says, as opposed to having to conduct penetration testing on every potentially affected application or business process.

Threat Modeling for Beginners

If you are just starting out with threat modeling, model high-value applications and business processes first, since they’re most at risk, and also because small mistakes can mean big problems. For example, Adams says his company recently tested the security of a Georgia bank’s online consumer banking application, which the financial institution rated as extremely secure. Yet when the security team opened several bank accounts in a short period of time at different branch locations, they quickly noticed a pattern: new bank accounts appeared to be numbered sequentially. So researchers created a list of 1,000 likely account numbers, then looked for a way to guess the four-digit PIN required to gain online access without tripping the "three strikes and you're out" safety check, which then requires a call to customer service to reset your password.

Accordingly, the security researchers tested two common PIN numbers for all 1,000 accounts: "1234," with 29 percent success; and "0000," with 14 percent success. In short, "using just two PINs, and counting on the psychology of users, we were able to get access to over 40 percent of accounts," says Adams. This, obviously, was a large risk to the business, especially because such access wouldn’t appear abnormal. Thanks to the threat model, however, the bank was able to quickly address the previously unidentified problems.

From Reactions to Rationality

Building a threat model helps security and compliance managers identify the actual risks their organization faces. Thus threat modeling allows an organization to know: "Do I invest time, money, or resources in this area, or do I invest those resources somewhere else?" says Adams. "That’s hard to do in the absence of a threat model, because you’re prone to fear, uncertainty, and doubt." These psychological predispositions, however natural, unfortunately don’t make for optimum security investments or IT compliance success.

Mathew Schwartz is a former contributing editor for the IT Compliance Institute. You can contact him about this and other articles at Mat@PenandCamera.com.

This article originally appeared in IT Compliance Insitute and is reprinted by permission of 1105 Media, Inc.