Threat modeling and architecture.

This is the third article in Adam Shostack's series on threat modeling. His previous two - Threat Modeling: What, Why, and How? and Rolling Out a Threat Modeling Program - further discuss how threat modeling is essential for organizations to become proactive and strategic in operational and application security.

After I wrote my last article on Rolling out a Threat Modeling Program, Shawn Chowdhury asked (on Linkedin) for more information on involving threat modeling in the architecture process. It’s a great question, except it involves the words “threat, “modeling,” and “architecture.” And each of those words, by itself, is enough to get some people twisted around an axle. Architecture, in particular, is controversial, but without a definition it’s hard to say if or how threat modeling might help. A colleague wrote, “my basic rule of thumb has always been that architecture is 'that thing above the level at which you work’,” and another said, “An architecture is ‘the fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution.’” I’ve used the definition that an architecture is “a consistent system model shared by the engineers on a project that enables faster delivery with fewer problems.”

One element that each of these definitions shares is that they implicitly invoke models: abstractions which help us reason. And there is the fundamental tie; both threat modeling and architecture involve models of systems and analyzing properties of those models. Billy Vaughn Koen, now a professor emeritus at the University of Texas, wrote about the engineering method, which he defined as “the strategy for causing the best change in a poorly understood or uncertain situation within the available resources.” Dr. Koen wasn’t writing about software, but his definition is helpful in understanding the goals of software architecture work, which can be seen as strategies for constraining how developers invest effort. That is, an architectural approach defines the choices that a developer should make: use these APIs, these languages, these components.

Before we get to “this is the architecture we’ll use for this service,” though, we talk about tradeoffs we can make. In threat modeling, I’ve defined this as a four step process:

  1. What are we working on?
  2. What can go wrong?
  3. What are we going to do about it?
  4. Have we done a good job?

Let’s look at the steps with a broader lens. Asking, “What are we working on,” and ensuring that the answers are aligned, is an important part of both architectural and threat modeling processes. (One important and difficult aspect of this is to explicitly state what’s out of scope, or non-requirements. This is challenging because someone wants to build that feature or improve that property, and we have to tell them no, this feature is more important. It’s easier to let it slide, but it results in less focused products with features that don’t make sense.)

The question of, “What can go wrong” is far more focused in threat modeling. Architecture discussions can and should incorporate threat models along with other types of analysis of other failures.

In a really fascinating blog post entitled “Multiple Perspectives on Technical Problems and Solutions,” former Etsy CTO John Allspaw wrote about his architectural review process. He talks about how architecture review is not about making a decision, but about informing the dialog around the choices to be made, and describes the process:

  • Describing the problem they believe needs solving. This may or may not be straightforward to explain or describe, so a back-and-forth is usually needed for a group to get a full grasp of it.
  • Generating hypotheses about whether or not the problem(s) being described need to be solved in more or less complete or exhaustive ways. Some problem descriptions might be really narrow, or really broad, some problems don’t have to be fully “solved” all at once, etc. Will the problem exist in perpetuity?
  • Evaluating options for solutions. What are the pros and cons? Can a group that needs to support a solution sustain or maintain said solution? Will some solutions have an “expiration date”? What possible unintended consequences could result in upstream/downstream systems that depend on this solution?

Does that look familiar? It should. It is remarkably parallel to threat modeling. Etsy, in general, has written powerfully on the topic of understanding the organization’s successes or failures during a project, and John, specifically, has written about learning as you go.

So let me make it explicit:

  • Threat modeling is a contributor to architectural analysis and decision making.
  • Threat modeling contributes by helping you define threats and the requirements they violate.
  • Threat modeling contributes by helping you define possible solutions to security problems.
  • It breaks security into smaller parts to help those who are not security experts grapple with security questions.

There are two ways this commonly happens. Architectural investigations can drive threat modeling or threat modeling can drive architectural investigations. 

In the first, there are one or more possible architectural directions that might be sensible, and one of the ways to distinguish them is to answer, “which ones create or solve security problems?” Threat modeling structures the way you start to answer those questions for each possible architecture. 

In the second, threat modeling leads to understanding that architectural choices have security implications, and those choices may need to be revisited. Those choices might have been made when security wasn’t important or wasn’t considered. Those choices might have been made when security was defined differently, or before clever new attacks were discovered. (It’s easy to get sucked into a blame game in these cases, and usually counter productive.) In these cases, threat modeling gives structure to a set of security issues, and enables you to start asking, “Can we change the architecture in a way that reduces these problems at a managable cost?” A managable cost involves both financial cost and the tradeoffs that a new architectural choice requires.

In my experience, when an architecture review brings attention to a problem and proposed solutions from multiple perspectives, decisions become less controversial. When a decision appears to be obvious to a broad group (“Question: should we (or should we not) take backups of critical databases? Decision: Yes.”) how a decision gets made almost disappears. — John Allspaw

It is tempting for security people to argue that security should take the place of pride in a discussion. (After all, we describe ourselves as security people; it’s important to us.) It is also tempting to accept the frame that we have to “fight for security.” Few people like conflict, and fewer handle conflict as constructively as they believe. In the “fight for security” frame, we can be heros, if just for one day. But security isn’t fixed in a day, and more useful than being a hero is being an engineer. (That’s a perspective that I’ve discussed before.)

In each case, the goal is to transform security from a vague worry to a crisply-enough modeled threat, and then ensure that addressing that threat gets the right degree of attention. That’s threat modeling, that’s architecture, that’s security engineering.

Interested in learning more about this topic and others? Attend our upcoming Threat Intelligence Summit in Austin, Texas this November!