Invisible Security with AWS

Clarke Rodgers:
Welcome to the Executive Insights Podcast, brought to you by AWS. I'm Clarke Rodgers, Director of Enterprise Strategy, and I'll be your guide through a series of conversations with security leaders.

Today's guest is Chris Betz, AWS Chief Information Security Officer. When Chris joined us last year, we dove deep into AWS security culture. Today, it's all about threat intelligence. We'll discuss how AWS identifies and remediates threats throughout its network and how that protects our customers, partners, and community at large. Please enjoy.

Chris, it's so great to have you back on Executive Insights.

Chris Betz:  
It's great to be back here.

Clarke Rodgers:
So the last time we met, roughly a year ago, you were relatively new into the role of CISO at AWS. Now that you've had some time in the role, what has really sort of impressed you about either internal operations at AWS, from a security perspective, or just the way we run our business? And additionally, any customer insights that you've had, from meetings you've had with multiple CISO customers?

Chris Betz:
Great set of questions. Let me start off with, it's been an incredible year. It's been so much fun being part of the team that's delivering these kinds of security improvements. And I don't think there's a day that goes by where I don't appreciate having been a customer for so long. Because we talk about customer obsession, we are customer obsessed. But being able to carry that viewpoint every day, about, “Here's what we need to do, here's the pain that customers are feeling, here's the opportunity where we can make things better for customers,” is just incredibly powerful.

I think as a second thing over the past year, has been just continuing to deeply realize how security, as a top priority, is something that really resonates throughout the company. I was just thinking last week, as I was talking with the head of engineering for EC2, about how easy it was to have a conversation about security, because he was already three steps ahead of me in terms of thinking through the security risks, how he wanted to address them, and so forth. Having that thought in every leader's head around the company makes my job so incredibly easier, and lets us move so much faster.

Clarke Rodgers:
It's amazing to see how real it is, and it's just not a marketing slogan, right?

Chris Betz:
It's lived, and it's incredibly tangible. The third observation is just the continued advance of generative AI, which I know we also talk an awful lot about, but the way it's moving so rapidly. The way we're moving so rapidly in response, is continuing to color so many of our conversations. There's a ton of advantages. There's a ton of capabilities that we get to bring to bear, and we do, and continue to do that really hard research.

Clarke Rodgers:
Let's go further on that innovation train, so to speak. So, if I'm a customer looking at AWS, I know you build features and functionality to make AWS security better, how do you think about building versus buying security tools to help run AWS security, and make AWS, and the larger HAQM even, secure?

Chris Betz:  
When I step back from those decisions, there's a couple of different things that I need to think about. My internal customers, the builders, the developers within AWS, how am I helping them achieve the security outcomes that we need as easily and seamlessly as possible? That's part of my job. It's just one of the things I must do. But you also bring up that really interesting build versus buy conversation, and there's a couple different ways I tend to think about that. One of them is in terms of core versus context. There are some technologies that are absolutely core to the way that we need to operate as a business. So, for example, how we secure each one of the EC2 systems that are running our AWS workloads, that are powering customers.

Achieving security at that scale is absolutely core to the business. Those are places where I'm more likely to invest in custom capabilities that operate at that scale, and that meet the threats that we're facing every day in that space. Because they're unique, and that's unique to us. There are other places where I'm much more likely to look towards a commercial solution, or look towards a vendor and challenge them to be able to operate at our scale. A good example of that is my laptop that I work on every day. There are security components on those systems that I buy from outside vendors, because we're not doing something that is absolutely unique. We want to leverage that capability that's already out there. It's not in the critical path for our customer delivery, and it's more context to the business.

And so that's part of how I go through each one of these decisions, is thinking through, what's available out there? How do I make sure we can go faster, leverage the best technology I possibly can? Whether that's an internal build or an external product. And how do I make sure that in the places that are absolutely unique, the places that let us deliver secure capability, let us deliver the engineering product that our customers expect at that incredibly high bar, in those places, I'm much more likely to build. In other places, I'm much more likely to look for, does the solution already exist?

Clarke Rodgers:
Being a CISO is also being a business leader, right? You have different leaders over different aspects of your AWS security business. So, you have some security engineering, you have operations, you have security assurance, among several others. How do you hold those business leaders to account to what they're supposed to be delivering to you, as the CISO?

Chris Betz:  
As a security leader, I'm responsible for delivering solutions, services, that work across AWS, to power all the teams across AWS. From that perspective, my quarterly business reviews, my monthly business reviews, my weekly business reviews, bear a lot of similarity to the way service teams run across AWS. And so, as a builder, as an operator of systems, that's one lens that I look through. And I get to draft from and learn from the best of what happens across AWS. There's a second set, is the operational aspect. I've got people providing direct response activities, threat intelligence, investigations, all those things. And in that space, I get to, both borrow from the best of our operational infrastructure, the field organizations, the customer support organizations, and I get to learn from them. And again, my processes look similar, but they're uniquely tuned. And again, my weekly, monthly, quarterly review process works there.

Clarke Rodgers:
You mentioned threat intelligence. Over the last year or so, AWS has shared more about how we think about threat intelligence, and some of the tools that we've built, internally, that help customers and help AWS. There's MadPot, Mithra, Sonaris, and I'm sure there's some others. Can you sort of give me an idea about how AWS thinks about threat intelligence, internally, and then how that helps customers?

Chris Betz:
One of the things that I did not appreciate when I was outside of AWS, is how much happens within AWS that's just not visible. As I went around and learned more about the security AWS is providing, and I talked with the organizations inside of AWS, I realized that one of the reasons why we don't spend our time talking about these capabilities, or we hadn't, is because we believed that these are just things that everybody does and everybody should do. And I had to share with people that, "No, this is not stuff that everybody does, but I certainly agree, it's stuff that everybody should do." And so, I think we're on this journey now to share more and more about what we do, that's invisible and transparent. So, part one of this conversation is, how do we help people understand what we're doing on their behalf, that they may not be aware of?

I believe that the best security is security that is seamlessly part of how people work. If they don't need to worry about, how do I turn on security? If they don't need to worry about the detailed nuance, if it just protects them, that's the best possible place for us to be. And so, I think our threat intelligence conversations have been at the intersection of those two places. One, we've been doing a ton of work that folks didn't understand. We should share more of that. It's a little bit of a show me thing. You know we're secure, you know we take security as our top priority, let's show you a little bit more to help bolster that confidence. And the second piece is, and the reason you don't know about it is because we've hit that place that we really want to be, that seamless security. And so, in general, as we think about this threat intelligence work, one, it's important to understand what malicious actors are doing. That changes our risk profile. It changes how we think about risk. That also powers so many defenses.

You want defenses tailored to threat actors, tailored to the malicious activity, and we're able to do that. Threat intelligence powers that. The second piece is that seamless piece. You want to enable a seamless defense. The conversations we had on Sonaris are a great example of that seamless defense, where we're able to protect customers without them even knowing, even having to know, because we're identifying malicious actors and we're blocking the malicious actors, and enabling the customers to operate. Threat intelligence is valuable when it's used to protect people. And so, how do we merge that threat intelligence into the tools, the capabilities, that our customers are using, so it's a seamless part of how they're operating their systems? The alerts come up, they're all integrated in, so that people can take the right actions at the right time.

Clarke Rodgers:  
I love that you brought up the integration question. How does the integration work when your team operates something like MadPot and the consumer product is GuardDuty. A lot of it, if I'm the customer, I'm looking at it, "Hey, these kind of do the same things." Does MadPot feed into GuardDuty itself, or are they two distinct products?

Chris Betz:
Spot on. So MadPot feeds into GuardDuty, as one of many feeds into GuardDuty. And where it's high enough fidelity, MadPot also powers things like Sonaris. Because if we know that a particular IP address and port is a malicious actor, we use things like Sonaris to say, “That IP address and port just doesn't get to talk to anybody on AWS.”

Clarke Rodgers:  
Oh, that's amazing.

Chris Betz:
We're able to focus on the bad actor and stop the bad actor, where we're not in that space where it's a mixed signal. We may just provide that tip to customers through GuardDuty. And so, these feeds go multiple directions. But the whole goal is for these systems, these threat intelligence systems, to power all the different layers that we operate, the infrastructure layer, the security of the cloud, that I sweat every day, and our customer security in the cloud, the services that they operate, and that I worry desperately about. How do we make sure that we're providing them all the capabilities that we can?

Clarke Rodgers:
Staying on technology, generative AI has been all over in the last year or 18 months. And I would say, broadly, the majority of our customers have been focusing on, "How do I secure gen AI?" Whether it's a third-party service, making sure the configuration is set appropriately on HAQM Bedrock, those types of activities. Where's my data, how do I secure it? All that kind of thing. I haven't heard a lot about using generative AI for security outcomes. And I would love if you take a few moments to share what we're doing inside of AWS, leveraging generative AI for security outcomes.

Chris Betz:
One of the first places that we, and customers, are seeing incredible value from generative AI is in simplifying and enhancing our security engineers work. One of the ways I think about my job is, my job is to find and hire the smartest security people, the best engineers that I possibly can. And I want those incredibly smart people solving hard, interesting problems every day. And where something is routine or well understood, I want computers to do that work, not our smart people. And I think generative AI is another tool in that toolbox.

So, for example, internally, within my security operations center, one of the things that has to happen, as we have a 24/7, around the clock, is we have to hand off security events that we're tracking from center to center, from engineer to engineer, around the world. That takes time. It takes time to summarize and diarize what's happening. It takes time to ingest that. That is undifferentiated heavy lifting, to provide those insights. What's been amazing over the past year is watching the team use generative AI to speed that process, decrease how much time people spend taking notes and tracking what happens, just have generative AI do that automatically. Summarization. And watching the engineers on the other end as they pick up those new events, being able to rapidly process a summary, dive in where they need to, and pick up that work. That improves the speed, that improves the throughput, and that frankly improves, not just the quality of the work, but the way people enjoy the work. They're able to focus on those hard problems that they enjoy.

One of the things that I've enjoyed the most about AWS, is how everybody, not just the security organization, sees security as their top priority. And so, builders from across AWS frequently reach out to our security folks, and say, "Hey, I'm building something this way. What's the best way to do it?" The frustrating part is when it takes us hours or days to get back to that builder, especially if it's a well-understood area and a well-understood question. Because through that time, we're holding up that builder. Sure, they may have other things to work on, but we've interrupted their thought, we've interrupted their progress, and it's my job to make security seamless. That's certainly not a seamless experience. And so, we've introduced suites of generative AI powered security tools that help answer these engineers' questions in seconds.

And when the generative AI tool isn't ready with the right answer, where this takes a deep nuanced thought, where we have to build and innovate new ways to do security, we can rapidly identify that, and our security engineers get to spend time on those really interesting problems. They get to dedicate their time to solving the hard problems, with the builders across AWS. So, this is really a win-win. It's better experience for the security engineers. We get better throughput, and we've made security more seamless.

So, I'm seeing customers do some of the same things we are, knowledge sharing, information sharing. But I'm also seeing them use some of the really interesting AWS capabilities that we've built, whether that's our integrations into GuardDuty and Detective, which we use as well. But I'm seeing customers do an amazing job leveraging the generative AI built into those capabilities. But I think one of the most interesting conversations I've been having is around CodeWhisperer. I know a number of customers whose security operation centers have fundamentally changed with the introduction of CodeWhisperer. In the old world, they used to have engineers, analysts, who would spend their time sitting in front of spreadsheets, looking at alerts, manually running queries, trying to put the data together, and stitching together a potential event, an investigation. And each one was a set of handcrafted custom work. With the advent of CodeWhisperer, the security operation centers have been able to really change how they work.

Chris Betz:
What used to take hours of manual work, takes minutes to develop a tool, and seconds to execute. That's incredibly powerful in a number of ways. First, the repeatability, the traceability, you're able to go and look, how did I do this? And the reduction in human error is incredibly powerful. And people are spending time asking about events very, very differently. And so, some of the security operations teams, I know, are getting much deeper into investigations, better able to go hunt for adversaries within their networks, because the power of code, for people who were not primarily developers, is enabling them to move faster, learn faster, and go deeper into their infrastructure. The other place where they're seeing CodeWhisperer be really powerful, is in inspecting their code as part of their application security processes.

As people develop code, that CodeWhisperer logic, to be able to go through that Q Developer capability, and say, "Hey, is this code secure? Are there errors that we've made here? Are there things that we could do to improve security?" In getting those hints, is meaning that people can shift code faster, the security reviews have fewer findings, and it's leading to higher confidence in the developer's code, and fewer fixes once they're in production. Better reliability. It's life changing for some of these organizations.

Clarke Rodgers:
When I speak to customers, there's often a mandate that comes from the top, that says, "You must use generative AI to do X, Y, or Z. Or you must investigate it." Et cetera. It sounds to me, from the way you phrase things, it's that security operations center team said, "Hey, gen AI can help us with the handoff, and I'm going to build this capability in there." Is that the case or are you ensuring that people are using the latest and greatest technologies? Or is it sort of bottom-up, of, "I'm going to make this the best service or product that I can, internally, therefore I have the freedom to go do these things."?

Chris Betz:
I work hard to make sure that I give the organization the bandwidth and the room, the amount of resources that they need, et cetera, to be able to innovate within their own space.

The kind of people that I want to have on my team, are the kinds of people who like to solve those hard problems. I certainly support this work. I'm certainly involved and engaged with it, but I don't have to direct it.

Clarke Rodgers:
That's fantastic.

Chris Betz:  
I have to give people the opportunity, support, and reward it. And look, not every experiment works, and that's okay too. We need to be willing to let people fail. Our job as security leaders, is to create an environment where folks are not just doing the same thing over and over again, to incentivize and to support people who are willing to try to innovate. Because I think the security leaders, the security engineers that we want on our teams are those that are willing to take a chance, to think about problems differently. And so, it's our job to create that environment and support it.

Clarke Rodgers:
Well, let's talk about that a little bit more. What is the ideal security hire for AWS?

Chris Betz:  
I'm always looking for somebody who raises the bar. I want us to continuously get better. But that's kind of table stakes for us, right? Raising the bar is the way we start the conversation. I'm looking for people who understand security deeply, have a deep understanding of the fundamentals behind it, and are willing to innovate and adapt. Because the problems that we have to handle across AWS, the scale that we operate at, is second to none. The speed that we have to operate at, is second to none.

I'm looking for folks that have demonstrated a willingness to take a chance, to fail fast. And I'm looking for folks that are deeply, deeply obsessed in results, in solutions that work backwards from a customer. When I see those things, those are most often the right folks.

Clarke Rodgers:  
How does one get prepared for the speed and scale of AWS?

Chris Betz:
I don't know that there's a way that you can do it, out of the box. And that's where the adaptability and the fundamentals become so powerful, because with the fundamentals, you've got that foundation to build on. And with the adaptability, that lets you start getting up to speed with the ideas of just the crazy numbers, the tens, the hundreds of millions of things that we talk about. Just, when you start operating at that scale, things are different, and that's special, and it takes learning.

Clarke Rodgers:
In my conversations with customer CISOs, one thing continues to come up, and it's reporting to boards. And there's really two sides of this. There's the, “I am a director on a board, I don't know what questions I should be asking of my CISO or my security organization.” And then the other side of it, “I'm a CISO, I don't know how to present to the board in a language that they'll understand what I'm trying to tell them.” Could you tell me how you navigate these?

Chris Betz:  
Boards are made up of people. And so, to be effective as a CISO, it's important to understand how your board wants to operate, how it thinks about the business that it's in, and who the people are. Because your job, as a CISO, is to communicate to the people on the board in a language that they understand. And so, my first advice, whenever a CISO brings this up, is to just take that step back and recognize boards are different, companies are different. Let's look at how other people are successful talking to those boards. Let's look at the people who make up those boards, and figure out how to communicate with them. For the boards, I smile, because I think the most frequent question I get is, are we spending enough on security?

Which is a good question, but it's also representative of what you said. Boards don't always know what questions to ask, and so that's a safe question. I think it's important for every board member to understand that security, as a risk domain, is different from many of the other risks that you look at as a business. When you're looking at financial risks, you know that if you go make an investment, a bet, in a part of your business, you can understand how big that bet is, right? Is it a 10,000, 100,000, a million, 10 million, a billion-dollar bet? But you're able to scope that bet. Technology risks, especially security risk, is unique in a couple ways. First of all, it's highly distributed. Every engineer can make a set of decisions that increase or decrease your security risk. In fact, security events are often about really small, narrowly made decisions, and a series of them, that lead to that risk. And so, it's highly distributed. Every engineer has a stake in security.

The second thing is that what appears to be a small event can actually have outsized impacts, right? And the third is, there are limited opportunities to test and learn. In many domains of risk, you can start off with a small bet and see what happens, and then make a bigger bet, and see what happens. And you can test and learn, and you can get feedback on how that works. Securities, sometimes a decision made five years ago, all of a sudden comes back to roost, in a really big way. And so, I tend to think of it more like an asteroid coming in. You can have these major, major events that are just hard to anticipate and test and learn. And so, the questions that I would ask as a board member, of a CISO, certainly include, what are our investments like? But they should challenge the CISO to come back and identify where the CISO sees more or less risk.

Because the question is, “Is this within the board's risk tolerance, and is the CISO doing that analysis to truly understand where those risks lie within the business? Do you know where your assets are, where your data is, where your systems are? What kind of visibility do you have on that?” Are really important questions to ask, because that helps you understand the quality of that risk analysis. How are you and the CIO, or the CTO, working together to make security easy for those engineers? People make mistakes. Folks are trying to get work done. If it's not a seamless part of how people work, you're going to have more of those errors.

You can put a ton of energy into security, it's still going to be imperfect. You've got to have multiple layers in order to be successful.

Clarke Rodgers:  
And are you looking for that sort of shift in risk, viewing the risk appropriately?

Chris Betz:  
Exactly.

Clarke Rodgers:
If I'm shifting my program, I understand the risks that's happening.

Chris Betz:
Spot on. You're looking for that dynamic thinking. You're looking for a CISO that is staying up to date with how their risk posture has changed, up to date on new threats, and making ongoing risk decisions to focus on the largest threats to the organization. And if the program remains static, if you're hearing about something that sounds very much like the program you had two years ago, that should lead to some really interesting questions, to understand, has the threat changed? Maybe it hasn't. Has the program updated, or is the program just stretched so thin that they're unable to take on the new and challenging threats? So that leads to a really good set of conversations.

Clarke Rodgers:  
That's fantastic. Very insightful. Thank you so much, Chris. It's been a pleasure to talk to you today.

Chris Betz:
Thank you.

Listen now

Listen now

Listen now