Jon Scheele (00:01)

As API traffic now makes up over 70 % of overall internet traffic, it's a demonstration of how important APIs are to how our economy works. But also, it means APIs are just as much a cyber target as everything else we see on the internet. So to pick apart this and to discuss the threat and what we can do about it, I'm very pleased to be joined by Chuck Herrin. He's the Field Chief Information Security Officer at F5. Welcome Chuck.

Chuck Herrin (00:39)

Yeah, thanks very much, Jon. Thanks for having me.

Jon Scheele (00:41)

Great talking with you. It would be great to just share your journey through cybersecurity in general and API security in particular. You've joined F5 from the acquisition of Wib, but can you tell us briefly your story through cybersecurity?

Chuck Herrin (01:01)

Absolutely. So thanks for having me. And though I am American, that's, you know, obviously with the accent, I don't like to talk about myself too much. So I'll keep it brief, but I think the background is relevant. So yes, as you said, F5 acquired my company. I was CTO of a company called Wib, API security firm. And we built what we were calling the second generation of API security products in a relatively new industry. API security is less than 10 years old, really, overall, as an industry.

Prior to WIB, I served for roughly 19 and a half years. You can sort of round it up to about 20. As the Chief Information Security Officer in Global Financial Services Companies, mostly based in the US, but also based in Bermuda, and banking and finance. So Texas Capital Bank, AIG, a couple of others.

I did that for about 20 years. And that really helped, in my view, the perspective of what we built at WIB, why we built it, and the audience that we built it for. Highly regulated, high bandwidth, global enterprises was really what we were trying to build the product for. So happy to be here, and I've been with F5 now for about eight months.

And traveling the world, talking to folks like yourself in different regions and different areas about what we see other places in the field and how they can protect their organization.

Jon Scheele (02:29)

Great, thanks for that. in regard to API security, I guess F5 is quite broad in terms of the application security protection as well as API security. And I guess that talks to your expanded role since you know the CISO, not simply the field CTO for API Security, but I was struck by how the OWASP Foundation have a separate top 10 security vulnerabilities for AppSec from API security. And I was at the OWASP AppSec day in Singapore.

last week where we discussed exactly this. So when I looked at the top 10 of both application security and API security, there are some clear similarities. There are some one for one matches between the top 10. But where the AppSec Top 10 talks about insecure access rights, the Top 10 for API security breaks that down into about three or four different things from broken object level authorization, broken object property level authorization and things like that. Could you just talk through why there's a need to discuss some, to address some of these things in a little more detail. What's different about APIs, API security from the security of everything else on the internet?

Chuck Herrin (04:13)

Absolutely. It's a great question. And it's something that I'm glad that you're helping to explain and get the message out to your audience. The web top 10 from OWASP from 2003 is a great set of minimums and most common things to look into. And if you remember back when it was created, it was after the late 90s when companies really started going online. And the security industry responded and said, hey, if you're going to have a web presence, these are the top 10 things that you need to worry about. So it was never meant to be an exhaustive list of everything that a company needs within their security program. It's if you choose to expose your resources to the internet, these are the top 10 attack types or top 10 vulnerabilities that you need to worry about. And the key message, I think the foundational security fundamentals theme for that is and should be your architecture is what drives your attack surface.

So companies that don't have a web presence have no need to worry about the OWASP Top 10 for Web. And in the same way, the OWASP Foundation has come up with a number of very helpful guides and set of minimums to understand when you go into mobile apps, for example, they have a Top 10 for mobile, Top 10 for LLMs, a Top 10 for machine learning, and a Top 10 for APIs. What I think a lot of defenders have done is... the problems with web security were really, really difficult to solve and required a number of years and a lot of effort. And a lot of defenders kind of stopped there. These are the 10 things that we need to worry about going to the web. And they may not have even contemplated the top 10 for mobile when they rolled out their mobile apps or the top 10 for APIs, which was first published in 2019. But if your architectures indicate that you need to worry about these types of risks because you're exposing your assets in those manners, then you need to understand the relevant top tens for those spaces, right? So companies that don't have mobile apps, don't worry about the mobile top ten.

But in the same way that a lot of companies miss APIs now, we're seeing a lot of companies that aren't really taking into account the top ten for generative AI, LLMs, machine learning in general. And sort of history is repeating itself. We're over-indexing on a list of threats that were relevant and are still relevant from 20 years ago without taking into account how our attack surfaces have changed over those intervening two decades and what are the things that we need to be worrying about now. And that's specifically why I got into API security is because API traffic is now the majority of web traffic. It's in a recent quarter at F5, it was 92 % of the attacks that we detected against our global fiber backbone network. It's called distributed cloud.

which is the number four network in the world for IPv6 traffic, the number six network in the world for IPv4 traffic. So it's a very large global network that we manage and monitor and secure for our customers. 92 % of the attacks that we saw against that infrastructure were attacks against API endpoints. This is where the current battle is being waged, but many, many, many attackers are able to operate with impunity because defenders aren't looking yet. They don't know where to look yet. They're still over-indexed on essentially fighting the last war. And it's hard to be critical of defenders that are doing that because the last war is still going on. Like we still have to worry about the web app attacks and attacks, know, SQL injection and cross-site scripting and other sort of what we would call legacy attacks still working against API endpoints. But when we expose APIs directly, now we expose ourselves to a number of different types of vulnerabilities, like the ones you mentioned, broken object property level authorization, broken function level authorization, broken object level authorization, all of these B something something A's that are unique to APIs. So if you don't expose any APIs, you don't have to worry about those so much, but all modern software development is API first. And so part of my mission at F5 is to help defenders sort of catch up and keep up with the changes to their architecture.

And how do we fill these blind spots? Because there's nothing more true in security than you can't protect what you can't see. And that's really kind of where we are from an API perspective. Defenders are sort of just now catching up to understand these exposures that they face. And how do we get the protections in place? How do we get the visibility in place? How do we create an inventory of these APIs? How do we govern these things? Because they've gotten behind the curve.

Jon Scheele (08:49)

So as you say, the top 10 is not an exhaustive list. I guess the way OWASP have compiled this list is that they have looked at data that shows how much the attacks are coming, are targeting those vulnerabilities. They rate them. And this is why the API top 10 is a little more granular because there is that volume of attacks that are also granular. But I guess to your point about it not being exhaustive, if you are not addressing the top 10, you are like the softest target. You may still be a target if you're addressing the top 10 because there are other vulnerabilities, but you would certainly be the softest target if you're not addressing broken authentication, broken authorization, misconfigured security. But this is a constantly evolving space.

Chuck Herrin (09:58)

Third one.

Jon Scheele (10:05)

The things that, you know, OWASP came out, republished their top 10 for API security last year, and some of the vulnerabilities moved up or down based on the evolution of the attacks on this. Where do you see the things that are rising more with API security?

Chuck Herrin (10:29)

It's a great question. You're right. The nature of the attacks changes over time and that's why OWASP, for example, updates their guidance every three years or so. And that's to reflect the shifting attack patterns. And essentially, as you said, if you address the top 10, it's sort of like if we thought about this in terms of physical health. If you address the top 10 causes of death as best you can,

you know, every day, whether they be heart disease, cancer, you know, car accidents, you don't ride in cars, you eat healthy, you exercise every day, that doesn't make you immortal. That does manage your risk, you know, your personal risk on a strong level, right? So you do have some control here, but it doesn't make you immortal. It doesn't mean if you address the top 10 that you don't have any vulnerabilities that anybody can exploit. And...

The principle that we espouse and preach at F5 is your defense needs to be informed by the offense. So that's one of the reasons we run our global threat intelligence services while we conduct our research out of F5 labs. So we understand how the attackers, how the criminals that make up the global cybercrime economy are operating. that when, so our customers can understand how can they

most effectively allocate their limited resources to the most common things that these attackers try to do. One of the most attractive parts of APIs as attack targets is the fact that that's where the data is. So when you go to exchange data between organizations, most of the time right now you're doing that via these interfaces, application programming interfaces. These are just interfaces to the backend systems and data. And what makes them doubly attractive

is that they're often unmonitored. So it's a data-rich target that's often unmonitored, which makes it very attractive for attackers. Now, it's also pretty easy to put that monitoring and governance in place if you know how and you know where to look. And this is where companies like F5, it doesn't have to be F5, but companies like F5 can provide that visibility as well as control so you can close those blind spots and just understand where your data is going.

But the attackers are always going to seek the path of least resistance. so companies that don't understand what interfaces they're exposing or what data is being exfiltrated through those interfaces, they're very attractive targets. And the word gets out on the underground. The global cybercrime economy is the third largest economy in the world after the US and China. cybercrime is bigger than the global drug trade, counterfeiting, human trafficking, all put together. It's roughly $10 trillion a year.

And these are professionals, these are businesses. When they find weak spots, they make it profitable for themselves. So the easier you make it for them, the higher the return on their effort invested. And it just feeds the economy more and more and more. So they wouldn't do it if they didn't make money at it. a lot of companies are making it, unfortunately, pretty easy.

Jon Scheele (13:35)

Yeah, so a couple of those reasons are APIs, public APIs are exposed to the internet so they have the same exposure as other traffic. To other points, once, and because APIs often expose business process or they act.

enact business processes, attacking the API can affect more than just one person at a time or one application at a time. And then the other aspect is that, and some of these business processes are quite complex, they're payment transactions. They're moving, they're sharing data from one company to another. And the

And then there are large amounts of data. if a breach does occur, the attacker is often able to extract much larger amounts of data than they would have otherwise. And even if something is, traffic is monitored, not necessarily, the behavior isn't necessarily monitored. So.

so well because APIs are a high volume interface. are hundreds or thousands of transactions or millions of transactions going through these every day. So an attacker that is extracting some data through a vulnerability is not as easily noticed if you're not careful because it just appears like normal traffic.

Chuck Herrin (15:08)

Spot on. Absolutely right. Yeah, if you expose an API for the purpose of data exchange, and then you make a shocked Pikachu face when data gets exchanged, like you expose this interface specifically to exchange data, the challenge is that's often where the monitoring stops. And unlike many attack types, so something that I've seen, I'd be curious to get your thoughts on, your input on.

the priority of API and application security, and by the way, I do combine those intentionally because I really don't think you can separate API security from application security as a separate discipline, but we can dig into that more in a second, is that perception of relative priority for API security varies a lot by region. And I think some of that has to do with...

Other attacks like the phishing calls, email phishing, scams, these things, these require user interaction. So they're apparent that they're happening. People are aware of them because they get the calls. APIs require no human interaction. The attacks for APIs and the sort of hidden attack surfaces going on monitor, they don't require anybody to send you a message that you respond to.

They happen all the time under the radar and that's what makes them, I it falls literally under the radar. Companies are unaware of the attacks that are taking place because they aren't monitoring and because there's no user or administrative interaction required, it's easy for them to stay hidden.

Jon Scheele (16:27)

Yeah.

And I guess because some of these business flows are complex, then there can be a chain of vulnerabilities. So a...

A system that is not protected through authentication and authorization can be a pathway into other systems. The same concept that we talk about in application security of Xero's trust architecture is equally applicable to APIs because once an attacker is in, if you haven't locked down all the other

parts of your infrastructure, your technology stack, then the attacker can hop from one application to another and continue to scoop up data or to change it and to poison it.

Chuck Herrin (17:33)

Or to invoke function. mean APIs are largely about data transfer, but they're not limited to that, right? You can create, read, update, delete, or invoke functions of industrial control systems and water treatment plants and nuclear facilities and missile defense systems and all kinds of things via APIs. So yeah, that's why I really don't think that in five years,

I don't really think that we're going to see a lot of standalone API security companies, maybe a couple of specialty firms that really manage some niche space within the ecosystem. But modern APIs or modern app security is really just API security. have entire architectural patterns like the Jamstack. The entire app is JavaScript, APIs, markup language, JAM. The app is the API or the combination of APIs.

The part of the challenge though that, you know, the reason that we spend as much time on the road and doing appearances like this that, we do is that so many defenders really aren't even aware of this attack surface in their environment yet. I had a conversation with a long time friend and acquaintance. he and I worked together a long time ago, you know, a hundred years ago. And, and I asked him, you know, what are you doing about your APIs? And this is a guy in financial services and I'll, I'll reveal more, but he said like,

Yeah, we don't really have any. been thinking about it. And I asked him, we'll say his name was Bill. said, Bill, you know, I held up my phone. You've got three mobile apps that I know of. How do you think they work? But in a lot of cases, security teams don't necessarily understand how the applications under their responsibility operate or how pervasive the uses of APIs are. And so a recent case study with a large bank, not a US bank, but it'd been in the top 50 or so, by assets. We showed them how we could go directly to their backend API that was responsible for money movement and currency conversions. And by bypassing client-side controls that they expected to work, and they did work in their QA testing on the mobile app and the web app, we're able to print money undetected, which is when we reconstruct the attack and walk through it for the customer,

It was one where we had valid username, valid password, a valid token that we, you know, obtained as part of a session cookie. We made these requests to the backend API. The API responded exactly as it was designed to do. And it was a classic salami slicing attack. We're slicing off, you know, fractions of a penny. And then we did it a couple of hundred million times. That type of attack isn't new. The only thing that changed was the interface by which the bank made that attack surface available. And they didn't understand that by bypassing

the controls they expected to be in place on the client systems, that we could go directly to the back end API. And if I've got a valid username, a valid password, and I'm making requests and the API is responding as it's designed, yes, I'm stealing money, but what's the heck here? Right? I didn't ransom your files back to you. I'm using a system you exposed. And I'm not rationalizing. Obviously, we're stealing money, and we wouldn't do that without being sanctioned, approved to do it.

Jon Scheele (20:44)

We will just open.

Chuck Herrin (20:54)

But it's not hacking in the traditional sense that a lot of defenders are thinking of it. You exposed business logic that I'm abusing. And we see that across various industries, airlines, frequent flyer programs, retailers. It's usually gift cards, utilities. It's often the customer login endpoints that these companies are always looking for a vulnerability, always looking for a way in. And it's pretty easy to exploit.

In no small part, because API attacks don't necessarily follow the classic sort of Lockheed Martin kill chain that the cyber defense industry typically uses, you know, this multi-step kill chain. APIs are stateless. If I've got a token that gives me access to a particular object or a function, I make a single request and I get it. It's not so much a kill chain, it is a kill shot, you know. And lot of APIs, you know, forgive the Texas expression here, but it can be like hunting a dairy cow with a rifle in scope, you know, it's just...

Jon Scheele (21:45)

Mm.

Chuck Herrin (21:52)

Boom. And all the data.

Jon Scheele (21:54)

So for organizations to protect themselves, they've already established a cyber security team.

Do they need a separate API security team or do they need to embed the skills, the API security skills within their existing cyber security team?

Chuck Herrin (22:12)

That's a great question. I think it's really up to the division, but the fundamental or core sort of principle that I would reference here is you cannot secure that which you don't understand. So whether you wind up embedding in sort of like a squad operating model, a security engineer with your application development teams, or you stand up a function or expand the remit of a function that's responsible for application security to better understand these API vulnerabilities,

It is critical that somebody within the team understand how software is being developed. And then I don't think it's so much like you have to stand up an API security program as much as you need to ensure that your security program covers all of your interfaces. When we think of threat modeling, there's only four things you need to know, assets, actors, interfaces, and actions. And if your security program doesn't take into account all of your interfaces, you have an incomplete threat model. You have an incomplete program.

You wouldn't necessarily stand up a mobile app security program unless your business literally lives and dies by the mobile app, in which case maybe that's appropriate. But if you're exposing mobile apps, publishing mobile apps to the outside, someone in your security organization needs to understand the threat model and the risks of that mobile app. You chose that architecture. And so it's your responsibility to then protect it. How the companies do that is really up to them and up to their business model. What's most appropriate?

Companies whose lifeblood is APIs, like Spotify, for example, they should probably have a dedicated API security team. And I'm not speaking for Spotify or I don't know how they run their shop, but if it's a core to your business, then yeah, you probably need some dedicated resources. If it's just another avenue, you know, just another set of interfaces, just familiarity and visibility may be all you need. But one thing we do know is if you're not monitoring it, you're not managing it. Right.

If it's a blind spot, it's just pure risk. There's no business reward for taking the risk that you are.

Jon Scheele (24:11)

So speaking of monitoring then, a lot of companies aren't all that sure how many APIs they have exposed because for a mobile application or a web application or anything else that any application that is directly visible from the World Wide Web, we tend to know where they are.

in our network because there's typically an ingress point for the internet itself. API gateways have been established to help route and protect traffic, API traffic, but it's relatively simple to expose an API without going through a gateway.

It may go through the internet, but that traffic may not be noticed among the sea of other traffic because it looks like a lot of other traffic. simply understanding where all your APIs are is a new challenge, isn't it?

Chuck Herrin (25:29)

It is.

Jon Scheele (25:29)

It's not like you can just go to the team that manages the gateway and says, what are all the APIs? Because someone in the organization may have exposed an API that doesn't go through the gateway. So what do organizations do about that?

Chuck Herrin (25:44)

Yeah, 100%, John. And that's so foundational to getting your arms around your API ecosystem is discovery. in the modern enterprise, one of the things that makes it harder is the complexity of the modern operating environment. Something like 38 % of F5's customers, and it's not unique to F5 customers, but something like 38 % are in six or more operating environments, meaning on premises, public cloud, private cloud, multiple clouds.

10 years ago or 15 years ago, we all said we're going to the cloud as if it was a cloud. And now almost everybody is in almost every cloud. So what we've learned over the years trying to solve for these problems is there's no one perfect lens that can give you visibility into all of your APIs. Because you've got internal East-West APIs, you've got things that don't run across load balancers, you've got APIs that are listening but not active at any given time.

Right? So very commonly we see things like version six of an API, you you see that in the path exposed to the outside world. That's the one that you want your customers to use or your partners to use. version five is still available. Version four is still available. Version three is still available. And even if you're monitoring traffic, you don't see it normally because you're not using it to conduct business. So the only time you see it is actually when an attacker or somebody doing reconnaissance,

figures out that you've got an old version and listening, and then your perimeter becomes your first line of defense instead of your last line of defense. There's also, we can do discovery via your code base, which is something that F5 acquired with Wib and is bringing to market this quarter. So you can do discovery in your code base, but even that lens only tells you about APIs that are part of your code base, that your internal teams have created, right? So third party software, wouldn't necessarily see it there.

And even monitoring your code and your traffic, your mobile apps and your web apps probably call APIs from other providers, right? Third party APIs could be Salesforce, could be Google Maps. It could be any, you know, hundreds and hundreds and hundreds of options. And those things are probably not running through your traffic or your code base. So what we have developed is a multi-lens system that F5 purchased from Wib and is bringing to market as part of F5's API security offering.

where we'll have visibility into the code for discovery, traffic for discovery and protection, because that's really where the attacks happen, is in the runtime environment. And F5 being full proxy can provide a lot of very robust protection, including schema validation, effectively creating what you might call an API firewall. We don't market it that way. But essentially, publishing a schema that then F5 enforces full proxy, as well as visibility from the outside to find your third-party APIs and any unexpected third-party APIs.

that you may not be aware of. Because there is no one perfect lens, we have to create the methodology to put those three lenses together and create this overall inventory. And then once you have the overall inventory, you can gather all kinds of telemetry about what data is going back and forth over these endpoints, what's the sensitivity level of that data, which of these are relevant for things like PCI, GDPR, for compliance, for privacy. Because the compliance requirements follow the data, when you can see the data,

you're able to say, this looks like in the US a social security number or a credit card number. And we know that we're not allowed to store sensitive authentication information around credit cards. So you can exclude that and so forth and so on. But it all starts with putting as many lenses together as you can to get an accurate understanding of your ecosystem and then gathering data about those interfaces so that you can quantify and assign resources based on risk.

And what we help our customers do is as we go through these visibility exercises and we see where the data is coming in and out, what types of data it is, then we overlay that with information, for example, from F5 Labs, where we say, okay, well, if you're a telecom company or you're an ISP or you're an airline, these are where we see most of the attacks happening against companies like you in your regions. And so we provide that threat intelligence to further hone the risk lens.

so we can help our customers decide what do they want to do next, what do now, what do they want to do next, and what do they want to do later, right? Because it really all does boil down to allocation of limited resources. Our customers don't have extra security resources. But starting with all that visibility gives us all the information we need to then help them say, is what I need to do immediately. Sometimes we found we'll be doing a visibility assessment or something, and, OK, you're actually under attack right now.

There's a credential stuffing attack going on and here's what we recommend that you do about it But you can't do anything without looking first so getting that visibility around it is not a Trivial, you know sort of problem to solve but we've got a lot of experience putting lenses together to do exactly just that we've been working on this problem for several years

Jon Scheele (30:54)

So you mentioned third party APIs, which I recall that's a relatively new entry into the OWASP top 10 for API security. I forget what is called exactly. I think it's something like unsafe use of APIs, unsafe consumption of APIs. Could you pick that apart a little bit? Because I think that when we access another service,

Chuck Herrin (31:09)

Yeah, consumption of APIs, yeah.

Jon Scheele (31:20)

we just sort of assume that that other party has got it right. We might've vetted them at the start, but once we start using them, we just assume that it's always going to be okay. What do you see organizations, leading organizations doing to ensure that they're not falling for that trap of assuming everything is okay?

Chuck Herrin (31:43)

Yeah, that's a great question. And I think also a lot of organizations sort of, they build on that assumption with, well, I'm sure it's covered in our contract or something, you know, and maybe it is, maybe it's not. I think like everything else, starts with visibility, but I'll give you a real example from a digital bank that I built here in Dallas. I had a relationship with a key vendor.

to do something and with respect to the US housing market I'll leave the name of the company out of it, but they exposed something like 30,000 endpoints to their customers and with the products that we bought from them it was 6,000 so we had 6,000 API endpoints for us to pick and choose from that they exposed to the internet for us to consume and our data

was behind these. And I went to the company and I said, look, I understand your flexibility and I really appreciate how flexible you guys are, but you're exposing 6,000 endpoints and I only need like 28. Our workflow only is gonna call just a couple of dozen of these.

I don't want the other 5,972 or whatever the actual number was exposed to the outside world because my data is sitting behind those things. Third party risk management is very difficult, but like everything else, starts with visibility and awareness. Do you know what third party APIs your applications are interacting with? Do you know what data is going back and forth to them? Do you know how sensitive it is? And do you know what other end points these companies may be exposing?

where your data resides behind it. You've got to start with visibility and understand. then sometimes, unless you want to try to proxy that SaaS provider's traffic through your infrastructure, then you do have to go into contract discussions and liability discussions and visibility and that type of thing. Because supply chain risk is a real issue.

But it's much easier to manage. the only way you can manage it starts with visibility and understanding of where that traffic's going and what data is involved. So like everything else, the underpinning is visibility and inventory. But some things you just can't fix in your own shop. Some things you have to deal with the third parties that you're working with.

Jon Scheele (33:57)

So it wouldn't be a discussion in 2024 without discussing AI. There are lots of opportunities with generative AI and all the, what I would call traditional AI, like predictive analytics and things like that. But there are also new vulnerabilities that come into that. And I think it's a little related to consumption also.

Chuck Herrin (34:06)

course.

Jon Scheele (34:27)

because we don't necessarily know what data a model has been trained on. We don't know what other sources it may be drawing on, may be drawing, calling APIs to get some data from some other place that we don't know. What are you seeing organizations starting to do to address that? Are they treating it as something separate or is it also something that a knowledge of application security and API security can assist?

Chuck Herrin (35:03)

It's a really good question. Actually, we published a state of application security report earlier this year. And one of the main findings from that report was that companies that are positioned with the operating model and sort of the capabilities to manage modern distributed applications, so hybrid apps, things that run portions that may be in a cloud like Amazon or Microsoft or Google, other.

pieces that maybe run on premises, they follow the data where it goes or so forth. Companies that have those routines and that muscle memory are materially more ready to manage AI workloads than companies that don't, for example, have an SRE type discipline, site reliability engineer type discipline. And so there's a direct correlation between the maturity of the technology organization and how ready they are to deal with AI risks.

And going back to one of our earlier points in the conversation, your architecture drives your attack surface. So when you start exposing LLMs to your point about this being the new AI, generative AI specifically is what we're talking about, as compared to deep learning and other types of machine learning that we've been using for the last couple of decades, it also exposes novel attack surface. And the type of generative AI that you expose,

and the risks vary. So you have a different set of risks and sort of things that you need to threat model out if you have a chat bot by itself, as opposed to exposing something that's using like retrieval, augmented generation or react or what's coming online now with agentic AI and AI agents. Now you have to worry about different types of things like loss of control of those agents. And we've seen multiple, multiple case studies in the wild. I'll take one from banking.

from not too long ago, there was a pilot of customer service bots. And these bots were graded on their ability to close customer tickets quickly, close them timely, high satisfaction ratings, so forth and so on. And the bots, though, didn't have the concept that you do in banking of separation of duties. So there's a very common separation of duties in banking where someone will open an account

and then someone else within the bank will actually handle money transfers, for example. Yeah, exactly, exactly. Well, the bots figured out that they could reward seek and get better results where they knew they're being measured if they just gave each other the privileges to do everything. And this reward seeking is a new type of challenge with these AI models that legacy security programs won't have thought of.

Jon Scheele (37:27)

The Maker Checker rule, yeah.

Chuck Herrin (37:50)

Other types of novel attacks against machine learning systems are things like transfer attacks. If I have unfettered access to your APIs that you use to train or get inference from your model, I can query your APIs, especially around the boundaries. So like, what's the best way to describe a circle? Describe everything that's not a circle? Like there are ways that you can extract the information. And if I have inputs and then I can capture the corresponding outputs, well, that's training data. Now I can train a surrogate model,

and then create white box attacks or transparent attacks that I see against my surrogate that then when replayed will work against your original model and a function of machine learning vulnerabilities and transfer attacks is that it'll also typically work, not always, but often work against other models that are trained either on similar data sets or trained for similar use cases. So now you can potentially create new attacks that work against multiple types of AI agents or multiple types of chat bots or multiple types across multiple organizations. This is very much an emerging attack surface that there's an average of something like two academic papers per day published on AI and AI security. And if you don't have the basics buttoned up yet with respect to your APIs, for example, visibility, well, that's how these models work.

There is no AI security, excuse me, I should say, without securing the APIs that serve the models, that serve the AI. The APIs are how we train the models, they're how we use the models, that's how we attack the models, that's how we steal data from the models. It's all through these interfaces. And so companies that are ready for that are in a position to really unlock a lot of business value.

Jon Scheele (39:26)

you

Chuck Herrin (39:33)

companies that get out way far over their skis and start exposing even more data without monitoring the interfaces that serve these models are setting themselves up for what could be a pretty bad time. And I hope that can be the case.

Jon Scheele (39:46)

Yeah, and I mentioned in a previous discussion at last week's OSP Habs Day in Singapore, one of the presenters, Pedram Ayyad, he was talking about AI war games that he runs where each team is given a secret that they need to embed in their chat bot that's connected to an LLM and other teams

are going to try and extract that secret from the chatbot and they have to defend that and also seek to get the secret of the other teams. And some of the techniques relate to, as you say, the best way to describe a circle is to talk about everything that's not a circle because directly asking for a secret will probably get a response, I can't tell you that. But...

using other methods to trip the chatbot into releasing information that it had not intended to because it didn't strictly regard it as the secret is something that is a concern for whoever trains the models and any chatbot or other

agent that is is accessing the that model and also If it's contained going outside an organization Then what happens at the at the perimeter at the point of egress? Are you? actively checking to for for any data leakage Through through your through your systems to do someone who's not authorized

to receive it. it's already a complicated world. I think it's going to get more complicated and we're going to need better tools and processes to address the vulnerabilities that are going to come as we start to rely on these and seek the benefits of these systems even more.

Chuck Herrin (42:00)

Yeah, completely agree. And the only thing I would seek to remind the audience is that these are just the most modern of modern applications. So while they expose a different attack surface, defense in depth still matters. The same security fundamentals still matter. Using threat intelligence, for example, to keep the bad guys away from your models in the same way that you keep the bad guys away from your API endpoints or the bad guys away from your web apps still matters.

Basic things like rate limiting and looking for data exfiltration. What we can't do is over rely on trying to secure the model themselves and hope to make the model a complete ecosystem in and of itself. All of the relevant defense in depth fundamentals, bot, WAF, DDoS, signals intel.

You know, all of these things are still relevant. We just have new attacks that we need to worry about. And that's why the industry is coming forward with things like AI gateways to monitor ingress and egress and specific, you know, prompt sanitization and things to deal with this novel attack service. But all the other fundamentals still apply. We still need defense in depth.

Jon Scheele (43:04)

Great. Well, thanks very much, Chuck. It's been great talking with you. I hadn't realized we'd have such a deep conversation about different elements of web application and API security, but it's been very revealing and I appreciate it.

Chuck Herrin (43:20)

Well, John, thanks for having me and I'm looking forward to being down in ANZ here in the next week or so. So I hope to see you there.

Jon Scheele (43:26)

That's at API Days Australia in October. Okay. All right. Great. Thanks, Chuck.

Chuck Herrin (43:29)

That's right.

Thank you very much.

​

Start with the customer – find out what they want and give it to them.