LIVE from Black Hat Las Vegas: AI, Nation-States, and the Battlefield That Keeps Changing

Episode 14 Transcript: LIVE from Black Hat Las Vegas: AI, Nation-States, and the Battlefield That Keeps Changing

Guests: Caroline Wong (Chief Strategy Officer, Axari; author of The AI Cybersecurity Handbook), Allie Mellen (Principal Analyst, Forrester; author of Code War: How Nations Hack, Spy, and Shape the Digital Battlefield)

‍Host: Mo Sadek

‍Also featuring: Madi Vorbrich (producer), live on mic for this special episode

‍Recorded live at the Play Playground, Las Vegas, during Black Hat 2026

Mo: Hello and welcome everybody, including the people over there because we have a full audience. Welcome to the podcast, but we're doing it live. So hello, internet friends. Some of you may have listened, some of you may have not. Some of you are probably from Luma, but all of you are from Black Hat or B-Sides. So thank you for joining us today on Curiouser and Curiouser. Super excited. We've got a bunch of people, some you've never seen before, some you may have. For the first time ever, producer Maddie in front of the camera and not behind, not the hidden voice and face. And then we are joined by our two friends, Caroline and Allie. So I'm going to let you introduce yourselves if you've never heard, I always skewer the introductions, so I'm just going to let you all introduce yourselves as the awesome authors you are. So.

Caroline: My name is Caroline Wong. I've been in the cybersecurity industry for about 20 years now, about two thirds of the time on the vendor side, about a third of the time on the in-house security team side. Right now, I've co-founded a company called Axari, and we're building digital twins for CISOs and for their teams.

Allie: And I'm Allie Mellon. I'm a principal analyst at Forrester Research. And I got my start in technology as a hacker before becoming a security practitioner and then joining Forrester. So I've seen the industry from a couple of different angles. And I'm also the author of this book, Code War, How Nations Hacked By and Shaped the Digital Battlefield.

Mo: Yeah. And that, guess, moves us to where we're at. So Allie's introduced her book, but both of these amazing humans are authors that have published books this year on stuff that we're really interested in. So Allie wrote Code Wars and we're going to ask a little bit about your book, but Caroline, you also wrote a book. Tell us a little bit about that too.

Caroline: Well, I feel like being in cybersecurity in the AI era is like a weird thing. And I think that we are seeing in near real time how AI is changing everything for attackers and for defenders. And so that's really how the book is structured. A lot of it is how is AI helping attackers? I talk a lot about deepfakes, for example. And then a lot of it is also about, okay, well, how can we as defenders use AI? And so there's a chapter on supply chain security, and it's been so much fun to learn all about this topic and then be in a position where I can share what I've learned with others.

Madi: And Caroline, I just want to open up with one of my favorite quotes from your book, actually, because I feel like it's really fitting given the nature of the podcast being called Curiouser and Curiouser. It opens up in the beginning saying, because in a world where technology evolves faster than policy, curiosity is the most powerful security control we have. And I think that just really opens up our conversation perfectly. So let's dive right in.

Mo: All right, cool. So one of the first questions that I have, at least reading both of the books, they're coming from two different angles and they don't necessarily mean to speak about the same thing, but it's almost as if you both wrote the book in the same library and you maybe heard the other person, but they converge in a really interesting place. So I think maybe what inspired each of you to write a part of your book or... What kind of brought you to the conclusions that you kind of reached in your books?

Allie: So I was inspired to write my book because I've worked with a lot of threat intelligence teams over the years. And one of the things that I've found working with them is that they always talk about, especially when they're releasing threat research, how if you understand the history of the nations that are performing the cyber operations, then you can really get to the root of what's going on in. And the problem that I saw was that while they talk some of the history in their blog posts about the research, they couldn't get into the nitty gritty, into the weeds about exactly why these nations made the decisions that they did based on the history. So what code war does is it goes into China, Russia, and the U.S. and does historical vignettes from many, many years ago. We're talking like Tsarist Russia, Imperial China, to compare those types of situations to why they made the decisions that they do today from a cyber attack perspective. Because it's really fascinating. There's a ton of overlap in the historical precedent that's been set by these nations and the reasons that they take the actions they do today.

Caroline: I think for me, I'm just like always thinking about security teams. You know, every security team that I know and that I have known, they never have enough people. They never have enough money. And so I really wanted to explore the idea of how does AI change that? At the same time, we are in a moment where AI and everything, cybersecurity seems to be changing so incredibly quickly. I wanted to kind of get out like a foundational primer so that folks could feel like they have their feet underneath them and understand at a foundational level, like what's going on so that we can be able to interpret things that are happening every single day. One of the funny things about the book is I finished writing it in September of 2025. And then when it came out in March of 2026, it was already out of date. So I do have like a companion website, AICyberhandbook.com. And that's where I continue to publish about just more things that I'm learning and seeing as time goes on. And I think that we have to give a shout out to Jim Minitel, who is our really fabulous editor and publisher. Thanks, Jim. We love you, Jim.

AI Lowers the Cost of Nation-State Attacks

Mo: So one of the things that you both have probably heard at least seen is that the cost of nation state attacks are actually going way down because of AI, right? And I think it's something that every conversation I've either had with a CSO or security team or threat intelligence team or a SOC, it's like we have a lot more noise coming at us, right? It's just so much easier for attackers to kind of pull this off. So I guess Ali, one of the first things, right? So nation state attacks are usually never random. So when this happens, like they're usually going for a real world goal. And I think it would be interesting to kind of walk through some of the mentality just so that like, think organizations, like they need to understand like when a nation state attack is actually in their threat model. And then from that, like how are enterprises supposed to scale at this size of attack and when it becomes easier and cheaper to do that.

Allie: Yeah, it's really interesting because it used to be a bit more static than it is today, right? One of the bigger problems right now, especially as the geopolitical threat landscape is so chaotic. We don't know what nation's going to attack another nation on a daily basis. And so that causes the threat model to change significantly on a regular basis. And so one of the things that I've seen pretty consistently is that, especially on large security teams, they're updating their threat model. They're looking at a geopolitical risk standpoint, even if they don't have a geopolitical risk team that's dedicated to that, they're still looking at it on a regular basis. We see at least quarterly, sometimes monthly, sometimes even more frequently, because if you don't, then you could get into a situation like we saw with Iran and the Handala hackers, where you become a target, even if you weren't in the weeks and months prior, you become one because of the changing geopolitical landscape. And by the time that you understand that you've already been hit. And so it's really critical to start as early and often as possible with evaluating the existing geopolitical landscape outside of cybersecurity and then apply the potential motivation of the attackers based around that and to make decisions based around that.

Now AI introduces an entirely different layer that's really interesting here because the attacks that they're able to perpetrate are not just much faster, but they're much more dynamic. We're not talking more static malware anymore where you have to do reconnaissance and understand exactly what operating system is running and what vulnerabilities are on that operating system. Now an attacker can go and just tell AI understand what's in the environment and build an exploit based upon that. So it's much more dynamic than it was before. And that's starting with the nation states, but it's not going to end with the nation states. We see nation state attackers that are currently using AI to automate as much of the attack as possible, very similar to what we saw with OpenAI and Hugging Face. But what we're going to continue to see is that the nation states, much like they have with more traditional malware, they're going to trickle down those AI agents and those AI-based attacks to the cyber criminal community so that they can prevent us as defenders from understanding their modus operandi and from basically trying to attribute it to them.

Mo: Yeah, so another part of that, again, like security teams are needing to catch up to a lot of this. Like Ali just mentioned, the cost to iterate across different vulnerabilities in zero days is so much easier. I guess like what, like you are especially in a place where I think you're automating a really interesting part of the security stack, right? We've seen a lot of like SOC and like SIM automation, but you're actually looking at like the people. And how to kind of empower them and get them to like force multiply through intelligence. So what does that kind of like look like? Like what does that signal look like when you're trying to automate something at that level and kind of make the human a little bit more efficient?

Caroline: You know, I have to say like I am and have always been kind of wildly optimistic about things. When I look at AI, I'm like, okay, if I think about my day and the type of things that I enjoy doing in my job and the type of things that I would kind of rather not do, how can I just have AI do all of that, like expense reports, I think is like a perfect example. Like I don't know that any of us get a lot of joy out of doing expense reports. You know, what we're building in terms of our digital twins is if I can just sort of like say, hey, Caroline's digital twin, you've got access to everything that I do, go and do this research for me. You know, go and take a look at this control and actually compare it to my entire policy library and let me know what that's looking like. Hey, here's a new piece of information. How does this change what my risk register says? Oh, one of the vendors that I've been working with, they actually just had a change. How does that affect my third party vendor risk management?

So I think what I'm seeing is security teams were made of humans and then we were using legacy security technology. I think that the future that's coming at us fast and furious is security teams who then have to figure out how to do risk management for all the AI agents, how to use AI agents to help us with that. And I think the pace and the speed is something that while security teams today can keep up with the human speed of innovation, I think that changes entirely when we get to a point where it's actually, how do you keep up with machine speed? I think that you could easily ask any security leader in Vegas this week, and you could say, hey, do you know how many agents are in your environment and what they're doing? And I think it would be a challenging question to answer.

Mo: Yeah, just to build off of that, again, like you said, there's a new system every day, a bunch of models, agents running around like crazy right now, like it's the Wild West. So with all the new models and the new prompts and the new techniques, what do tested and safe even mean now?

Caroline: I think it's really important for us to just go back to basics. You know, 20 years ago, data classification was really important. Today, data classification remains really important. Data leakage prevention was really important, is still really important. Having an up-to-date inventory, you know, and so I think that actually some of these things that may not have been the sexiest in the past, like I don't know that data classification was ever sexy, but now in the age of AI, it's super important. And so I think it's really important for security teams to just focus on the fundamentals. You know, ideally if we're able to know what our attack surface looks like. And I think AI can help us tremendously in terms of OSINT and keeping up to date with a continuously changing attack surface. And then we just have to know, how are we vulnerable? How can we address those vulnerabilities? Risk management as a concept and as a practice has been the same throughout the ages. And I don't think that this moment is any different.

When Your AI Model Provider Goes Down

Mo: Yeah, so one of the things that I think about when we talk about enterprise adoption of like AI systems and new models and just moving fast, we have like these two categories that we're really focused in, right? We have these frontier models that are provided by these really large providers. These companies that are, know, names that we all recognize. See Opus or like Fable or you see GPT-5 and... We have all these kind of household names, and Gemini, sorry, can't leave them out. But at the same time, they've become kind of like a really big threat surface. If you look at just a couple of months ago, we saw one of these models completely just shut down, right? Because of a policy. So I'm wondering, this creates a gap and kind of a dependency in all organizations where it's like, what do we do when this model's down? All these services are affected, all these different products that we have are impacted, our customers may be impacted, it creates an availability crisis. So I think, like kind of what does the model dependency like landscape look like? I think from both sides, it would be really interesting.

Allie: Yeah, I mean, the main problem is it's it was never going to be one model to rule them off, right? We have to have experience with and get used to using multiple models. That's the reality of the situation. That's every enterprise in the world is doing that in their own way, whether it's for specific applications or just because it's preferences of different developers. The thing that I will say is like, safety guardrails are our friend, but they're also our enemy. And what showed us that truly was the open AI hugging face incident because Hugging Face was in a situation where they didn't have access to the models that they needed at the time that they needed them to do the analysis of the data that was coming in from the attack that was perpetrated by OpenAI. They needed a model that didn't have those safety guardrails or that at least had them lowered to the extent that they could actually interpret the data coming in. And that was for good reason, right? It's not like the safety guard rails aren't there for a good reason, which is to prevent a hack from being done with that model. But the reality is there needs to be a balance. And a part of that is having a backup model in place, like an open weight model in the case of Hug Me Face that they were using to do that data analysis and interpretation. That requires a lot of things on the backend from actually being able to record that data to accessing that data to then having a backup model if you need it. But, to me, it's one of the most important things that we can all do is to make sure that we're experimenting with models, not just the ones from the frontier model providers, but also some of the open weight models so that we have a backup in place in case and when this does happen.

Caroline: Yeah, I think this just is... It's a foundational concept, which is resilience. I don't know that disaster recovery was ever super sexy, but all of a sudden now it absolutely is. Disaster recovery and business continuity 20 years ago looks like, oh, we don't want all of our company executives flying on the same airplane. What could happen? And these days we simply have to ask ourselves, and this is where I think incident response tabletop exercises really come in handy and we just say, hey, like what dependency do we have on anything, whether it's a foundational model or not, and what is simply our backup plan if that's not available? And these are the things that, you know, maybe they're not like the funnest and they don't feel like the most urgent questions to ask, but they are fundamentally important.

Mo: So going off of that, right? Let's like pick something like really specific because you mentioned the hugging face thing and I think that incident actually showed us a lot. But we also have on top of that right before hugging face happened in June, I think there was another incident where we had the model providers actually be restricted internationally as well. So I think at that moment, I don't know if anybody else kind of remembers that, but everyone was really disappointed when there was no access to Fable. And they were like, oh my gosh, we have this really powerful model, but now there's no access to it, right? Fast forward a couple weeks later, we had open source models that were performing at par with some of the frontier models that the US was providing. Now, when you look at that, as like, I'm guessing an organization leader would say, well, if we can go and bring this back in-house, why would we like have this dependency that can go and disappear at a moment's notice, right? So like we say, we go to backup models, we have these kind of places to go. At the same time, I think there's also this risk, and this is a little bit more going into like American dynamism and like, how do we like make sure that we are like the best, right? We do have like competition in the open source and open weight model space, where organizations can go and take these models from companies outside of the US that are pretty good in the same vein. We do understand that there is like, there is a competitive nature to this and these models do have their own biases that we end up bringing into our environments. So I guess it's a two part question. This first part I'm going to kick off to you, Ali, right? You've seen a lot in code wars, you go a lot into kind of like the geopolitical climate around models and development and how kind of a this code and this innovation is actually kind of like a weapon or even just a nation advantage, right? So it would be really interesting to understand the power dynamics behind open source models and what that means beyond freedom and beyond capability, or maybe what it means to each nation.

Allie: Yeah, it's really interesting because one of the things that I focus on a lot in the book is China's digital Silk Road, and everything that they have done to really make sure that their technology is in many different nations across the world. And they've put a lot from a government perspective, a ton of investment in making that be possible. And that goes for typical regular technology to surveillance tech. Like it is, it runs the gamut. And this is really important because there are certain situations where especially like in China or in nations that China considers to be a part of its broader global reach, where they're going to be using surveillance technology or the technology that they have access to, to get additional information and to perpetrate cyber attacks against those nations that are using that technology. And so we have to be cognizant of that as we make these decisions about what technology we're going to adopt. Now, all of that is very dependent on the access that the providers have to that technology. So in those cases, you could use a open weight model. Maybe it's provided by a Chinese company, but it doesn't have access directly to a cloud service back to, back to China and back to the provider. That's one option there. I do think that the challenge here is that there are biases that are introduced in each model that we don't fully understand all of the time, right? That changed the output that the model is going to give you based upon the context of who you are, who you associate with and anything that it knows about you. And so it's a tricky situation because we need to balance, of course, using a lot of these models, understanding a lot of these models while also protecting our own interests and our own data. And like one of the things that we saw, especially when like DeepSeek first came out is that there were a lot of organizations that had banned it and it did not matter. People were using it regardless, because they don't understand the geopolitical implications or they just don't care about the geopolitical implications, which is an issue in and of itself. But our job in security is to balance giving that access and allowance to innovate while also making sure that the business is secure.

Making Open Weight Models Enterprise-Safe

Mo: So Carolyn, on that, like building off of like this kind of interesting climate, right? Even recently we had like an American company release Inkling, right? So Thinking Machines came out this really great model. And one of the things that they really touted on was that it's really good as a generalist based model for organizations to kind of tune. Thinking about what Ali said and like, we have like these open source models that have biases and write these open weight models that we now need to, if we actually want to use them in an enterprise, we have to train them out. I don't think, and maybe like someone in the crowd knows this better than I do, but like, I don't think we all have like the training or at least the resources in an enterprise to train out bad behavior in open weight models, right? I think that observability stack is just like not available or like maybe the transparency requirements to actually understand how those changes are being made or alignment systems just aren't in place. So I guess what does like safe adoption of like kind of these open weight models look like? I guess how do you, how do you kind of make use of this, this amazing technology that it almost feels is like still walled off?

Caroline: I love this question. And I think that like security and convenience are always going to be sort of this tension and this trade off. And I think we're really seeing that. You know, I think that actually organizations need to kind of decide like, what do they care about? And what do they not care about? Because I think even before like we asked the question of, okay, do I have the time or the money or the expertise to go and find out these things? Do I care or do I not? And why? And I think it's really important for organizations to look at themselves and examine sort of their own crown jewels, their own criteria for what is acceptable. I think we're in a super interesting moment where we actually don't know how much AI costs. I don't think that tokens and token cost is predictable in any way. And so I think there's actually going to be a lot of reasons for organizations to decide for themselves what sorts of models are going to be acceptable or unacceptable for different use cases.

Mo: So on that, right, I think when we think about acceptable models and like how we form programs around them, I think there's also again, and we covered it a little bit briefly before, but I really want to dive into it. A CISO now has to make a decision into their own threat landscape where it's like, if I imagine like my model provider can be turned off by a government, how do I prepare my organization for these kinds of things? Having backups are great. At the same time, how do you actually calculate that cost? I think an incident response or even a tabletop exercise around this is very difficult, especially with how intertwined, not only the GPU thing and how do we provide them to companies, but also, okay, well, again, what if we lose access to these models? How do I prepare my organization for this? Especially when security controls and in order to move fast, right, in order to keep up with the speed of attackers, we're now relying on technology that may just be gone in the next minute. So I guess, where do traditional security controls like really still need to remain foundational and like owned by humans, or like processes that we can still have autonomy over? Versus like, okay, well, like, how do we like really pick the ones where we can just kind of, we are okay with losing accessing the outcome.

Caroline: I think the most important thing for any of us to keep in mind, whether we are on an information security team or in a different role at any organization, is that we can outsource the research and the work and some of the recommendations to AI. We can't outsource the accountability. At the end of the day, there's still a human or an organization who is accountable for whatever decision was made. And I also think that, you know, I've actually before I got super into sort of the topic of AI and cybersecurity, I was really interested in cybersecurity metrics and I continue to be, you know, there are these questions, like how much should we invest in cybersecurity, but ultimately that's a human decision. And I think that we have to decide at each of our own organizations, what our risk management objectives are gonna be. And if something makes a security person feel uncomfortable, then we have to find sort of an executive in the business or in the technology function who's going to be willing to sign off on that risk. I think that, you know, security can come to the table with all the recommendations we want, but at the end of the day, we've really got to be partnering with leadership in the business and in technology to figure out what's gonna be kind of the best fit, risk management decision for any given situation.

One one more like quick thing that I want to say is I've been thinking for the past couple of years about this concept of human in the loop and it almost seemed a couple of years ago like we were like oh like we're good we'll just put a human in the loop but I think today in the year 2026 in summertime there's actually this reality where if a human is getting 400 notifications a day then how effective is that human in the loop really? You know, are we just clicking yes, allow, allow, allow, allow? I do think that when we're thinking about what decisions to hold onto from a human perspective versus what do we allow the AI to do, I do think about this concept of like, is it a two way door or is it a one way door? Like once you make that decision, can you like, how easy is it to go back and turn around and go the other way or are you stuck? And so I think that one-way door decisions are really good ones to continue to have humans be in charge of.

Allie: I just want to add, I actually don't think that a model provider having their model inaccessible is as big of an issue as it might originally seem. It's very interesting. Like the vast majority of the vendors that I talked to that are building some type of agentic system or AI agents, they don't rely on one model. They have kind of like a multiplexer so that you can choose the best model for the use case. And that means that if one model goes offline, it'll just default to the secondary model that maybe it's not the best quality. It might be a little bit less higher quality, but it can still get the job done. So as we're thinking about this too, if you're relying on one model, that design pattern is going to break consistently and you need an approach that is dedicated to using multiple different models, depending on which one is going to be the best fit, both from a quality perspective, but also from a cost perspective. Cause one of the things that we saw with, with Mythos and Fable as well is that sure, it was able to find a lot of vulnerabilities, but the core difference between Mythos and the models previously was not that it was able to find vulnerabilities, but that it was able to do it at a lower false positive rate. That's the differentiating feature there. And so you could technically use a different model to achieve very similar results, you're just gonna deal with a lot more false positives. And so balancing out those aspects with like, you don't need the latest frontier model for the vast majority of things that you're going to do. You might even need like small language models or very focused models is very important in this conversation. And having those redundancies in place will help you avoid an issue even in a geopolitical scenario that's not ideal.

Caroline: I'm so glad you brought that up. I work for a startup and a vendor who's doing sort of exactly that. And while Ali brought up sort of the use case of, know, if you're kind of going for the best, you know, kind of most expensive model and then like if it's unavailable for some reason, you just sort of use the next best one. You know, I actually think there's really good business reasons on the other side, which she also mentioned, which is you don't always need to drive a Ferrari to the grocery store. You can just like take your Toyota Corolla, right? And so there is this proxy happening on the back end where it is becoming, it's so interesting because I feel like six months ago, companies were trying to decide, okay, which foundational model are we gonna go with and are we gonna do an enterprise license with so that we can have the configurations and the controls? And then now it's different and you wanna actually have this proxy that allows you to change based on use case, based on a lot of different criteria.

Mo: Yeah, I think I was just like listening to something where they basically said when we first started to pick up like AI as like a service in organizations, the first thing we did was we just wanted to adopt. So we got it into the hands of everybody. The second part of that was usage. So show of hands, were you at a company that like had a leaderboard for AI to see how many tokens you could spend as fast as you could? Anybody? Amazing. I'm so sorry. Because that costs literally like, you know, millions and millions and millions, even like half a billion, right, for someone, some poor organization. But we want, like they wanted to see more usage, right? Get as much as you could out of it. And now we're in a really interesting space where it's like, let's reduce and maximize the most that we can get out of this token economy. And like you said, you don't need to drive a Ferrari to a grocery store. There's also not a lot of trunk space, not that I would know.

Caroline: But no, no, no. But now with small language models, you take these models that use way less parameters. So maybe instead of going for a 1 trillion parameter model, you go for a 12b, or you go even smaller. We're even seeing on devices, on device models that are super small, that can run on Android phones. What would that look like if you just had a process that only did one thing and it only just called that model? I think the architecture of how AI is used in enablement functions for organizations will need to kind of change and adapt and totally agree there.

Mo: Ali, you mentioned the magical word mythos. So I'm just going to bring up this one thing and then we never have to talk about it again. It's kind of funny because a couple, like when it first started happening and all the news about Mythos was dropping, it was around the same time a report came out where like foundation models will soon have these capabilities to automate attacks, right? And they'll be able to do this really well and they'll be able to chain because reasoning is really the big mode that prevented models from doing this at first. After a couple months later, we have like the Fable 5s and the GPT 5.4s and whatnot, and they were able to do the same thing, but they were now in the hands of consumers, right? So the gap between Mythos and a consumer model is way smaller. And now you go even further to nowadays, we have open source models that are performing just as good as the frontier models that are very close to performance in these like dangerous models, right? Or these lab models. So when we think about attackers and kind of like how they now have capabilities that are really, really dangerous even when we think about it from like a consumer model perspective. Like how has that kind of changed the attack economy and maybe what the new threats will kind of start looking like or emergent threats and emergent patterns that we're gonna start seeing?

Inside the OpenAI-Hugging Face Incident

Allie: Yeah, I mean. OpenAI and Hugging Face, the attack there was a great example of this. In addition, last week Anthropic came out with a report that showed three different instances where they had been like, whoopsie, we hacked to this company, we didn't realize it. So there's a lot of problems happening right now with this because the reality is that with the right harness around these models, that's very important, you're going to be able to execute attacks that are entirely automated. If you can't now, then you're going to be able to tomorrow or the day after. But what is really important is it's not just about the model itself. You have to have the right harness around it, the right harness that gives you access to the right tools, that lets you do cost optimization around which model you're choosing to use, that is able to make better decisions about which path to take.

And something I want to highlight here too is if you haven't read the cloud security alliances read out on the hugging face incident, I highly recommend it because in that report, they go through in a lot more detail than we saw in the open AI blog or in the hugging face blog, exactly what happened and how we need to be thinking about it from a defensive standpoint. And it's really fascinating because some of the things that they pointed out were the fact that like the model is not perfect. It's not doing a great job 100% of the time. It is creating a bunch of random text at random times. It is taking actions that no human or anyone with intelligence would ever take. It is doing things that it shouldn't do because that's just a part of what the model is experiencing in that moment. And it's not going to be making the best decisions every time. And so that gives us an opening as defenders to think about this in a different way, especially from a detection and response standpoint, where we can detect that type of weird random activity much better. One of the things that the report called out is the importance of deception technology now, because deception technology will give you an idea of the attack before it's actually hitting your system. I used to be a big skeptic of deception tech. I was like, I don't care about this. I don't know that we actually need this. But now I actually think it becomes very important because it gives us an early warning and it can also be kind of a diversion from your more important infrastructure so that you can know that the attack is happening before it actually gets to what's really important in your organization.

Caroline: I remember being on that conference call with CSA and they invited Hugging Face, and you know, Hugging Face got this question like, how did you know it was AI and not a human? And they were just like, it was so obvious, like anyone would know. It was like a thousand really dumb humans doing stuff all at once. Like it was so ridiculously obvious actually that it was AI. And I think that's super interesting.

Madi: We've talked a lot about how AI can be used, but I want to get into the human thread running through both books. Caroline, your book comes back to accountability, that you can never fully hand it off. Allie, yours puts it on the reader, and on data and worldview, and what nations and companies want from you. Why did each of you land a cybersecurity and AI book on the human element?

Caroline: At the end of the day, it's like just all about the human, right? I'm in this industry because I love the humans that are in this industry. I'm in this industry because we're trying to protect the software of the humans in the world that I love. I think that right now I actually use the word frothy to describe sort of this AI and cybersecurity era that I think we're in. And I think it's like, you know, it's like popcorn popping. It's like you're in the middle of a hurricane. I mean, like this is crazy times and it can feel so uncertain. And it can feel like, you know, I don't know what's gonna happen literally five minutes from now that's just gonna like change my entire worldview all again. I don't think that it's gonna be like this for forever. Things will calm down. And at the end of the day, I think for each of us as humans, like we can just like look around and learn. And I think there's so much fun learning to be done. And I'm just like obsessed with learning. And so that's kind of where I brought it back to.

Allie: Yeah, so for me, I wrote this book so that anyone could read it. I didn't want this to be something where only technical people could read and understand and get value out of it. And what I realized about halfway through was if I didn't come up with something nice to say, this was going to be a really depressing book and people were going to be sick of it. And so the thing that stuck with me throughout writing it, however, was in many, many scenarios, whether it was a cyber attack against another nation or a cyber attack against their own nation themselves, their own people, the thing that made the difference time and time again was a person at the other end of it. We do this because it affects the real world, even if the majority of the population doesn't fully understand how it affects the real world. Everything that we do is important because it affects people and the systems that they operate in and the success that they have in the real world. And so highlighting that is just the most important thing to me that we can possibly remember as we go about our jobs, even though it takes place on the internet, it is not about the internet. It is about protecting everything that makes it possible for us to be having this conversation and for people around the world to be having the conversations that they're having.

Why It All Comes Back to Humans

Mo: Yeah, I think one of the things when I think about the human impact, it's actually something I kind of learned from working at Alice for a while. When I was first onboarded, was kind of like, it was an AI security company. So you think they're always using AI to do cool things, right? But the interesting part is that all of the data that was used to feed the guardrails product and the automated [unclear in source: "retina"], right? It was all actually gathered from human source threat intelligence data throughout the history of the company. And when you see that being put into real world use, and when you actually go and you see RFPs and stuff and whatever, we were actually doing really well. And I was kind of surprised, but someone at my company on the research team was telling me that as good as AI can get, it will never be as good as a creative human. And I think that's something I kind of took from your books. Whether it's the accountability aspect or whether it's the ingenuity of humans and what they can do, and just the optimism that is left at the end of a dark day for humans, some days it's hard to see with all the layoffs and stuff that are happening, right? Because of this inflated AI thought. But at the end of the day, seeing how humans create opportunities for humans and how we've kind of used AI to even expedite some of those opportunities in different ways in different places, it always seems like we come back to the human element. So it was really just, it was great to see and refreshing from your books. So thank you for spending time and like writing them and like, hopefully you got as much from writing them as we have gotten from reading them. And about humans, we have a couple of humans in the room. So wanted to open this up for Q&A. So if anybody has a question, let us know and we'd love for you to like ask it live. Don't all rush. I'll come to you, don't worry. You don't have to show up on the camera if you don't want to.

Audience Q&A: Open Weight Model Geopolitics

Mo: What a great question. And actually maybe I'll just repeat it for the podcast. So the question is, you know, one of the people in the audience, they were at an event yesterday and they were talking about the hugging phase in OpenAI incident. And one of the things that was talked about was Hugging Face didn't find out about the incident until five days. And so the person asking the question is asking, like, Caroline, how do you reconcile that with the fact that once they found it, it was like super duper obvious?

Caroline: And I actually don't have a great answer to that question, to be perfectly honest. I'm not familiar sort of with, you know, all the details of the incident, but I do think that what I want the industry to do and what I think we've gotten better at doing in the last decade or so is when incidents happen, companies and organizations are way more open than they used to be about it. And I think we just have to keep doing that because I think that everything that we can learn from someone else's incident becomes something that we can integrate into our own defenses.

Allie: Also, just to add onto that, if you go through the cloud security Alliance report, they talk about how the AI system, wasn't like an immediate thing where it was like, oh, we're in, we're done with this. It, the attack took place over days and there were interestingly enough, like very long dormant periods where the AI model and harness didn't do anything. It was just sitting there waiting. And so the attack technically started many days before, but it started with things like reconnaissance, which aren't necessarily going to be picked up by hugging face immediately, right? And so the steps over those days weren't necessarily enough to add up to a detection and to identification, but the attack also lasted many, many days. And by the end, when it was starting to like spawn multiple different systems to target multiple different aspects of the infrastructure, that was where they started to really see the attack. My concern and question here was really around why didn't OpenAI know that this was happening? Because there's no reason that they shouldn't be monitoring actively for whether or not their system has escaped containment and what it's doing in the real world. And so I think one of the biggest questions that some people are talking about, but there's not enough accountability around right now, is the fact that OpenAI didn't do nearly enough with the guardrails they put in place to make sure that that model could not escape and get out and take some action that was meant to achieve the evaluation that they set, but still was not something that ever should have been able to escape containment.

Mo: Does anyone else have any other questions? Anything?

Audience member: [Partially audible in source: something about a question focused on the geopolitics between model adoption from the perspectives of the US, China, and the EU.]

Allie: I'm very concerned about this because we're in a very uncertain period as to what the U.S. government is going to do. I think a worst case scenario is going to be that they choose to restrict access or try to limit or ban some of these open weight models. I'm very glad to see that a lot of cybersecurity companies have come out in support of open weight models and trying to push that that is the approach that we need to take and that we need to have that competition. But from a geopolitical perspective, it's just complete uncertainty right now. And unfortunately, there's no better answer than that because in a lot of these cases, we don't know what's happening behind the scenes and we don't have enough visibility into those decisions. And the historical precedent, which like I talk about so much in this book, at least in the U.S. right now, has gone out the window. It doesn't matter anymore what historical precedent was set. The reality is going to change day by day. So there's no real great answer around that, but I do think that it's the responsibility of everyone in this room to push for the competition that we get with open weight models and the ability to actually use and access them freely even if they are from a country that we don't necessarily see eye to eye with in all areas.

Caroline: I don't have any sort of like perfect response, but there's just a couple of comments that I want to make. One of them is I think like all people in power with money throughout history have used propaganda and AI is just like the next thing that we can use for propaganda. I think for these three that you mentioned, you know, US, China, EU, we can actually see that each of these countries has kind of a different philosophy when it comes to, for example, data privacy. And you can see ways in which, in some cases, that's advantageous, and in some cases, that's not advantageous. So great question. I don't know the answer, but it's certainly going to be interesting. And I want to have access to all the models.

Mo: And with that, think we are just about ready to wrap up. So I'm sure some of you are still curious and have questions. I would say, you know, we're going to be around for a book signing and not like it's my book, but they will be around for a book signing. I'll still be here. We have a couple of surprises for everybody in the audience. So stick around. And yeah, please enjoy some drinks, enjoy some food. And in a couple of days or a week or so, you'll be able to enjoy this episode and all its gloriousness on everywhere we launch the podcast. So Apple Music, Spotify, YouTube, all that. Carolyn, Ali, thank you so much. Maddie, thank you so much for playing this. Yeah, and I think that wraps it up. So thank you so much, everybody. Please stick around and feel free. We're in a very fun place, so enjoy yourselves.

New episodes of Curiouser & Curiouser every two weeks.