Building AI We Can Actually Trust

Building AI We Can Actually Trust

Curiouser & Curiouser, Episode 3 with Laura Powell

A lightly edited transcript. Disfluencies and false starts have been cleaned up for readability. The substance is unchanged.

Laura Powell: AI is going to drive everything in the future. Let's not pretend that's not how it's going to be. We're on the precipice of either something really meaningful or something really terrible. We have an opportunity right now to fix this problem, and that gap can still be closed. But the longer we wait to close it, the longer we wait to find creative ways to diversify the data going into these systems, the worse it's going to get, and the harder it's going to be to reconcile in the future.

Mo: If AI has ever made you stop and think, "wait, what is happening?", you're not alone. I'm Mo, and I'm a security researcher asking the same questions. On Curiouser and Curiouser, we have open conversations with experts, researchers, and leaders working at the edge of this space, talking through how AI is taking shape, what's shifting, and how the people inside the work are thinking about it as it happens. So join us and listen in as the conversation takes shape.

Meet Laura Powell

Mo: Welcome back. This week I'm super excited to be sitting here with Laura Powell, who I'm actually not going to introduce, because luckily you won't see this, but there have been at least four takes where I've tried to introduce her. She's got an incredible background, a really fun story, and so much technical knowledge. I'm going to let her introduce herself. So, Laura, please save me more embarrassment and go ahead.

Laura Powell: My name is Laura Powell. I'm Senior Director of Partnerships for LatticeFlow AI, a startup based out of Zurich, Switzerland, with employees worldwide. I bring an interesting background to this partnerships role. I'm not a traditional business development or sales person; I actually have more of a technical background. I spent many years in tech working through different roles in product and engineering, in cybersecurity, privacy, and legal risk management. I spent a couple of years building out responsible AI programs at a large tech organization. So I have lots of thoughts about AI and how you manage AI risk in business, and I'm really looking forward to our conversation today.

Wishing something would go wrong

Mo: You have this really interesting background. The first time I met you, you'd been at Indeed for a really long time, and you essentially built up the privacy program there from zero to a hundred. Now Indeed is known to have one of those top-tier programs, and you're bringing all that experience to this new role. So let me look back a bit: when you were at Indeed trying to figure out GDPR compliance, did you ever have a moment where you thought, we need to mess this up really badly first before we can actually get things done?

Laura Powell: It's funny, I'm thinking about what I'll say that my old boss would have a heart attack listening to, but yes, absolutely. We were fortunate and unfortunate at the same time that we never had any serious incidents. At Indeed we put a lot of resources into building a really robust program, but those are difficult conversations, and not everybody is bought into the idea that we should do this because it's the right thing. People say, okay, but what's the bare minimum of the right thing we really need to do? So there were definitely moments where I wished something would go horribly wrong, so people could see what happens when you don't put the right controls in place. It's a really weird space to be in, thinking, I wish something would go terribly wrong so I'd have an easier time arguing for doing the right thing. It's challenging, but I think anybody who's been in risk management will relate.

Do we actually need a major AI incident?

Mo: I know that was a left-field question with no context, but it brings me into what I want to think about today. AI adoption is super fast, and everyone is putting it somewhere, which I think is a good thing and very necessary for business enablement, empowering all sorts of business owners to keep up and compete with organizations they never could have before. At the same time, with anything, we adopt a certain amount of risk alongside the innovation. There's this paradigm in security we've been tracking for a while: the more convenient something is, the less secure it is. And things are super convenient right now.

I don't know that AI has become as insecure as we think, but I think that's because we haven't had a major AI incident yet. Almost every piece of research I've seen is theoretical, or a very niche case. Even the automated attackers coming out of AI are pretty niche, some open source, but we haven't seen it scaled and affecting organizations widespread. That's part of why in some cases we lean back and say, let's improve as things happen, but let's not slow down innovation. So the skeptic in me is like, I really wish we had a good AI incident. I don't think it's going to happen this year, but I think it should, for a lot of reasons.

Laura Powell: We're only about three weeks into the year, so there's plenty of time for that to happen. Obviously nobody wants a major catastrophic incident, but it's kind of sad that as humans, that's what it takes to get us to pay attention. There have been a lot of little incidents. One of my favorites is a car dealership that had a chatbot on its website where you could negotiate your purchase price, and somebody got the chatbot to agree to a one-dollar purchase price for a brand-new SUV, and the dealership had to honor it. People love stories like that because you're rooting for the underdog, the regular Joe off the street who got a one-dollar SUV, good for him. But it's also a really good illustration of how you can't anticipate all the potential ways things could go wrong. I think that's what we'll see with a major incident: something a developer at one point called an edge case and said we don't need to worry about, with potentially catastrophic results. Things like that might be what's necessary to get people to really start thinking about what protections, guardrails, and controls we actually have in place, and what's not being done.

Why AI governance still feels theoretical

Mo: It's been an interesting Tuesday-afternoon activity to sit in on an engineering or product meeting and hear about all the reward without any of the risk: this is going to be great, we're going to do this. Playing devil's advocate, there's always someone in the room saying we need AI governance, but so much of that is still theoretical. Yes, we should theoretically have it, but there haven't been many real examples of, this is exactly why you need it. Catastrophic AI risk is very theoretical, scenarios we can't imagine and haven't seen. You can read more about these risks in depth in a science fiction novel than you can see in practice. So, as someone I'd consider a great leader and a sounding board for this space, what do you think about the actual technical work we need to do? Are we ready to prove some of these theories and give people a way to move forward?

Laura Powell: My brain is going in five different directions, because there are a lot of thought trails off that setup. Part of me thinks about the actual risk AI systems pose to the average consumer. There's an MIT study from a while back, I think it was MIT, that said something like 95% of AI systems in development don't make it into production. Companies either don't trust the system enough, they don't feel they have a handle on the risk, or they can't get the return on their development investment. There are too many gaps or inaccuracies in performance, so they can't ship, mostly for performance reasons, sometimes for risk reasons. That's one framing.

Then you have what you're describing, where a lot of these risks are extremely esoteric and academic, described in published papers with no real-life incidents to point to. So people see that and think, conceptually I get it, but it doesn't seem practical, it doesn't seem like it'll happen tomorrow, so I don't need to worry about this highly systemic, impact-the-entire-world type of risk. Then you have your legal folks who are really worried about this and want cut-and-dried, black-and-white compliance answers from their engineering teams. That's just not realistic in AI, because so much of it is stochastic by nature, not deterministic. There's a lot of unpredictability.

And then there's the reality of business operations. So many people responsible for risk management don't actually understand the technical aspects of these systems. They don't understand why an engineer can't guarantee a 99% accuracy rate 100% of the time, but that's what legal and compliance want. So you end up with all this gray space: what risks do we actually need to care about? What's realistic? What's actually preventing us from getting a return on this system, and what's even a reasonable expectation?

Ultimately it boils down to, how do I really understand these systems, how do I know what they're supposed to be doing, and how do I know if they're doing it? In my mind, that almost always comes down to technical control and assessment. I'm not here to promote the company I work for, I'm here as Laura today, but I'm fortunate that I believe this personally, and it's also part of what we do: we take that high-level regulatory, legal, or risk-management framework, which comes in legalese, and translate it into how you actually technically assess an AI system. What assessments do you run? What perturbations do you put into the data set? What scenarios do you measure these systems on? And then, when you get a blob of data back, how do you interpret it into something meaningful that feeds back into the business and risk-management cycle?

The transparency problem

Mo: I know you said you're here as Laura, not selling the company, but let's be real: this transparency piece is the hottest thing right now. Everyone wants to know what's happening in the black box. Nobody wants a Schrodinger's cat scenario where you assume the best, but on the inside your AI is dead, or something's happening. We want to understand the outcomes and what the inputs are doing. It's this weird tech that only exists because we're building in an iterative loop, stacking layers on top of a solution we don't fully understand.

Laura Powell: It's almost like getting interest on credit card debt.

Mo: Exactly. It feels like we've got this amazing black card and we just keep swiping. There's no limit, but there's massive interest accruing, and even the minimum payment takes 30 years to pay off. We're not here to talk about what credit cards do to consumers, but we've got this AI credit card we continuously swipe. I also think of it as Jenga: you never know which piece is going to make the tower fall. Sometimes the one that looks like an easy pull is the one that collapses everything. So I wonder if there's something you've noticed that's a danger in plain sight that no one thinks of as the bad part. For me, in agentic workflows, it's always how one agent interacts with another. With more agents, there's so much more variance, and you don't know when a response starts getting weird, because it's not as traceable as watching a single chatbot go step one, step two, step three. Now you've got one agent's step one telling another agent to do its own step one, and it changes the whole tree of probabilities. You don't know which agents are influencing the others.

Agentic systems and the terminology problem

Laura Powell: I think you're right, though I don't even know that agentic systems are that hidden a risk; it feels pretty obvious. We don't have a full handle on a single LLM-based system, let alone what it looks like when you chain them together or give them highly autonomous decision-making. This is a little pet peeve of mine: people talk about "agentic systems," but there are different types. You could have a single LLM system with decision-making autonomy, connected to tools, able to take actions on its own. You could have a system that's just a string of multiple models chained together, where maybe none of them have agency, or only one does, or all of them do. There are many ways agentic systems can be built, and they present different risk profiles and require different ways of managing risk.

As an industry, we haven't even standardized how we talk about these systems. If we can't use the same terminology to refer to them, there's no way we're going to meaningfully govern them or manage their risk. Everybody gets excited, and I'm a visionary person too, I get excited about all the cool stuff you can do with AI, but that stat I mentioned, so many AI systems don't even make it into production, means we need to slow our roll as an industry. Let's get these systems working as independent units, and then maybe we can start chaining them together and figuring out really cool things. Instead everybody's full steam ahead, rushing in and not paying attention to the fact that we haven't figured out step one yet.

Trial and error in production

Mo: It's a weird time to be very excited about things. There are so many good, interesting AI models out here, and they're all getting better. I'm a huge user of AI assistance in coding, to the point where I have a whole team of agents I work with. I do the architecture and design, set the standards, and I have a little agent product manager who helps shape the product vision, and an AI architect I go back and forth with until we agree on something. Recently I started playing around with different models, and one of them, specifically for image generation, got really weird. I'm sure you've heard about it, but Grok got into a really weird issue, and I think it's a prime example of an AI incident that we're not making a big deal about, because of how much innovation is in the product itself. There's so much power behind Grok that there's a bit of validation, not from me, but from the community, that says we're going to allow these bad things to happen because it needs more data, or it needs to generate these things so it can generate more. So it feels like trial and error in production, and we're watching it happen.

Even on the compliance side: in California last year there was a case where an individual generated explicit images of people without their consent, which is illegal. Here we have something similar, maybe not as explicit, still clothed but still revealing, which is the problem. So we have an incident that's happening but isn't really being treated as an incident. Is there actually a solution? We're allowing innovation to happen, but we aren't taking the ethics and principles we need, in favor of, we'll do better one day. Loosen the safeguards, let more things happen live, and then figure out what we need to do.

Risk versus innovation is a false dichotomy

Laura Powell: It's scary and difficult, because when you stifle or try to control the creative process, that limitation can be detrimental to the output. But this isn't just somebody making art. This can have real impact on real people in negative and terrible ways. Especially as a parent, I think about the potential risk to children and people who don't even know they're at risk of the kind of stuff that can be done with AI.

There's always been this false dichotomy of risk versus innovation, and I call BS on it. I don't think it's real. I like the metaphor of rock climbing: you can go farther, go higher, and take riskier climbs if you have safety gear than if you don't. It's not stifling you from climbing higher. It may take you a little longer, but it also means you can go farther. The same is true for technical innovation. Yes, in some cases there's a lot of overhead to risk management, and you'll have to put in safeguards and controls you won't feel like implementing, and it'll feel like it's slowing you down. But in the long run it's not just ethically the right thing, it also helps the company, the organization, and the developer get farther. It gives structure to what you're doing so you can actually measure your progress and know where you're going. Otherwise it's uncontrolled, organic growth that isn't necessarily going in the right direction.

Mo: You mention rock climbing and I think about the different types of climbers. I know nothing about it, but I know there are people who climb without gear, the ones who just stick their hands in and do that crazy thing. They're the fastest, and you look at them and think, that's so cool, but I'd be horrified to do it. It feels like that's where we're at, where we said, go free-climb and let's see what happens, which is scary. So I wonder if we can enable that free-climbing a little more safely. If we think about the 80-20 principle, where you get 80% of the way and 20% is the rest, it feels like transparency is how we get to enablement, and the hard part is actually enabling transparency. So where's the 80-20 there? If we do 20% of the work, do we prevent 80% of the catastrophic outcomes, or do we need to do 80% of the work to protect against 20% of them? It feels like we haven't figured out which side we're on, and in a lot of the organizations I've spoken to, it seems like all of this work is only going to close a small gap.

The 80-20 of risk, and designing for failure

Laura Powell: I love that you brought up the 80-20 rule. Anyone who's been on my teams will hear this and know Laura's super happy right now, because it's a principle I live and die by. There are two angles. One is that, realistically, we're mostly talking about businesses meant to make money. Sometimes they'll want to do the right thing, sometimes they'll feel they have to, but all of it costs money and resources to operationalize. To get them to do it and stick with it, it has to be scalable. We have to come up with solutions that are automatable and easy for people to enact, or it just won't happen. And you don't run technical evaluations and put guardrails on systems for fun; you do it because you get data back that lets you make informed business decisions. Businesses aren't getting the vast majority of these systems into production, and not getting something into production has an opportunity cost. So if you can put controls and assessments in place in a way that lets you release something and achieve a return, that's really valuable, but you have to do it at scale and in a way your organization can actually make meaningful decisions from.

I'm a firm believer in the 80-20, and it depends on how you interpret it: 20% of the effort for 80% of the results is probably the interpretation you want in risk management. But the thing that worries me is that AI systems are built off pattern recognition and making sense of their training data, and inherently, outliers are just noise. The system generally rejects them. Now think about the fact that most catastrophic events are probably triggered by something a developer considered an edge case, an outlier the system had never encountered and didn't know how to handle, so it does something horribly wrong. So with the 80-20 approach, that small section is a little scary. You then have to ask, what's the 20% of effort for 80% of the coverage on that small section? Do you build in certain capabilities for failure modes? Literally telling the system, if you encounter something outside this set of procedural guidelines, reject it, or pass it to a human. That's what human-in-the-loop is actually meant for. But a lot of people developing these systems aren't thinking about failure modes from the beginning; they're thinking about operational success and performance modes.

Where AI governance actually lives in an org

Mo: That makes a lot of sense, and I'll come back to human-in-the-loop, I have thoughts. But 2026, yeah, we need an AI incident. I also think governance, compliance, and transparency are going to be the competitive advantage for a lot of organizations. Trusting your AI is the easiest way to think about it: if everyone's using AI, now it's "trust your AI." AI is always meant to speak confidently, so we're going to believe it, but can we actually trust it? Belief and trust are totally different. You've worked cross-functionally with the C-suite and high-level decision makers. So where in an organization does this competitive AI-governance advantage actually come down to? Is it a technology problem, a people problem where we need to upskill, an incentive problem, or something else entirely?

Laura Powell: It may be a little bit of all of the above. Every one of those was part of the challenge I experienced building out the privacy program at Indeed, which is very analogous to AI trust now. When privacy was a nascent area in tech, it was, hey Laura, you're the privacy person, go solve this. But to do what I was supposed to do, I had to touch literally every area of the business, from marketing to HR to sales to product and engineering. Everybody had some level of responsibility, even if it was just taking a privacy training course. It required buy-in from senior leaders, dedication of resources people didn't want to give up, and that in-depth translation of legal and compliance requirements into actionable operational plans and product and engineering requirements. There aren't a lot of people who can bridge that space; it's a challenging skill gap.

So some of it is upskilling, and some of it is helping people understand that responsibility is spread across the entire organization when it comes to AI. But a big part is efficiency and tooling. I'm a firm believer that most humans are good and want to do the right thing, but you have to make it really easy and straightforward, and sometimes tell them what to do, and then they'll do it. Getting to what the easy thing is, and how to operationalize it, is a unique skill set. If organizations don't have that in-house and don't have people they can upskill, they should seek resources outside. Most people don't want to go to one of the big four consulting companies, and some organizations can't afford it, but there are lots of boutique firms, or even individual consultants, who can do this. From what I've seen, the most successful organizations are the ones willing to commit to a cross-functional approach. It's not one person's or one department's responsibility. It takes a whole village.

The representation gap: who gets left out

Mo: I want to go back a bit. As someone in the Silicon Valley bubble, I've found my views on AI are totally different from other people's. You're based in Colorado. I was in New York and people there are still very skeptical about AI. What's it like on your side?

Laura Powell: It's interesting to see how different geographical regions respond. On the eastern slope of the Rockies you've got Denver and Boulder, with a lot of tech, and people there are generally excited and forward-looking. I live in Southwest Colorado, on the Western slope, and there's not a lot of tech out here. I live in a town of about 20,000 people, and there's a local college with a whole program around AI and building AI capabilities in the area. What I've really noticed since living here is that in Southwest Colorado you have a lot of exposure to Native American reservations and highly rural populations. You start to see the spread of populations exposed to AI, whose data is available to AI systems, who even have access to those systems. The local community college is really trying to bridge that gap. The town has this weird pull: a lot of highly technical people who work remotely, involved in that space at the college, and it's been interesting to watch that organization try to build up capabilities for more rural, less exposed communities that otherwise don't really have a voice in AI and, in a lot of cases, don't even know much about it.

Mo: That's interesting. When we think about AI in communities that are less connected, most of the country doesn't even have reliable internet, which immediately disconnects them from this massive wave of tech. Basic things like broadband access aren't available in some places. So who's actually being excluded from AI and the ability to use it? AI is fed by data, that's the gasoline for the engine, and so many folks just aren't included and can't contribute to this massive data set. Especially now that we're reportedly out of data, we've trained on all data in existence. I've said this in another episode: we're out. And now we're telling people we'll pay you to come train AI, but I don't think these folks are the ones being engaged. So we're training on a data set of people who already have access to AI, who are already using it, who are probably being influenced by it and now training it based on that. We've created a weird reinforcement loop that I don't think is moving us forward in the best ways. So who's actually being excluded?

Laura Powell: It's like the world's biggest echo chamber. I mentioned that outlier data gets largely ignored. As we continue to source data from the most well-represented populations, the data for underrepresented populations could be there, we could seek it out, but it's not going to be a huge percentage of the overall set we feed in. Because of the way these systems work, and the speed at which they evolve, it becomes a compounding problem, and that representation gap is just going to get bigger and bigger, faster and faster. We really need creative ways to reach these underrepresented populations. Everybody wants to increase access and the ability for these systems to keep innovating, because it lets us build bigger and faster. Okay, if you actually want to build more incredible systems, diversify the data set. As it stands, what's going into these systems is not representative of the real world. It's representative of pockets of privilege. AI is going to drive everything in the future, let's not pretend otherwise, and we're on the precipice of either something really meaningful or something really terrible. We have an opportunity right now to fix this, and that gap can still be closed, but the longer we wait, the worse it gets and the harder it'll be to reconcile.

Underserved communities as unconsented test beds

Mo: There's an uncomfortable reality we don't address often. We've seen it in a few fields: underserved communities and people who don't have access usually end up as the beta testers for newer waves of innovation, not through voluntary requests. They can get tested without consent, and they sometimes don't get the benefit. I did some creeping and saw you work with a food bank. I do a lot of food-based volunteering too. I'm usually hands-on, putting food into boxes; I don't work on the side that determines which systems decide who gets what. But I wonder how these AI systems are going to help or hurt these folks, not just in how we distribute food, but further back in serving communities. At what point do AI systems start determining things for people, like healthcare access, or which medication you can get? When is it being done as a test? There are enough times it's inaccurate or makes a mistake. Food banks are one place I think about, and the Salvation Army: how are they using AI to increase services, and if so, they're likely testing on people who never went through a consent workflow. These are the folks we're missing data from, and now they're also subjected to AI and all its risks.

Laura Powell: A lot of people don't realize this has already been happening for a decade. Think about traditional machine learning used for predictive modeling. A major food distributor has been using traditional machine learning to predict where it will have its highest sales, and therefore its highest inventory needs, in certain areas. Then you have problems like lower-socioeconomic-status areas traditionally having poorer diets and poorer access to fresh food and produce. These models are typically optimized to make money, so they look at the past and say, given the distribution of food sales in this area, these are the products most likely to succeed, and those are the products that get ordered and shipped to those communities. So you create a systemic problem of continually underserving these communities with poorer food choices, because the systems are optimized to make money, not to make better food choices for those people. This has been happening a long time, and it's just going to get worse, faster, and more efficient with AI.

Until people start thinking about risk not just from a risk-to-the-business perspective but from a risk-to-the-human perspective, what's the ethical thing to do here, we're not going to make better choices. You mentioned healthcare. Different people have very different genetic makeup, predispositions, and responses to treatments. If these systems are built off data sets that aren't representative of the person the decision is being made for, they won't make good recommendations or help the doctor make a better choice. These problems will keep compounding unless we start optimizing these systems differently and realize that what they do is replace a lot of human choice, where AI just doesn't have ethics the way a human does. We have to account for that.

Looking to 2027: education and over-reliance

Mo: Last question. Imagine it's 2027 and there's been a catastrophic AI governance failure that makes headlines, costs a ton of money, and hurts a lot of people. Looking back, what could we have done differently in 2025 or 2026? Something very preventable that we're doing now that would stop that horrible 2027.

Laura Powell: I think it's educating people. People don't understand what they're doing. Everybody loves ChatGPT and wants to talk to it about anything and everything, but it's the classic product angle: the more information you give the system, the better it can make your life easier and more convenient. ChatGPT is the mother of all of that; it sucks information in like a crazy sponge. People don't understand how much information they're giving it, or the kind of profiling that could be happening on the back end. So I really think educating people, teaching them to be aware of privacy and security considerations online, and helping them understand how AI systems work even at a high level, matters. The more educated we are as a group of humans, the better decisions we can make and the more informed we can be when these companies are doing all this stuff behind the scenes that most people don't understand.

Mo: I think we're totally aligned. Mine was also education, but from the over-reliance side. Over-reliance is going to be a massive issue. We've already seen cognitive decline from people relying on these systems, and a lot of misinformation; AI has made it much easier to make misinformation seem legitimate. We're seeing de-skilling too, people losing critical thinking and reasoning. I'm sure one day, when voice AI is cheap enough, we'll start losing the ability to read; I think reading comprehension is going to get hit hard, but who knows.

Laura Powell: I saw something the other day: better start eating healthy, because your future doctor is using ChatGPT to pass medical school.

Mo: You never know. Over-reliance is going to cause a huge issue, and people are building systems they don't understand, products that are essentially houses of cards. But I'm excited for 2027. Hopefully GTA 6 is out before then, because I've been waiting for years. That's what we've got for today. Laura, thank you so much. Where can people find you? Any conferences or cool things coming up?

Where to find Laura

Laura Powell: I don't have any specific conferences planned, but people can find me on LinkedIn. That's the easiest spot. I'm not really a social media person; I'm not on anything other than LinkedIn. This was really fun, and if we do a sci-fi episode, let me know, I'm totally in.

Mo: Of course. If we don't do it here, I'll get a sci-fi episode done somewhere. Someone will listen to us rant about sci-fi. I'm honestly more excited about sci-fi than about security, and I want everybody to know I'm a sci-fi nerd. Laura, thank you so much for your time. It was great talking to you, and I hope to see you soon. If this episode helped cut through the noise, like or subscribe so you don't miss what's next. Thanks for spending time with us. Until next time, stay curious.

‍