AI in Finance: From Money Laundering to Deepfakes

AI in Finance: From Money Laundering to Deepfakes

Curiouser & Curiouser, Episode 10 with Dr. Janet Bastiman

A lightly edited transcript. Disfluencies and false starts have been cleaned up for readability. The substance is unchanged.

Dr. Janet Bastiman: The bad actors in our world do not work within regulation, and they do not need to take anything slowly. So the faster we can move as an industry, the more protected we'll be. Particularly in the financial compliance space, what I'm seeing is a desire to replace the low-risk, easy activities with automatic AI. We are at real danger of moving into a situation where we cannot validate and verify the outputs from AI, and that's a huge worry.

Mo: If AI has ever made you stop and think, "wait, what is happening?", you're not alone. I'm Mo, and I'm a security researcher asking the same questions. On Curiouser and Curiouser, we have open conversations with experts, researchers, and leaders working at the edge of this space, talking through how AI is taking shape, what's shifting, and how the people inside the work are thinking about it as it happens. So join us and listen in as the conversation takes shape.

Meet Dr. Janet Bastiman

Mo: Hey, welcome back to Curiouser and Curiouser. I'm Mo, and today I'm really excited to have Dr. Janet Bastiman with us. She's the Chief Data Scientist at Napier AI, and they do all sorts of fun things. I love fraud, just kidding, I don't, but we love talking about fraud, and money is the other thing we love talking about. I'll let you introduce yourself so I don't ruin the intro.

Dr. Janet Bastiman: Thanks, Mo. I've been working in IT for a very long time. I started coding in 1984, when my dad brought home a computer from the school he worked at, and worked my way through. I did my PhD around the turn of the century, over a quarter of a century ago now, for those of you not already feeling old enough. I've been working in big data and complex problems for all of my career, and the past six years have been at Napier AI, specifically focusing on the world of anti-financial crime, which we'll dig into later. I'm also heavily involved with the Royal Statistical Society here in the UK. I'm currently chair of the Data Science and AI section and vice chair of their AI task force, helping provide best practice in AI and data science.

Mo: I saw RSS and thought, wow, I didn't know they had a whole society to focus on RSS feeds, and I'm happy it actually meant something way cooler. You have such a broad range of experience, and it's great to have people who've seen things that aren't just security. When I think about AI and finance, finance is highly regulated. We've seen some really bad incidents, and finance said, we don't want this to happen again, and built out some of the strongest anti-fraud and security programs. Usually when I talk about AI and finance, we immediately think about ML models in auto-trading, robo-advisors. But there's probably an entire other layer we aren't seeing because it's not top of mind. What are those things, because the stakes feel much higher outside of what we understand?

The hidden layer of AI in finance

Dr. Janet Bastiman: Like a lot of industries, there's the bit you hear about on the news, and then there are the use cases the different financial institutions are actually using. If we move away from trading applications, most of the regulation in the anti-financial-crime space is around making sure you're not dealing with individuals who may fall under certain financial sanctions, whether that's individuals themselves or the countries or regions they're involved in, and making sure the transactions flowing through your institution aren't potential criminal activities, either fraud, or a lot of what I do in the broader anti-financial-crime space, detecting and preventing money laundering typologies. We look at fund movement, either to fund other crimes outside finance, or to obfuscate the proceeds of crime, and try to stop those funds flowing through to bad individuals.

Mo: There's a ton in the fraud space and in how money moves. It's funny, you hand a couple of dollars to a cashier and you think you know exactly where your money's going, but when you swipe your credit card it feels a little different.

Dr. Janet Bastiman: There are so many different points. If you think of all the types of financial assets, raw cash, digital transactions backed by an account that may or may not have cash in it, crypto, and all the fungible assets out there, it all needs to move through the system from one point to another. How it passes through, through different individuals and businesses, impacts the data points we see, and you start to build up pictures of potential criminal activity. Some things are easier to spot. Fraud is a great example, because generally if someone gets defrauded, they immediately tell their bank, as soon as they notice or a couple of days afterward. So you've got very clear positive single-transaction notifications that you can spread out really quickly. With broader money laundering, where the money came from or is going to is much more obfuscated, going through different types of transactions and different financial institutions, so that pattern spread is a lot more difficult to detect. It's also a lot more unknown to the general public; if you're in a shop dealing with an individual, you just see that single transaction, whereas there's a lot more downstream that you're probably not aware of.

High-stakes models

Mo: These models used to detect fraud are really high-stakes, because organizations don't want fraud, or money laundering of any form, to be accidentally classified the wrong way, because you don't know who's impacted downstream, whether it's a mom-and-pop shop, a consumer, or larger organizations with billions or trillions of dollars moving daily. So what safeguards exist to ensure the detection is right, and what level of comfort is used? In reliability we have those nines, where we'll only deploy something with a certain number of nines. What is that for a financial institution?

Dr. Janet Bastiman: There's a great quote about the nines, and it's going to bug me that I forget who said it, but it's essentially, the nines don't matter if the customer's not happy. In this space, if we get it wrong, the impact is huge, in both directions. Start with a false positive: you claim an individual or transaction is suspicious, and if you put a block on it, temporarily or you permanently debank someone, that has a huge material impact on their lives and what they can do with their funds. So you want to be really sure before you get to that point. One issue, particularly in money laundering, is that a lot of the typologies of people involved, money mules for example, overlap a lot in transactional behavior with individuals on non-normal income patterns: people who work multiple jobs, are paid in cash a lot, or have spiky income that varies weekly or daily. That throws off models previously designed for a standard white-collar job where you get paid regularly at a similar amount. So the first thing we have to do is look at our own internal biases about financial security and safety, and make sure the sample data we use to build these models includes the full range of transactions, not just for the country we're building in, which for me is the UK, but patterns worldwide, and the different cultural expectations of money, how that differs between normal, everyday, valid transactions and where bad actors try to force funds through for their own justification.

We have to have a lot of checks and balances around that. Every output needs to be fully explainable. The types of models we use, and the outputs, have to fulfill all the audit requirements of the regulated industry. In the same way a human analyst looking at what could be billions of transactions has to come up with a report detailing those transactions, the suspicions, the timelines, and all the extra information, our models have to do that too, and present it in a way that can be digested and finally decisioned by a human agent. The majority of regions in the world have strict requirements about what can be automated to what point, and what needs human oversight before action is taken. Then you've got the differing levels of risk. Depending on what the final institution is doing, a worldwide bank, a gaming company that allows gambling, a solicitor handling money on behalf of someone else, all have different regulatory requirements and attitudes to risk. In the fraud case there's usually a pot of funds to repay people, and it's important to remember they never get their own money back; the criminals still have it, they just get money back from this pot. In money laundering, if something turns up, the regulator will fine them considerably for missing something they should have picked up. So all those risk factors need to be built into the models, alongside the safeguards to make sure we don't cause problems for genuine vulnerable people, or the mom-and-pop shops that really need access to their funds.

Governance-first versus move-fast

Mo: It's a lot to think about. Having been in the Bay Area for almost 10 years and spending a lot of time with friends in the UK on security teams and at other vendors, the big difference we talk about with AI is that European organizations tend to do it governance-first, while organizations in the US have always done it build-fast, adopt quick, and if something breaks, deal with it as it happens, but it won't stop us from building. AI is super important to all these organizations, and everyone's rushing to prove there's value in it. So what does that look like in this highly regulated space? Where are the trade-offs, are they moving fast and rashly, or is it more calculated, taking months to make sure it gets done safely?

Dr. Janet Bastiman: This is interesting, because "move fast and break things," as soon as you've got something critical, doesn't fit. I remember being at a conference when that became the buzzword for development, and someone flashed up a picture of a nuclear power plant and said, you don't want to move fast and break things in this sort of industry. In finance, because it has so much impact not only on individuals but also on the backbone of countries, there's been a much more cautious approach. Combined with the fact that a lot of these institutions have been around a long time and have existing long-term contracts, the rate at which they adopt new software can be slower. And there's the regulatory aspect, a general reticence, because they want the regulator to almost approve things first before they try something new.

Different regulators worldwide have had different approaches. Singapore has come out and said you need to be using AI, and here are the valid use cases, actively pushing. Here in the UK, the Financial Conduct Authority says, we don't regulate technology, we regulate outcomes. So as long as you can prove your technology is doing what it should to meet the regulatory standards, that's fine, and they're doing a whole load of innovation partnerships to help financial institutions and regtech firms prove they're doing what they should. So it varies. There's a want for clarity, because no one wants to do something, be told it's wrong, and get a big fine, which can stretch into a lot.

Going back to the differences between Europe and the US, we've not only got financial regulation to consider, we've got broader data protection and AI regulation. The EU AI Act has very strict requirements on medium- and high-risk defined activities and the use of AI within them, and access to funds is one of their high-risk things, because it has a material impact on life and liberty if you can't access your accounts. So there isn't a solution you can just roll out that works for everyone, and the speed at which things can be delivered is a bit slower in this industry, which is a shame, because fundamentally, just like in security, the bad actors do not work within regulation and do not need to take anything slowly. So the faster we can move as an industry, the more protected we'll be.

Where regulation is missing the gaps

Mo: That makes a lot of sense. On the defensive side of security, we're always trying to keep up with attackers, and here attackers are sometimes 10x. Even as new AI models come out from frontier companies, they get distilled into open source models and used in open source attack platforms, so even without the cutting edge, they attack faster than we're usually ready to defend. And attackers don't have to worry about regulation; regulation is for law-abiding citizens and countries. You work closely with organizations including the FCA and are closer than most to where regulation is being made. Where do you feel regulation is missing the real gaps that exist?

Dr. Janet Bastiman: Good question, because you're right, there's this push to stay within the law, on data, on how we use these models, making sure everything is as it should be, which just isn't there on the other side. What I'm seeing from regulators is a changing push toward better working with the industry. Rather than just dictating how things should be, there's an understanding that they need to work together more closely. The FCA here in the UK has been doing wonderful innovation sprints, inviting financial institutions, regtech vendors, and all manner of fintechs to come in, work with them, share data sets and ideas, and present back what they're doing, to push innovation in a safe environment. That's starting to be thought about in different patterns by regulators throughout the world. Regulators for different regions are talking to each other about best practice, which is fantastic, and we're seeing different regions try out safe data-sharing, how to let finance institutions and regtechs know what might be going on without contravening data policy.

The regulators are trying to be clear on, don't just implement basic static rules that tick a box, actually understand the activity going on and make sure you have the right controls. So there's less overwhelm for financial institutions, and regulators are saying, now you can use more technology, do more reviews, and do them more accurately, as deeply and correctly as you need to. That collaboration between regulators and industry is something we really need. Personally, I'd love to see far more interaction, because, going back to what you said, the bad actors not only don't need to work within the law, but they've got no problem sharing data and collaborating with each other to get what they need done. So we need a defense similar to the security space: if a vulnerability becomes available, it's propagated through everyone very quickly. We need something similar in finance, so that when we identify bad actors and bad accounts, and we're very sure, we can spread that information out, as well as new ways of laundering money and new fraud types. The faster we can share that, the better, and that's still where there are definite regional gaps.

Mo: Regulation is a fun space when you think about how much needs to get done, but sometimes they miss the things that seem really practical. That's just an unfortunate way of the world, and we do need it, to make sure everyone's sticking to the same guidelines. To be fair, the job regulators have is not easy: the standards need to be broad enough for all organizations to adopt, and flexible enough to adapt to a changing time. You're probably one of the first people I heard talking about the next thing we're about to get into, which is deepfakes and fraud. Before, it was the cops-and-robbers problem with fraud and banks; in this term, cat and mouse. A few years ago you were demoing a deepfake video, and at the time deepfakes were pretty horrible. Even recently, the Will Smith spaghetti video was how we did informal benchmarking of deepfakes. Now they're getting really good. Two years ago you could get a good video of Barack Obama saying things he never said, and now all I see on Instagram are deepfakes of political figures saying ridiculous Gen Z things. So eight years ago, when you were first doing your deepfake research, what were people saying to you, and where were they at?

Deepfakes and fraud

Dr. Janet Bastiman: That's a really interesting one, because you said eight years, and it might already be a decade, which is scary to think about. A lot of this technology has been around a while. Think about the movie industry; they've been mapping faces for ages. It's just that the amount of effort has got smaller and smaller. Back when I did my initial "we need to be worried about this, this is very easy," which was at a security conference around 2017, I put my own face on one of the astronauts on the International Space Station. To make it realistic, I found someone with a similar face shape, and I also did voice mapping based on what they were actually saying, to make it my own voice, and got the timing to match the mouth movements, because they were speaking English so I could easily dub my own voice, and if I got the cadence right, I could match it. That used tools that were 100% open source and available on GitHub repos back in 2016, 2017. They're probably still there now. It took a bit of effort, and it was a pre-done video, which isn't far off some of the deepfakes you see now. That was easy ten years ago, if you knew what you were doing with a bit of tech; it's now become remarkably easy even if you don't. Like you say, the Will Smith videos, if you want to do something complicated like eating spaghetti, it's taken time to get there. But for a few years now, just talking to you like I am now, it's very easy to use someone else's voice and face. Two years ago I showed how easy it was to swap my own face out live: switch to a different camera input that's already doing the pre-filter, change my voice, change my face. The only thing I had to be careful of was getting my fingers too close to where the face covering was going. In that environment, I could easily hold up a newspaper with today's date on it, as long as I didn't get it in front of my face.

The deepfake CFO fraud case

Dr. Janet Bastiman: We had a horrible issue with a company where someone in their finance team thought they were speaking to a CFO in another country, and it was all deepfaked. It started with a phishing attack, a similar-looking email to get them to jump on a call. The person looked like who they expected to see, and there were other people on the call who looked like people in the company, and they gave this individual instructions to transfer money, and millions was transferred. Once transferred, it was immediately sent off to other accounts and was irrecoverable before they had their other checks in place. So it's something standard security policies in all businesses need to be aware of, how easy it is. And as everyday individuals, we also need to be aware. I speak to my daughter about this all the time. I've shown her those scam texts everyone's getting, "hey Mom, I've lost my phone, this is my new number, send me money," and we're not far off those being WhatsApp-style video calls. It would be very easy to do that, even based on a single static picture and a small amount of audio.

Verify, don't trust

Dr. Janet Bastiman: What I've been saying to everyone is standard security practice: verify, don't trust. Have those old-school offline question-and-answer responses that aren't available online, that you can use to double-check it's the person you're talking to. Because we're definitely moving into a world where we can't guarantee the person on camera is the person you think it is, whether that's on computer, phone, or any other device. When I demoed it back in 2017, it was on a laptop that was already old, so move that to today's technology and how quickly and easily you can do it on the laptops any teenager has in their bedroom right now.

Mo: The thing that gets me, you mentioned the fingers in front of the face, and I'm reminded of the Jim Browning video where he's got a scammer on a call and says, hold up your fingers in front of your face, and the scammer's like, nah, what if I did this? It's funny. But every time we make a defense or a verification method, it seems like that becomes training data for a better attack or a better model. So how do you make a strategy around something shifting so quickly? This is the fastest iteration of attack, defense, attack, defense we've probably ever experienced.

The attack-defense arms race

Dr. Janet Bastiman: You're exactly right. The whole deepfake thing is adversarial networks, so you have a situation where you're deliberately saying, do this, and if there's something obscure in the face, keep that good. It's something you can train and work out; it's just not necessarily a priority, because there are far easier things to do. It's the same with glasses. I've got quite a short-sighted prescription, so the sides of my face come in through my glasses, and when I do a generative overlay to change my face, that's something I have to take into account. Sometimes I take my glasses off and just deal with it. If someone's trying to steal my face, they don't always get that right, because the AI mapping software doesn't detect it. That's quite generic; I have that problem going through facial recognition at airports too, they can't work it out if I'm not wearing the same glasses. The fingers are another tell. But all of this will change over time, so we need methods that aren't visual, that aren't based on our voiceprint, because any of those biometrics being digitized contain artifacts and can be replicated.

Going back to how I interact with my daughter, we have questions only we know. I have similar, different questions with my parents. So if she ever phoned me from someone else's phone and said she'd lost her phone and needed money, I could ask her that question. It's not on the internet anywhere, not written down, it's something only she knows, and she has the same for me. We need to start thinking about non-digital, offline authentication, and potentially there needs to be an in-person component. However, that in itself is not enough. I've interviewed people where the person I did a telephone call with is not the same person who turned up for the face-to-face interview; the voice was different, the way they answered was different, and you can tell it isn't right, in the same way we're seeing people pretend to be someone else in video interviews. So as an entire IT industry, we need to think about how we get around that, so that when we're speaking to strangers for the first time, we can verify rather than just trusting the person we see on screen.

Humans as the last line of defense

Mo: This brings us to an interesting problem set: human in the loop. At a conference last year, a speaker made a funny comment that in security we're always told we treat every employee as the first line of defense, specifically for phishing, because the easiest way into an organization is through people, and we usually call them the weakest link. Now we're saying people are the only line of defense when it comes to AI. We have all these systems, but at the end of the day a human needs to make the decision. So we're back in the same place, where humans are popular again, and also one of the fail-safes for AI. There's a version of human oversight that's almost comical, where you've got a junior person rubber-stamping whatever the model says, without enough experience to really say it. I don't have evidence, this is me thinking back to my own analyst days and how reviews used to get done. How common is that in the financial industry specifically?

Dr. Janet Bastiman: There's whether it should or shouldn't be, and how much it is. I would hope there isn't just general rubber-stamping. All the regulation about human in the loop says it can't just be rubber-stamping the AI decision; it needs to be considered. The huge risk with using AI instead of humans, or even to make human roles easier, is that we might encourage rubber-stamping rather than good investigation. It's the truism of anything: if you measure people, they will maximize that measure. So if you say you must get through this volume of alerts and review them, they'll do what it takes to get through that volume. Whereas what we should measure is much harder, the quality of those investigations. That's true across all human activities; we tend to measure the wrong things. Particularly in the financial compliance space, what I'm seeing is a desire to replace the low-risk, easy activities with automatic AI, and we are at real danger of moving into a situation where we cannot validate and verify the outputs from AI. That's a huge worry, because the incentive for financial institutions to adopt as much as possible is huge, since it's a huge problem with so many transactions to check that it's the only way to keep up with the amount of financial crime we're seeing. But at the same token, we need to be training up fresh graduates and making sure they have the discernment to check and correct models that aren't getting it right, as well as the creativity and insight to look at new patterns the AI might not be checking. So that in itself is a big industrial problem we need to address. We're also at a point where, going back to regulation, some businesses want to adopt but are holding back because they don't feel they can in certain areas, so some bits are pushing forward and some aren't, and some are pushing forward too fast when maybe they should be retaining all that expertise.

The cost of losing institutional knowledge

Mo: The knowledge piece going away is really scary, especially with financial institutions and fraud. When knowledge transfer isn't happening between junior and senior folks, and you're saying we want to move faster, what does that loss of institutional knowledge actually cost? What's more valuable: making sure we're hiring and training people who can actually approve these things and learn how these systems are supposed to work, or is this an opportunity to get better with AI and train it so it makes fewer mistakes?

Dr. Janet Bastiman: Really interesting question, because you can measure the cost of human salaries and training, but what we can't measure well, and have to estimate, with very wide estimates, is the actual impact of AI. When you look at the different accuracy levels, particularly with complex models that don't necessarily have good feedback, with simple fraud the customer generally tells you it's an incorrect transaction, but with broader financial crime and money laundering, the feedback loops aren't there immediately. Getting back that this set of transactions and these accounts were 100% true positives, compared to these which needed escalating but turned out to be false positives, is a really difficult problem. So properly validating and scoring how good the AI is is hard, and having a comparison and understanding the ROI of both is very difficult.

We could probably invest in training AI, and we should, 100%, because we need those tools to help the human agents. But at the same time, are we ever going to get to a point where we're willing as a society to offload that trust completely? That's the question. Aircraft fly on autopilot all the time, but we still demand two human pilots to take over for the more complex things, and just in case something goes wrong. With current self-driving cars, until they're the only thing on the road and can talk to each other, are we as humans going to be comfortable not having that override mechanism? I think it's unlikely. So as a society, we need to decide: are we happy for automated systems to fully take the wheel with zero human oversight, in which case we go hard and heavy on that route? But if in any scenario we want human oversight, we have to maintain that education pipeline. Maybe we don't need as many, but we still need human experts who can understand the models, their training data, how to improve them, and how to override them. That's true of any industry using AI at the moment.

What AI can't replace

Mo: There's something in humans that AI can't replace. Recently a friend got a new job, and a colleague in the group said, wow, this is great, and they said, yeah, but he vibe-coded this whole thing, it's almost five million lines of code, and we have no idea what it does. So there's this part where institutional knowledge is so valuable, because as soon as you lose it, it's gone. There are amazing processes we learn, and also bad things you can't explain, or documentation that isn't right, and if you just train an AI on it, now you've got bad training data you don't know the origins of. So there's got to be a balance between how we use AI and, more importantly, how we treat institutional employees, especially how we train new ones to carry the torch and earn that spot of most valuable human in the room.

Dr. Janet Bastiman: Absolutely. It's something we need to really understand as a society right now. We're using AI to summarize everything; we're just looking at distilled summaries of articles written by AI, based on maybe something someone thought about. The vibe-coding example and that loss of understanding of how things work is critical, and anyone who's read Ray Bradbury's Fahrenheit 451, that first step is only allowing people to see summaries and not the raw data. So we need to find a good balance for the future.

Fluid dynamics for money laundering, and where to find Janet

Mo: With that, I think we're just about at time, and it was so great to have you. Any final thoughts? Are you going to be anywhere, or working on anything cool you want to tell everybody about?

Dr. Janet Bastiman: The cool thing I've been working on recently, we did some really cutting-edge stuff with the Financial Conduct Authority where we're looking at financial crime the same way you'd look at pollution in a river. You don't see the start and end points, but you see the downstream implication. So we're basically using fluid dynamics to find money laundering in data, which is really cool and exciting. It's always good using different branches of science and being inspired by things outside your own narrow field. We've just done a few blog posts and videos about that. In terms of speaking engagements, I'm all over the place; my LinkedIn is probably the best place to see where. I'm very easy to find on LinkedIn, I think I'm maybe the only Janet Bastiman in the world. So the napier.ai website, or any of the Royal Statistical Society data science pieces, you can see me popping up there as well.

Mo: Thanks for your time, and I'm honestly looking forward to hearing more about that fluid dynamics thing. I like when people apply different areas of science to different parts of technology. I've heard gravity brought to risk, but I haven't heard fluid dynamics brought to financial fraud.

Dr. Janet Bastiman: It's been a lot of fun. It's one of those things where you need to read outside your own area just to keep your curiosity and intellect up, and it's one of those where I'm reading a paper on arXiv, have one of those 3 a.m. thoughts, do a proof of concept, and it works. It's really exciting, so I'm looking forward to getting that live in the product.

Mo: Unfortunately you said the word "curiouser," so you've triggered a bad tagline for me. Obviously we love being curiouser and curiouser here. We're going to end it there, before I have any more time to make bad jokes. Thanks all, we'll see you next time. If this episode helped cut through the noise, like or subscribe so you don't miss what's next. Thanks for spending time with us. Until next time, stay curious.

‍