Resilience by Design: Inside Estonia's Digital Nation

Resilience by Design: Inside Estonia's Digital Nation

Curiouser & Curiouser, Episode 4 with Joseph Carson

A lightly edited transcript. Disfluencies and false starts have been cleaned up for readability. The substance is unchanged.

Joseph Carson: My view over the last couple of years is that we've used AI more on the defense side than attackers have used it for offense. We've accelerated on using it because a lot of the traditional attacker techniques still work, which is unfortunate, but we're still seeing a lot of innovation and acceleration. The core, fundamental things you have to think about with any AI system are the ethics, the rules, the law, the risk, and then making those decisions: whether you accept the risk or not. And if you don't accept it, what can you do to minimize it where you possibly can?

Mo: If AI has ever made you stop and think, "wait, what is happening?", you're not alone. I'm Mo, and I'm a security researcher asking the same questions. On Curiouser and Curiouser, we have open conversations with experts, researchers, and leaders working at the edge of this space, talking through how AI is taking shape, what's shifting, and how the people inside the work are thinking about it as it happens. So join us and listen in as the conversation takes shape.

Meet Joseph Carson

Mo: Hello, hello, and welcome back. I'm very excited to have our guest here today, Joseph Carson, Chief Security Evangelist and Advisory CISO at Segura. He's got an incredible history, and I don't want to butcher it, so I'll let him tell us how he got started. Joseph, thanks for being here.

Joseph Carson: It's a pleasure, and honestly a privilege to get to chat with you today. My history is a long one, so I'll summarize. I've been in the industry for more than 30 years. I started way back when we called it digital transformation, and it was a true one, because it was moving from typewriters to computers. One of my first jobs was digitizing medical records for hospitals. Back then records were in physical folders on a shelf, and we were connecting mainframes with McDonnell Douglas dumb terminals so doctors could have immediate access to patient records. It was a massive undertaking.

A large part of my early career was in system administration: managing systems, keeping the infrastructure running, keeping users productive. I did a lot of that in the medical field, then moved into ambulance service, so critical infrastructure. When people called the emergency line, my systems were responsible for routing those calls and dispatching an ambulance. When that system didn't work, it was a life-and-death situation. After that I transitioned into security, which at first was just something you did during your day, a task among many, not your whole job. But for the past 20-plus years I've focused my career on cybersecurity.

The transition was interesting: the company I worked for at the time became the victim of a massive DDoS attack. I had the privilege of working with Steve Gibson of Gibson Research, who was also a victim of that attack. I became so fascinated dissecting and analyzing it that I switched my career to protecting governments, organizations, and critical infrastructure, specializing in the identity space over the past 10 years. It's been an exciting career. There's always a digital transformation every couple of years, and of course we're in the AI one now.

Estonia's digital transformation

Mo: You mentioned never letting a good incident go to waste, and each incident teaches us something new about the environment we exist in. That's something I want to talk about, because a lot of people don't know you're based in Estonia, which I've learned has one of the highest levels of data GDP per person. Estonia is so far out there in how it's integrated technology into everyday life. You've mentioned you can open a business in minutes, and tax returns that take me weeks are something people just show up and sign. I'd love to hear more about that, because digital transformation must have meant something different for Estonia than the typewriters-to-computers version for us.

Joseph Carson: For the audience, I've been based in Estonia for over 23 years, through the entire digital transformation. I'm originally from Ireland, so when I came it was all new to me, but I was fortunate to work with a lot of the pioneers who started it. It began in 1991, when Estonia had its re-independence from the Soviet Union. That was really the start.

One of their biggest challenges: when you're occupied or run by dictators, they control your history. Property registers, land registers, a lot of that had been doctored and manipulated by whoever was in control. So the fundamental thing Estonia wanted was to make sure its history could never be erased. That was 1991, which was also the boom of CERN and the internet, when everything came to collaboration and accelerated. When Estonia became independent, they wanted to become a paperless society to take advantage of computers. They also had great education, lots of mathematicians, computer scientists, and cryptographers, so the knowledge in the community was significant.

But they had a problem: the land register, population register, and housing register needed to be tamper-proof so no one could ever manipulate that data again, so whoever's in control cannot change history. So they gave their cryptographers and scientists a challenge: how do you do that in a digital world? I always hear this story that they went into the woods, into a sauna, the government gave them a bunch of bottles of vodka and said, go solve this problem, and then forgot about them. It wasn't until 1997 that they came out. Somebody must have drawn the short straw to go get more vodka so they could carry on a few more years. They published one of the first papers in 1996, went through peer review, and by 1997 they were confident they had the right solution. It was essentially the foundation of blockchain elements, to make sure history could not be erased.

Identity, signatures, and the mandatory digital ID

Joseph Carson: That set off a fundamental path. They knew they needed identity at the core, an element of identity for citizens. They needed digital signatures, a way to verify, sign, authenticate, and authorize changes to data. And time was an important element too. In 2001, all those legal requirements came together into a mandatory digital document, which was innovative, and the Estonians made it their own. It became mandatory for every citizen to have this physical document, even more powerful than a passport, because it had PKI certificates in the document itself, so you could use it for authentication, authorization, and digital signing. Digital signing was put into law in 2000.

Then they started building the first systems. They wanted citizens on board, because you don't want it to look like the government is doing oversight or wants a back door into citizens' information. In fact, it's more of a front door for citizens to interact with the government, and that's how they proposed it. One of the first systems was the tax system, because it was the most painful for everyone. It used to take hours: park your car, pay for parking, stand in queues, fill in forms, find out they were incorrect, do them again, amend them. That's wasted time, a cost to citizens every year, and a profitable business for others. So it was the first system they digitized, and by 2002 it took about 15 to 20 minutes to do your tax returns online. You still had the option to do it in person, but you faced a choice: waste time queuing in the cold, and Estonia is really cold, right now it's like minus 18, or do it from the safety of your home.

From paperless to a digital society

Joseph Carson: As people started using the tax system, it moved into online banking, then healthcare, then voting. The more systems became available anywhere, anytime, from any device, the more it changed. Around 2002 and 2003 it was a significant shift: no longer just a paperless society, but a digital one. Now they were integrating all these systems and services, with interoperability, security, blockchain for non-repudiation, and PKI for authentication and authorization.

I've had a couple of opportunities to interview the former president who led this initiative, and he always says it was education. You had to prepare people for what's coming, keep educating and informing them, keep them involved. It all came together in 2002, but the whole thing started in 1997. That education became critical to the success. People knew what was coming, they were ready, excited, and involved, and they got value from it, no more wasted time in queues, no duplicate forms or duplicated records.

The great thing is when you move from a serial system, where everything waits for something else, to a parallel one. One example is becoming a parent: the minute the baby is born, all the government systems already know you're a parent, rather than you filling in a form and waiting for a name, benefits, and healthcare. It's all done immediately because of the integration and interoperability. That accelerated Estonia to the global stage. Of course, no system is perfect, so there are always challenges, and we've had our share over the years.

Why Estonia hasn't built its own OpenAI

Mo: Estonia has one of the highest data economies as a percentage of population, small but rich in data because of how much technology is in the infrastructure. But it sounds more like implementation than development on the AI piece. In the US it's a lot of, we need to build our own models. Estonia has all the makings, high-quality data, so why isn't Estonia building the next OpenAI? What's the strategy for AI implementation?

Joseph Carson: AI is going throughout all the different systems in Estonia, to the point where almost every system is going to touch an AI agent in some way. Why hasn't Estonia created an OpenAI? We do have local models, a lot built locally, LLMs. I just don't think we've had the computational power and access to other data to make context of it. We've done a really good job at the local, region, and country level. Recently Estonia handed over its language archives to Meta and a few others, to enable true, live translation into Estonian and make it more globally accessible. So there are initiatives to expand use outside Estonia.

They've taken a more cautionary approach, because they had a few issues years ago that made them think about security and go a bit slower. One was an ID-card vulnerability back in 2016 and 2017, through a change in vendor, which caused a massive challenge; they had to reissue new crypto algorithms for the PKI on the card. That doesn't mean going back to something historic or old, it means they have the finest solution to move forward while keeping the trust, ethics, and transparency intact. Estonia has done a really great job using AI in society, and it's only going to get better, especially as it extends into healthcare and automation. One goal is to get to a single click: authenticate, sign, society. You authenticate that you are who you are, and you sign whatever you're signing, whether it's a vote or a legal document.

As of last year, every single government service is now online. On voting, it's not just internet voting; there's electronic voting in the booth, and paper voting if you can't get to a location. Two years ago was the first time internet voting, actually voting with your laptop, overtook all other mechanisms. And when you have a choice, each system provides oversight to the others, some integrity and additional visibility into what's happening.

Standardizing data and keeping it simple

Mo: It sounds like Estonia has given everyone the ability to interact however they're most comfortable, but the most important part is standardizing all that data so that no matter how someone interacts, it always goes back into a format the underlying systems can easily consume. Whether you fill out a paper ballot or press a button, they all translate to the exact same outputs. That's the best part, making it easy to consume, standardizing it, and simplifying the data classification so no one is confused.

Back in high school, I'm not very good at sports, but I was on the football team, and our coach had this saying: keep it simple, stupid. It was so confusing to learn all these plays, but if any of the guys on the line could just do one thing, one action, we could translate it across many plays, and it was easier to execute. In that same vein, from my research and creeping, you've proposed this idea of "AI yes" and "AI no" classification across data, to practically understand risk across different services. It sounds simple, and I'm thinking of Estonia as one massive organization, so doing that across a bunch of different services and providers might be a nightmare. How do you actually enforce these yes-and-no decisions without creating a bunch of security theater?

Enforcing "AI yes / AI no" without security theater

Joseph Carson: One of the critical things is how it's constructed. They refer to the whole back-end system as X-Road, the interoperability between these systems, and they're decentralized, not one big data lake. It's all decentralized and deduplicated. If you want to bring a new service to X-Road, you can't bring the same data somebody else is already hosting. You have to establish where the data is available for you to use and what new data you're bringing. So it's decentralized, de-risked, and interoperability is key, along with the security of the transactions and the blockchain that provides non-repudiation.

On the yes-and-no side, when you get into critical decisions, and this goes back to the subject-matter working group in the EU AI Act, the question is: is there a risk to life? On the medical side, there might be a set of prescriptions that, taken incorrectly, in excess, or with other medications, could be fatal. Do you want an AI-driven system making that decision, with the potential for mistakes? Or do you want it to handle things that are low-risk and repeatable? Do you want an AI system making life-support decisions? Probably not. That's where we have to classify.

A lot of the time, I'm becoming more of an observer to all the transactions happening in the background. There are certain transactions I get to participate in, authenticate and sign. When tax returns come up at the end of March, I authenticate and sign. I can't delegate that to an AI bot. So yes, AI can be used to create and bring the data together to make it quicker, but at some point a human has to make the decision and have accountability, because there are legal ramifications and financial outcomes. So we have to take the EU AI Act, that risk-based model, and run through every scenario.

At the end of each scenario, is there a possibility that a mistake in the AI, which is probability-based, could occur? We have to remember these aren't rules-based systems. If you want a system that makes definitive yes-or-no decisions, that's a rules-based system. These are probability-based, meaning "highly likely." And when it's highly likely, we have to decide: will that carry a risk to life, or significant impact to people's way of living or standards? Then we have to be critical about how we apply it to decision-making.

The e-Residency and open banking lesson

Joseph Carson: A few years ago it started with the data embassy, and then it became e-Residency, so anybody around the world could become an e-resident and use Estonian digital services no matter where they came from. One challenge was that without a financial component, the e-services weren't valuable if you couldn't transact. If you opened a business but couldn't open a bank account, that created problems. So around 2014, they came up with open banking, where you could become an e-resident, get the digital services, and open a bank account online. Something in my gut was telling me there was something wrong, and in retrospect you can see it.

What happened was they treated all e-residents and all open banking equally. People in Southeast Asia and other parts of the world were legally getting e-Residency and bank accounts, but then selling them to criminals who used them for money laundering. That created a massive money-laundering scenario, one bad apple in the batch. Everybody, not just e-residents but anybody who wasn't an Estonian citizen, then had to go through a know-your-customer process, which was a massive challenge. The way it should have been done was to bind e-Residency and banking to the legal framework. An EU citizen, a US citizen, a Canadian citizen have binding agreements between regions; in Southeast Asia there wasn't one.

So when you get into those critical yes-or-no decisions about AI systems, there are multiple things to consider. One is the type of data; data classification is critical. The second is whether there's an impact to human life; if it makes a probability mistake, will somebody suffer? The third is legal: is it doing it in a legal way, adhering to legal requirements, not just within countries but across borders? These are the core things to think about with any AI system: the ethics, the rules, the law, the risk, and then whether you accept the risk, and if not, how to minimize it.

That's ultimately where the EU AI Act came out of. In those working groups we were each given different scenarios. Mine was law enforcement's acceptable use of AI in criminal investigations. What we found was that to earn trust in that scenario, you always have to be right. In a forensic lab, if you find one contaminated swab, they assume all the swabs from that batch might be contaminated. Same with legal cases: if someone did something incorrectly, all their cases might be under review. So we have to be very cautious about unleashing this into everything, which is why Estonia is progressing on certain things but being cautious about which systems it exposes, especially around the data itself. Because once it's out, it's out. There's no turning around.

Persistent versus non-persistent PII

Mo: Data is unfortunately more permanent than we can really understand, and it lasts a very long time.

Joseph Carson: I want to bring up an important point. One mistake we made with EU GDPR, to your exact point, is that we didn't do data classification correctly. We called it personally identifiable information, PII. I think we should have had two classifications: persistent PII and non-persistent PII. Persistent data is data you can never change. You were born on a certain date; you can manipulate or poison it, but you can't change that fact. You can't change who your parents are. There are events in history that are persistent personal information. Health data, past surgeries, those have happened. But a home address is changeable; you can move, change your name. Credit cards, phone numbers, IP addresses, passwords, those are non-persistent.

We should have classified it as two categories, and that would have been much easier, especially in incidents and breaches. If a breach happens on persistent data, it's out there, that's done, you can only monitor. You should always monitor assuming it's out there, but with non-persistent data you can take action: get a new credit card, change your password, take precautions to minimize exposure.

Mo: That brings up a really good point. Having gone through the EU AI Act, and having been on a working group for the AI code of practice for general-purpose AI, there's a recurring theme: the need to reconcile a requirement for really dynamic security with very static regulatory requirements that are almost literally written in stone. It's been said many times, but cybersecurity is a living organism and has to be treated as such. The requirements we have one day, like around PII, need to be modernized. There are pieces of PII that are persistent, I can't change my date of birth, but it feels like we're in an interesting age where we need to reconcile the things that just aren't working anymore, especially when it comes to regulations.

Joseph Carson: I completely agree. Security is a living organism; it has to be dynamic and adaptive. It really comes back to the data, which is what we're protecting. If we can change certain pieces, that changes the risk, but the ones you can't change carry persistent risk.

When AI fights AI: the Formula One race

Mo: A lot of people use the double-edged-sword framing. I like to think of it as a shield: we try to protect everyone with a shield, but you can also bash someone across the head with it. As a security person, we're always told we'll be behind the attacker, behind the threat, and we have to move faster to get ahead. This feels like the one time innovation is moving faster than we can think about defending it. We're so far ahead of ourselves that we're thinking about threats a little too slowly. And it's scary, because we're implementing AI in our defenses in a way that attackers can turn around and leverage against us.

Joseph Carson: You take the guardrails off and you figure it out. My view over the last couple of years is that we've used AI more on defense than attackers have on offense. We've accelerated because a lot of traditional attacker techniques still work, unfortunately, but we're seeing a lot of innovation and acceleration, and we're hoping to keep that pace.

Mo: As we move toward a space where attackers use AI and defenders use AI, but eventually it's just analysts watching AI fight each other, what do you think that ends up looking like? What does the rubble look like when that chaos happens, and who's facilitating it?

Joseph Carson: I have a couple of metaphors for that. You're right, we're becoming observers who can intervene when we need to, to modify or optimize the algorithms and data. A couple of years ago I started thinking AI was almost like a Tamagotchi, a companion you have to keep feeding, entertaining, and giving data. But not long ago I actually got a Tamagotchi as a gift and realized it's the opposite: we're becoming the digital companions to the AI itself. It has to keep feeding us information and context.

To answer your question, it comes down to the context of the data. It's not more data, it's the quantity and quality of the data, plus computational power. That's what wins: whichever algorithm is refined and trained on the best data. Sometimes the best models come not just from having the right data but also some incorrect data, because it lets the model contrast, balance, and learn what's good and right, you hope it learns the good and right side and doesn't go rogue. So you train it with the data that gives the best value and highest confidence, and then it comes down to computational power, GPUs that can crunch that data as quickly as possible, so you can make critical decisions faster than the attackers.

That's the difference. It's going to be like a Formula One race. We're all racing in the same cars. Whoever has the best analytics, the best-performing engine, and the best SOC person analyzing it, the driver, is going to prevent and stop attacks quicker than attackers can abuse them. The better we optimize and decide which data helps us best, that's who goes through the checkered flag. And we'll always have to be ahead.

Shiny buzzwords versus real value

Mo: One more question. You've been across so many organizations and seen products built, fail, and wildly succeed even when maybe they shouldn't have, and you've seen the dark sides too. We're in a space where everyone is adopting AI, all sorts of vendors and products, and in security I've seen too many conferences where vendors just say "we're using AI, we're using AI." We're all responsible for how AI is implemented in products and how we talk about it. I'm not asking this to advertise either of our organizations, I'm asking to challenge both: from your experience, what are the shiny things being talked about too much that add no value, versus the things that actually make security better and make it the trusted partner in the room?

Joseph Carson: Excellent question. We have to change as an industry to get where we should be going. At a lot of conferences over the years I've seen every buzzword and trend word. When you talk about "next-gen this" and "AI part of that," it really comes down to value. What are you making easier for the person? How are you making their lives better, the organization more resilient, more innovative?

Years ago, when I did a pen test for a power station, I was advising and trying to get more budget, and the CFO said, budget denied. We'd taken a fear approach, scare tactics, like everyone scaring people into buying something. What I realized was the CFO asking, how are you helping us make our employees' lives better? We weren't thinking about that. So I sat down and put myself in the shoes of several of their employees to understand what a day in their job looked like. Putting security in place should always make the person's experience better and help them do their job better. That's the fundamental change: any time you see a new buzzword or trend, ask what fundamental value it adds to society, and whether it makes that person's life better. If it's the SOC analyst helping them analyze things, or the employee at a desktop making financial decisions, how does this help them do that more safely while helping them succeed?

Fundamentally, we need to shift from a pure security perspective to a return-on-investment perspective. That return could be anything. What Estonia measures is reducing wasted time, which has a monetary value to society, helps us do more with the budget we have, reduces the risk of people becoming victims, enables better integration and interoperability, and accelerates innovation. You have to ask how it makes society and our lives better, and you have to have a metric for it. That's what my most recent book, currently in draft, is about: taking identity and turning it from something looked at through a security lens to the actual outcomes that add value to our daily lives. We don't do security for the sake of security; it has to have value to the business.

So instead of "AI-powered this," it should be, we're using AI to help your SOC analysts analyze 10 times more incidents, alerts, and indicators of compromise, and detect them three times faster than today. That's clear: it helps analysts work more efficiently on a larger volume of alerts while detecting the malicious ones three times faster. And that turns into, am I staying ahead of the attackers, stopping the attack before it becomes a financial, business, or employee impact? That's what we need.

Where to find Joseph

Mo: Joseph, thank you so much. I really enjoyed our conversation. You mentioned a book coming out, so I'll be excited about it, but where can people find out more? Where can they find you, and what do you have coming up?

Joseph Carson: The easiest place is LinkedIn. That's where I post a lot of my content and do my engagements, and where people connect for mentorship, feedback, or collaboration. You can also find me on my own podcast, Security by Default, on all streaming platforms. And I'll definitely be at the RSA Conference in March, where I'll be speaking, doing a talk on the evolution of Estonian digital resiliency. So come catch the talk and find me at the conference.

Mo: Thank you so much, Joseph. If this episode helped cut through the noise, like or subscribe so you don't miss what's next. Thanks for spending time with us. Until next time, stay curious.

‍