To explain how we got here, we need to talk about recent history. This actually gives me a real excuse to reference The Wire, just because it’s a perfect snapshot of what electronic surveillance looked like way back in 2002. If you’ve seen the first season, you’ll recall that it’s about cops wiretapping drug dealers who use payphones and burners. The mobile phones in the show are relatively new technology for the time, but from a technological perspective nothing in this scenario would have shocked a cop who jumped forward from, say, 1989.
In less than a decade from the premier, everything in those episodes became totally quaint.
The change began in the late 2000s, thanks to the rise of smartphones and texting. Because smartphones can actually store data as well as conveying it, the contents of those phones quickly became a useful new source of law-enforcement capability. Or they were until 2010, when Apple began encrypting iPhone storage using a key derived from the user’s passcode (Android phones followed shortly thereafter.) The next year, Apple deployed end-to-end encryption in iPhone text messages. By 2014, a tiny texting startup named WhatsApp had gathered 600 million users worldwide. By 2016 those users, now nearly a billion strong, were all using default end-to-end encrypted messaging and calls. These two trends — the move from calls to texts, and texts to encrypted data — happened very rapidly. The chart below gives one view of the transition:
The FBI and law enforcement agencies were not insensitive to what was happening. In 2014, Director Comey announced an initiative called Going Dark, which would launch a “*national conversation” *about what providers could do — or be compelled to do — to make these new communications media legible to law enforcement and counterintelligence.
In 2016, the agency quit talking and took their theory to court. When a terrorist attack left the FBI holding a shooter’s locked iPhone, the agency ordered Apple to give them access. The company refused. What broke the stalemate — and, to some extent, ended “Going Dark” itself — was something that neither the FBI nor Apple expected. An outside company announced that there was no need for Apple’s assistance: they could simply hack the phone.
The Apple v. FBI case turned out to be microcosm of the whole Going Dark debate. For the next decade, law enforcement and intelligence agencies continued to ask for “exceptional access” backdoors. But the urgency was gone: both agencies and manufacturers knew that law enforcement could purchase targeted hacking tools like GrayKey (for phone unlocking), or even remote exploitation tools like NSO Group’s Pegasus, assuming they needed them badly enough. Vendors like Apple and Google played a vigorous defense, closing vulnerabilities as soon as they learned about them. But offensive vulnerability hunters consistently managed to keep the edge.
And now there’s a very good chance that all this is about to be history.
This April (just four months ago!) Anthropic announced a new model called Mythos that happened to be unusually skilled at software vulnerability finding. The U.S. government temporarily blocked its export, restricting access to U.S. agencies and trusted vendors. While the ban was dramatic and made for good PR, it turned out to be mostly pointless. OpenAI, along with Chinese open-weight model labs like Z.ai and Moonshot, have since demonstrated that vulnerability finding isn’t something that a single lab is likely to hold a monopoly on. The list of serious vulnerabilities that these models have found is getting scarier (or more impressive) by the day.
At first glance, this might seems like good news for the offensive team, and for hackers in general. But I doubt that’s how this will play out in the long term. Defenders are now in the process of patching every bug they can find — often decades worth of bugs — and the backlog feels huge. But they’re making progress. Entire CI toolchains are being rebuilt to incorporate AI-based vulnerability scanning before software ever reaches the point where a human will touch it. While I doubt this means that every bug will be found (even calculating the number of bugs in a piece of code is probably uncomputable), in the real world, it does feel likely that we’re going to hit some sort of a ceiling on the number of useful bugs, and probably we’ll hit it soon.
Thus: over the next two years, major pieces of software are likely to run out of remotely-exploitable bugs.
Obviously *I *think this is great. But for law enforcement and offensive intelligence agencies, it’s going to be a nightmare. For the first time since 2010, law enforcement might experience what it looks like to really “go dark”, across a huge category of advanced (well-maintained) devices and pieces of software.
The debate over “exceptional access” mechanisms never really went away. In some places, like the UK, it even metastasized into something worse. Here in the US it mostly went into hibernation. Some of the slowdown can legitimately be attributed to expert pushback — academics and industry engineers pointing out the risk that backdoors might be abused by the very adversaries that Agencies are supposed to be protecting us against. But I fear that this was less of a principled pause, and more of a market that was just pricing supply.