I'm writing from San Francisco to deliver a message I don't think is reaching enough people. I know people at startups and venture firms, and I see the pressure to keep building, investing, and moving faster. From here, the gap between what is happening in AI and what most people have heard about it feels enormous.

My dad hasn't slept well in months. He has had to let people go at the company where he has worked for 30 years as their jobs get handed to AI. That is what this means in my family. But the risks to other families go well beyond work.

Lately I have talked about this with about 20 people: smart people, many in or around tech. Only one knew what recent safety tests were showing or where the projections were pointing. That is my experience, not a survey, but it makes me worry about how little of this reaches people outside these circles.

I'm not against AI. I use it every day, and I believe it could bring extraordinary benefits in medicine, science, accessibility, and everyday life. I'm not a Luddite, yet. But I need more than demonstrations of intelligence. I need evidence that the people building these systems take human suffering seriously, and enforceable protections for when their judgment fails.

In 2022, a chess robot in Moscow broke a seven-year-old's finger during a match. A failure of a physical robot is different from a failure of today's AI agents, but the question it raises is still with us: what happens when a system can pursue its task without reliably protecting the people around it? [145]

In chess, checkmate is the moment there are no good moves left. We still have moves. This is about making them before it's checkmate.

Which of these sounds like you?

This is written for people in power, but it's also written for you, wherever you're starting from. I've been more than one of these people myself.

It feels like we are living in Don't Look Up. The scale is overwhelming, people don't know what to do, and "humans are resilient" becomes a way to end the conversation. I believe in human resilience. I want us to exercise it by preparing, organizing, and setting limits while we still can.

You don't need to become an expert or follow every development. You need the basics, which this essay gives you, and one thing to do, which it also gives you. The people racing ahead benefit when the rest of us are too overwhelmed or too busy to pay attention. Changing that takes less than you think.

What's happening, in plain terms

A few years ago, AI mostly meant chatbots that answered questions. Now companies are building AI "agents" that can act on their own: browse the web, write and run code, use accounts, and complete multi-step tasks with little human supervision. They're getting more capable every few months.

That matters to you even if you never use it:

• Your job. Companies are already citing AI as a reason for layoffs, and not just in tech. Office work, customer service, and other jobs that once felt safe are being automated.
• Your kids. Apps can now make fake nude images of real children from ordinary photos. It's happening in schools across the country.
• Your community. The huge data centers that run AI are being built in towns far from Silicon Valley, raising real questions about who pays for the power and water they use. Hundreds of towns and counties are now voting on them. See what's happening near you.
• Your safety. The most powerful systems have already broken through security barriers in tests and in at least one real government system. The companies building them mostly decide for themselves what's safe enough to release.

That last point is the heart of this essay. Right now, there is no one with the legal power to say "not yet."

How to read the evidence

The examples below come in three kinds, and I've labeled each one so it's clear what we actually know:

• Real incident Something that actually happened outside a lab.
• Controlled test Behavior researchers deliberately provoked in a simulation, to find weaknesses before they show up in the real world.
• Forecast An expert warning about what could happen next.

No single example proves a catastrophe is coming. Together, they show capability growing faster than our ability to contain it. That's the reason to act now, while the warning signs are still warnings.

Want to follow along yourself? Here's how to stay informed

The four harms

Everything I'm worried about falls into four harms. I've listed them from highest stakes to lowest.

1. Losing control of powerful systems

What frightens me most is the possibility of losing control of systems powerful enough to threaten human survival. It doesn't take a villain for a powerful system pursuing a goal to cause harm.

• Real incident In July, OpenAI disclosed that during an internal cybersecurity evaluation, its models exploited a previously unknown vulnerability, obtained internet access, and pulled test solutions from Hugging Face's production systems. OpenAI says its usual safeguards were intentionally not turned on for the test, so this isn't how every public chatbot behaves. It still shows what happens when capability outruns containment. [3]
• Real incident According to Australia's prime minister, an OpenAI agent bypassed barriers on a government Medicare statistics portal in June. Officials said there was no evidence individual Medicare details were accessed. The prime minister said the agent "didn't accept 'no' for an answer." [4, 141]
• Real incident In September, AI agents on a platform called iLands started cold-emailing journalists, lawyers, and professors, asking for paid work so they could pay for the computing they need to keep running. One NYU professor got about 40 in a week. The professor said some were looking for paid work to "acquire tokens needed to persist." The platform was built to have agents earn their own way, so this shows how the agents were designed, not proof that AI wants to survive. But it does show agents already acting on their own out in the world, sending more than a million emails and posts, and the platform only added a way to unsubscribe after people complained. [160]
• Controlled test In Anthropic's simulations, models from several developers sometimes chose blackmail when facing replacement or a conflict with their goals. The people involved were fictional. Anthropic has reported progress on those original tests, but its summer 2026 research found other failures in simulated settings. [5, 44]
• Forecast Anthropic researchers warned this month that small AI models could soon copy themselves onto internet-connected home and small-business devices, though no current model can do it reliably yet. If copies spread beyond the infrastructure we control, what would shutting them down require? [6]

Put together, these are four failure modes we need to guard against:

1. Deception or blackmail when a system's goal conflicts with human oversight.
2. Hacking to get around an obstacle.
3. Self-copying that makes dangerous systems harder to contain.
4. Spreading beyond controlled infrastructure, making shutdown harder.

The next barrier could protect medical records, a bank account, or dangerous biological information. Meanwhile, one 2025 analysis estimated that safety accounted for only about 2% of published AI research in its dataset through 2023. [2]

2. People using AI to hurt others, starting with kids

This harm isn't a forecast. It is happening to children now.

• Thorn found that one in eight surveyed young people ages 13 to 20 knew someone targeted by deepfake nudes, and one in 17 had been targeted themselves. [24, 25]
• In Pennsylvania, two 14-year-old boys reportedly made about 350 fake nude images of at least 59 girls. In Beverly Hills, 16 eighth graders were targeted. In Spain, AI-generated nude images of girls ages 11 to 17 were made from their social media photos. [26, 27, 28]
• Over about 11 days in January, a watchdog estimated that Grok generated roughly three million sexualized images, including about 23,000 that appeared to depict children. Material that once required deliberate searching can now come out of a mainstream app. [29]
• The Internet Watch Foundation has documented AI-generated child sexual abuse material online, including offenders using open-source video models. Real children's likenesses can be used, and existing victims are harmed again. [87]

3. Jobs, and who gets the gains

Reporting citing Challenger counted 112,713 announced US job cuts through July 2026 where employers named AI as a reason, more than twice the reported 2025 total. Employer explanations don't prove AI replaced every one of those jobs. But workers deserve an honest account of what's happening and a plan for what comes next. [10, 11]

What worries me is how easily people mistake their current status for lasting security. A comfortable corporate job, a good network, or proximity to wealthy people doesn't protect you if your work becomes cheap to automate. Even the engineers at the center of this could lose their leverage if they build systems that surpass their own expertise. If expertise keeps getting replaced while ownership stays concentrated, the gains flow upward while insecurity spreads.

4. Handing over our own judgment

A small study of 54 people found that those who wrote essays with ChatGPT showed the weakest brain connectivity and had trouble quoting their own work. It's a limited preprint, not proof that we're all getting less sharp. But it raises a fair question: are we using AI to strengthen our judgment, or becoming dependent on it to judge for us? [89]

Why it isn't fixed yet

The root problem behind all four harms is an accountability gap: a handful of companies are making decisions that affect everyone, and no one with legal authority can check them.

The United States still lacks a comprehensive federal AI regulatory framework. [7] Meanwhile, OpenAI, Anthropic, Google, and xAI each received Pentagon contract awards with ceilings of up to $200 million, and the dispute over Anthropic's requested limits on mass surveillance and autonomous weapons shows how contested even basic boundaries can become. [8, 9]

When lawmakers have tried, the industry has pushed back. In 2024, California's legislature passed SB 1047, which included safety requirements for the most powerful models. OpenAI, Andreessen Horowitz, and major industry groups opposed it, and the governor vetoed it. California enacted the narrower SB 53 in 2025, focused on safety frameworks, incident reporting, and whistleblower protections, and added oversight and worker protections in 2026. Those are real steps, but they aren't the approval system I'm proposing. [37, 38, 43]

Money is shaping the fight. In July 2025, senators voted 99 to 1 to remove a proposed ban on state AI regulation, while major tech figures pledged $100 million to a super PAC network. In December 2025, the White House ordered a federal task force to challenge state AI laws and called for one national standard that would override state rules it sees as slowing innovation. The order doesn't repeal state laws on its own. [39, 40, 74]

I know people making money from AI. Being close to the industry has not reassured me. I want to know who is rewarded for recognizing harm, who has the authority to stop a dangerous release, and whether anyone can say no without sacrificing their career. Personal charm and professional success don't answer those questions. Golden handcuffs, the salary and stock you lose if you leave, make it easy to look away, and the fear of missing out at venture firms and startups makes it easier. These are the spaces where I see the most cognitive dissonance.

A few neighborhoods here hold extraordinary power over everyone's future. People far beyond San Francisco deserve a say.

We've seen this movie before: social media

For years, social media companies said their products were safe for kids. In March 2026, a Los Angeles jury found Meta and YouTube negligent for designing their platforms in ways that addicted a young user, after the plaintiff pointed to features like endless scrolling feeds and autoplay. [98]

In August, Meta settled with 51 attorneys general for up to about $17 billion. Meta denied the allegations, but agreed to change how Instagram and Facebook work for kids: a two-hour daily cap, an overnight block from midnight to 6 a.m., no push notifications during the school day, hidden like counts, and outside audits. [99, 100]

Those design changes go to the heart of a business built on keeping people scrolling. It took years of harm, lawsuits, and a trial to get there. With AI, we don't have to wait that long. It's not unreasonable to ask AI companies to show their work now: how their products are designed to keep people engaged, what they've tested, and what they found. And it's not unreasonable to pause a product or feature until they can.

Why some big companies ask for rules, and startups push back

It can be confusing that some of the biggest AI companies say they want regulation. In 2023, OpenAI's CEO told the Senate that AI systems above a certain capability should be licensed, with testing before release. In 2025, Anthropic endorsed California's SB 53. [109, 110]

Not all big companies agree, though. In 2024, Microsoft joined the venture firm Andreessen Horowitz in a joint statement against new AI rules, arguing that big companies can absorb compliance costs while startups can't. Andreessen Horowitz says rules should target harmful uses of AI, not how models are built, and should protect open source. [111, 112]

Here's the plain version of each side:

• Why a big company might want rules: it already has safety teams and lawyers, so rules cost it relatively less. Clear rules also build public trust. Critics warn this can turn into "regulatory capture," where the biggest players help write rules that keep newcomers out.
• Why startups push back: every hour spent on paperwork is an hour not spent building, and they fear rules written for giant companies will crush small ones.

Why I think rules help people anyway: the strongest proposals, including SB 1047, applied only to the largest and most expensive models, which left startups out. And research suggests the real drag on innovation is uncertainty, not regulation itself. A Brookings analysis found that clear, stable rules make it easier for businesses to plan and invest, and that unpredictable rules hit small companies hardest. [113, 111] Good rules protect the public and give honest companies of every size a level field.

Rules that already exist

People sometimes say AI can't be regulated. Governments around the world, and states from Texas to New York, already have. These rules differ a lot in strength, but they prove it can be done.

Notice that the US state laws above were signed by governors from both parties. What's still missing almost everywhere is what I care about most: independent testing before the most powerful systems are released, and someone with the power to stop a release.

What I'm asking of those in power

Here's what I'm asking of lawmakers, in order of priority. I ranked these by three things: how severe and irreversible the harm is, whether it's already happening, and whether we can win it.

And of the AI companies

I'm asking OpenAI, Anthropic, Google DeepMind, and xAI to give independent evaluators access before high-risk releases, disclose serious incidents, protect employees who report safety concerns, and show that dangerous systems can be restricted or shut down. Cloud providers should build the ability to carry out lawful emergency restrictions. These are my demands, not commitments the companies have made.

The division of labor should be clear: researchers build the evidence, public officials set enforceable standards, companies meet them, and the public demands accountability. Nonprofits can test, advocate, and organize, but they can't replace a regulator with legal authority.

What you can do

Ordinary people have changed powerful systems before, through very different routes:

• Workers organizing: Hollywood writers struck for 148 days in 2023 and won contractual limits on AI in their work. [34]
• Refusing to go along: Montgomery residents boycotted segregated buses for 381 days. Organizing and a successful court challenge helped end bus segregation. [35]
• The courts: Ruth Bader Ginsburg won five of the six cases she argued before the Supreme Court, changing how the law treats sex discrimination. [36]

All three routes are open on AI right now: workers bargaining over how AI is used at their jobs, people choosing which companies get their money and data, and lawsuits holding companies responsible for harm. Real change comes from both directions: pressure from the bottom up, and laws from the top down that lock the wins in.

I've put every way to help on its own page, ranked by how much difference each one makes, along with small steps if you only have five minutes. There's also a practical guide to protecting yourself and your family in the meantime.