Politics is bad for the soul of EA
A lot of people I know seem excited for EA & AI safety to “get into politics”.
Political giving: the meme is that the most effective use of your own money is to give to political candidates like Scott Weiner and Alex Bores, who are going to be good for AI safety. I myself made these donations. And yet, I’m very unexcited about a world where the primary things individual EAs are doing with their money are campaign contributions; in effect, trying to buy votes.
Why?
- Politics puts you into a zero-sum frame. It’s bad for epistemics & thinking clearly. Going all the way back to rationalist roots: “politics is the mind killer”.
- Politics might encourage you to be secret, to not share your thoughts and true beliefs. It might encourage you to say the thing that you think will play well or convince a voter. It doesn’t encourage open, honest retrospectives, or the Mistakes page that Givewell has.
- Politics is noisy, low-feedback. It takes a really long time to figure out whether you’re doing the right things; whether your approach gains votes; whether your preferred politician will actually fight for the policies you asked for.
- Money and politics don’t mix well. People don’t like it when someone tries to influence who wins via lots of money (see, Carrick Flynn, or the OpenAI PAC). There are a bunch of legal constraints around how much money can be spent
Policy: in the abstract, writing good policy seems good. But in the specifics, I guess I’m really unsure. Key questions are like: What have been policy successes in AI safety? What would they look like? If we had a lot of political leverage, what is the thing we would pass?
What am I excited for in politics? I would be excited for EA, AI safety, rationalist folks to actually run & win office.
Open weight models seem good
I never understood the fear of open weight models.
- open weights models seem good
- would go farther, Total Research Transparency, open data, full open source
- secrecy bad, transparency maximalism
For what it’s worth, I currently think OpenAI has been vindicated on their founding stance, of building AI and trying to distribute its benefits among the world. (I’m aware that this might get me lynched).
I guess the main argument against OpenAI is that they accelerated timelines, or created a race dynamic. But:
We might be in a pace-by-default world
An old critique of ML researchers was that, because they’re always having to deal with all the hard parts of scaling up models, they don’t know
I kind of think the AI safety community suffers from a mirror of this. They’re always trying to get the word out that AI might be bad, that people need to pay attention. But they’re not pricing in the idea that doominess is actually pretty intuitive. People have been making movies about killer robots for years.
Another analogy: the rationalist and EA community were absolutely right on COVID trendlines circa Jan 2020. But nobody, afaict, predicted lockdown and other massive societal responses. I think the AI safety community (comprised of many of the same people!) is likewise not pricing in massive societal wakeup, the second and third order effects of rapidly increasing capabilities.
Anthropic seems good
Okay to be fair, my own position on Anthropic has shifted earlier in 2026, when they eclipsed OpenAI in revenue and also (figuratively) everyone I know joined. I think some scrutiny and skepticism of the leading AI lab seems healthy.
But, idk, all the individual people in Anthropic still seem great. They have the people who I look up to the most in the world, people who have informed my thinking and guided my work. Holden Karnofsky and Joe Carlsmith in particular from their writing, and then numerous people I’m lucky to call friends, choose Anthropic as the place to put their efforts.
I also think critiquing Anthropic too harshly is biting the hand that feeds you. I suspect a good chunk of the money that people in AI safety have, comes from Coefficient Giving or SFF, are downstream of either direct investments in Anthropic (from Dustin Moskovitz and Jaan Tallinn), or from growth in investments that come from the takeoff of AI. Moreover, everyone’s busy preparing for the wave of philanthropic funding to come from AI labs. It seems incongruous, inconsistent to on one hand critique them, and on the other try to take their employee’s donations.
Also: selling data to Anthropic seems reasonable. People tried to tar-and-feather the cofounders of Mechanize (Tamay, Matthew, Ege) when they left Epoch to start an RL environment company. But I kind of think the net effect of that was to help Anthropic have more aligned models
I don’t understand why everyone’s afraid of China
Pluralism good, tolerance good, broad-tent good
- we might be in a pause/pace-by-default world
- timelines might be medium. or broad.
- general fearmongering seems bad
- i don’t understand why everyone’s afraid of china
- pluralism good, tolerance good