AI safety can be a doom-y scene. One might say “my p(doom) is 30%” and everyone else will just nod along as though this is totally normal and expected.
Lately, when people ask me, I say that my p(doom) is below 1%.
“Why even work on AI safety, then?”
“Because, I expect that the reason p(doom) is low is that we (me as Austin, we as Manifund, we the AI safety community, we the world) will work hard to make sure things go well!”
Maybe this reflects my Catholic roots. While I believe that at the end of days, good will triumph over evil, and it will be on Earth as it is in Heaven — this doesn’t imply that I, personally, should just sit on the sidelines. That would be a dereliction of duty, a waste of my talents, and, possibly a way to consign myself to Hell. Instead, I am called to do what I can on Earth now, to serve my fellow children of God.
I imagine this argument would not convince you, dear rationalist reader. So let me try and give a few more intuitions for my weird hybrid Catholic + EA/rat morality, the headspace where I live now.
Let’s start with: why do good?
As a kid, the answer was obvious: so I could go to heaven. Plain and simple.
As an adult, the answer that the EA people sold me on was a bit more circuitous: to have… impact…
This leaves off, “why does having impact matter at all”? Or maybe, “why does it matter that any individual make the choice that maximizes impact”? When thinking about this, impact is perhaps framed in increasing total utility in the universe.
The classic consequentialist utility-maximizing framing kind of bakes in an assumption, which is that all other moral agents in the universe share the same goal of maximizing utility. And, I think that this is actually a good model, in the limit, as more moral agents converge on a shared decision framework. (While obviously not yet being a good model for the world today.)
And actually, therein lies the reason to have impact, to do good. I aspire to figure out what it is which is good, and to act in such a way to uphold it. I wish to “become” good.
My view of morality is that it’s a framework shared between all actors, a framework which outlines a set of rules which, if abided by all other actors, would lead to the most positive sum set of interactions, lead to surplus and abundance and joy.
Small examples: common-sense morality includes maxims like “love your neighbor as yourself”, “always tell the truth”, “give to the poor”. When universally adopted, these lead to better outcomes among all actors. It seems to me that all of the arc of moral justice is about expanding our definition of who counts as moral actors so that they can join this framework; and about spreading a better understanding among actors of how to cooperate within it.
I’m a moral realist. I believe there is a single universal ruleset, which if we understood, would lead us to act in the maximal creation of what all the participants (throughout space and time) want.
We are insufficiently wise, and do not yet grasp the entirety of this ruleset. But we’ve made some progress through the years. Some likely pieces might include: liberalism, trade and capitalism, religion, virtue and honor, decision theory. (Maybe also basic bits like physics & math).
And so our utmost priority should be to uncover, act by, and propogate this ruleset.
Maybe this ruleset is out of line with our modern understanding of morality, or the norms which are in fashion today. Is that a problem?
I think that for one, the future’s understanding of this ruleset will be better than ours today, and we should be excited to update our values in the direction of the future’s.
But I also think that the preferences of the present (and the past) are importantly valid, and that the future is going to be excited to honor those preferences and trade with us to fulfill them. Don’t worry, the future won’t be crazy and alien. (Or: it will have those bits; but it will also have the bits that you recognize and love).
A quick list of AI safety heresies:
- Goodness/morality, power, intelligence, knowledge are all convergent in the limit
- Relatedly, I don’t believe in orthogonality
- Relatedly, I’m a moral realist