Thoughts on the Eliezer vs. Hotz AI Safety Debate

I just got done watching
the debate
on AI safety between
George Hotz
and
Eliezer Yudkowski
on
Dwarkesh Patel’s podcast.

The debate was quite fun to watch, but also frustrating.

What irked me about the debate—and all similar debates—is that they fail to
isolate the disagreements. 90% of the discussion ends up being heat instead
of light because they’re not being disciplined about:

1. Finding the disagreement

2. Addressing their position on that disagreement

3. Listening to a rebuttal

4. Deciding if that’s resolved or not

5. Either continuing on 2-4 or moving onto the next #1

Instead what they do (they being the General They in these debates) is
produce 39 different versions of #2, back and forth, which is dazzling to
watch, but doesn’t result in a #4 or #5.

It feels like a Chinese martial arts movie from the 80’s after you’ve seen a
lot of MMA. Like why don’t you just HIT him? Why all the extra movements?

I think we can do better.

How I’d characterize and address each of their positions

I’m not saying I’d do better in a live debate with either of them. I could
very well get flustered or over-excited and end up in a similar situation—or
worse.

But if I had time and notes, I’d be able to do much better. And that’s what
I have right now with this written response. So here’s what I see from each
of them.

Hotz’ arguments

I’m actually not too clear on Hotz’ argument, to be honest, and that’s a
problem for him. It’s also why I think he lost this debate.

He’s flashy. And super smart, obviously, but I feel like he was just taking
sniper shots from a distance while mobile.

Wait, I think I have it. I think he’s basically saying:

Hotz Argument 1: We’ll have time to adjust

1. AI’s intelligence will not explode quickly

2. Timing matters a lot because if it moves slowly enough we will also have
   AIs and we’ll be able to respond and defend ourselves

3. Or, if not that, then we’d be able to stage some other defense

My response to this is very simple, and I don’t know why Eleizer and other
people don’t stay focused very clearly on this.

1. We just accidentally got GPT-4, and the jump from GPT-2 to GPT-4
   was a few years, which is basically a nanosecond

2. The evolution of humans happened pretty quickly too, and they had no
   creator guiding that development. That evolution came from scratch and
   no help whatsoever. And as stupid and slow as it was, it lead to us
   typing this sentence and creating GPT-4

3. So given that, why would we think that humanity in 2023, when we just
   created GPT-4, and we’re spending what?—tens of billions of dollars?—on
   trying to create superintelligence, would not be able to do it quickly?

4. It’s by no means a guarantee, but it seems to me that given #1 and #2,
   betting against the smartest people on planet Earth, who are
   spending that much money, being able to jump way ahead of human
   intelligence very soon is a bad, bad bet

5. Also keep in mind that we have no reason whatsoever to believe human IQ
   is some special boundary. Again, we are the result of a slow, cumbersome
   chemical journey. What was the IQ of humans 2,000 years ago compared to
   the functional (albeit narrow) IQ of GPT-4 that we just stumbled into
   last year?

Hotz Argument 2: Why would AI even have goals, and why would they be counter
to us?

1. There’s no reason to believe they’ll come up with their own goals that
   are counter to us

2. This is sci-fi stuff, and there’s no reason to believe it’s true

Eliezer addressed this one pretty well. He basically said that—as you
evolutionarily climb the ladder—attaining goals becomes an advantage that
you’ll pick up. And we should expect AI to do the same. By the way, I think
that’s exactly how we got subjectivity and free will as well, but that’s
another blog post.

I found his refutation of Holz Argument #2 to be rock solid.

Now for Eliezer’s arguments.

Yudkowski’s arguments

I think he really only has one, which I find quite sound (and frightening).

1. Given our current pace of improvement, we will soon create one or more
   AIs that are vastly superior to our intelligence

2. This might take a year, 10 years, or 25 years. Very hard to predict, but
   it doesn’t matter because the odds of us being ready for that when it
   happens are very low

3. Because anything that advanced is likely to take on a set of goals (see
   evolution and ladder climbing), and because it’ll be creating those
   goals from a base of massive amount of intelligence and data, it’ll
   likely have goals something like “gain control over as many galaxies as
   possible to control the resources”

4. And because we, and the other AIs we could create, are competitors in
   that game, we are likely to be labeled as an enemy

5. If we lots of time and have advanced enough to have AIs to fight for us
   this will be our planet against their sun. And if not it’ll be their sun
   against our ant colony

6. In other words, we can’t win that. Period. So we’re fucked

7. So the only smart thing to do is to limit, control, and/or destroy
   compute

Like I said, this is extremely compelling. And it scares the shit out of me.

I only see one argument against it, actually. And it’s surprising to me that
I don’t hear it more from the counter-doomers.

It’s really hard to get lucky the first time. Or even the tenth time. And
reality has a way of throwing obstacles to everything. Including
superintelligence’s ascension.

In other words, while it’s possible that some AI just wakes up and instantly
learns everything, goes into stealth mode, starts building all the diamond
nanobots and weapons quietly, and then—BOOM—we’re all dead…that’s also not
super likely.

What’s more likely—or at least I hope is more likely—is that there
will be multiple smaller starts.

A realistic scenario

Let’s say someone makes a GPT-6 agent in 2025 and puts it on GitHub, and
someone gives it the goal of killing someone. And let’s say there’s a market
for Drone Swarms on the darkweb, where you can pay $38,000 to have a swarm
go and drop IEDs on a target in public.

So the Agent is able to research the darkweb, find where they can rent one
of these swarms. Or buy. Whatever. So now there’s lots of iPhone footage of
some political activist getting killed from 9 IEDs being dropped on him in
Trafalgar Square in London.

Then within 48 hours there are 37 other deaths, and 247 injuries from
similar attacks around the world.

Guess what happens? Interpol, Homeland Security, the Space Force, and every
other law enforcement agency everywhere suddenly goes apeshit. The media
freaks out. The public freaks out. Github freaks out. OpenAI freaks out.

Everyone freaks out.

They find the Drone Swarm people. They find the Darkweb people. They bury
them under the jail. And all the world’s lawmakers go crazy with new laws
that go into effect on like…MONDAY.

Now, is that a good or a bad thing?

I say it’s a good thing. Obviously I don’t want to see people hurt, but I
like the fact that really bad things like this tend to be loud and visible.
Drone Swarms are loud and visible.

And so too are many other instances of an early version of what we’re
worried about. And that gives us some hope. And some time.

Maybe.

That’s my only argument for how Eliezer could be wrong about this. Basically
it won’t happen all at once, in one swift motion, in a way that goes from
invisible to unstoppable.

Here’s how I wish these debates were conducted

1. Point: We have time. Counterpoint: We don’t. See evolution and GPT-4.

2. Point: We have no reason to believe they’ll develop goals. Counterpoint:
   Yes we do; goals are the logical result of evolutionary ladder climbing
   and we can expect the same thing from AI.

3. Etc.

We should have a Github repo for these. ? 

Summary

1. I wish these debates were structured like the above instead of like
   Tiger Claw vs. Ancient Swan technique.

2. Hotz’ main argument is that we have time, and I don’t think we do. See
   above.

3. Eliezer’s main argument is that we’re screwed unless we
   limit/control/destroy our AI compute infrastructure. I think he’s likely
   right, but I think he’s missing that it’s really hard to do anything
   well on the first try. And we might have some chances to get aggressive
   if we can be warned by early versions failing.

Either way, super entertaining to see both of them debate. I’d watch more.