As concerns about AI filled the headlines in the past few weeks, I decided to delve into the immediate controversy a bit. I was lucky enough to find this video/podcast with the instigator of the now famous March 28th letter calling for an immediate, six month halt in the development of large language model advanced AI. After listening to it, I changed my mind about the urgency of this question.
For the record, I think the headlines about losing jobs, or as Henry Kissinger (yes, of all the skeletons to reappear at 100 years old!!) writes the risk of nuclear war as in the film WarGames, ask the wrong question.
The basic notion is I want to discuss is summarized by Max Tegmark, MIT professor of physics and AI, this way,
This is the most important fork that humanity has reached in its over 100,000 years on this planet.
We're building effectively a new species that's smarter than us. It doesn't look so much like a species yet 'cause it's mostly not embodied in robots. But that's the technicality which will soon be changed.
Tegmark makes the point that AI will not destroy humanity as in the notion that it will attack us directly like the Terminator nightmare. Instead, humanity will be wiped out, incidentally, as AI outcompetes us for space/resources on this planet. AI will barely be aware of us while it ignores us in the same way we ignored other species as we dominated the biosphere, denying them resources and even space to roam and breathe.
Both Tegmark and Eliezer Yudkowsky, another scientist who is widely regarded as a founder in the field, fear the extinction of humanity from AI. Both have mentioned they wonder if their children will have the chance grow up. From their viewpoint, the tipping point is imminent. It could be reached at any moment.
Yes, we are not talking about poverty from the loss of jobs or WWIII, we are talking about the destruction of human civilization.
Lest you think I am alarmist, I want to make it clear that I believe that there are any number of current conditions which could extinct us. Climate catastrophe being the chief threat. What I see is a concatenation of threats, all coming to a head now, something I did not think I would live to see as I am of the age when a natural death could come at any time. Everything is moving much faster than even my doomsday fantasies.
My mentor, Donna Haraway, wrote the now world famous essay, “A Manifesto for Cyborgs,” in 1985. And I have written my own take on how science fiction has been engaging with the theme by reimagining the human body. I am no Luddite. Like Donna, I embrace the penetration of technology into the human body. We must reimagine our monsters. Donna has stated we must write new myths for our cyborg bodies.
However, runaway AI is another beast entirely. We need to take it seriously, and we need to do so now.
Before I ask you to watch Max Tegmark ring the alarm bell about “superintelligence,” I would like you to understand his use of the term Moloch. It derives from an essay by Scott Alexander which riffs on Alan Ginsberg’s poem, “Moloch.”
What’s always impressed me about this poem is its conception of civilization as an individual entity. You can almost see him, with his fingers of armies and his skyscraper-window eyes.
A lot of the commentators say Moloch represents capitalism. This is definitely a piece of it, even a big piece. But it doesn’t quite fit. . .
Moloch is introduced as the answer to a question – . . .– what does it? Earth could be fair, and all men glad and wise. Instead we have prisons, smokestacks, asylums. What sphinx of cement and aluminum breaks open their skulls and eats up their imagination? [my emphasis]
And Ginsberg answers: Moloch does it. Moloch is an almost mathematical force, something, some power which catches us from behind.
As Tegmark uses it, moloch is an inherent principle which nips at our heels. (This is the best I can do with the time I have available to pursue Moloch as Moloch pursues me, i.e. the time I have left to finish this essay, the best I can do with Moloch and this posting overall as medical issues ate into most of my time this week).
The following is a clip from an almost two hour video/podcast with Max Tegmark, the originator of the March 28th letter asking that higher AI development be halted for six months. This clip addresses the connection between the Moloch of capitalism and the Moloch of AI. (A rough transcript is below.)
Transcript:
You brought up capitalism earlier. There are a lot of people who love capitalism and a lot of people who really, really don’t. And it struck me, recently that what is happening with capitalism is exactly analogous with the way that superintelligence might wipe us out.
So, I studied economics for my undergrad. I was interested in how you could use market forces to get stuff done more efficiently, but give the right incentives to the market so it wouldn’t do really bad things.
Dylan, et. al. ? wrote this interesting mathematical paper where they proved that if you take one goal that you optimize for, what basically always happens is that in the beginning it will make things better for you, but if you keep going, at some point, it is going to start making things worse for you again. And then gradually it is going to make it really, really bad.
. . .
My summary: When you take one goal and optimize for that goal, you will go very wrong in the end. (i.e. Moloch comes out from the shadows)
. . .
Pretty much all AI systems optimize. . .and it is almost impossible to give AI exactly the right directions to optimize.
Capitalism is exactly like that. We wanted to get things done efficiently, so we introduced the free market. Things got done much more efficiently than they did in say, communism, and ‘it’ got better, but then it just kept optimizing and kept optimizing and then people realized that we were optimizing a bit too much, e.g. why did we just chop down half the rainforest? It is just the same optimization that has been running for too long.
If you have an AI which starts optimizing the world, in the beginning things will get better and people will say, “Yeah, things are getting better.” But it is almost impossible to give AI precisely the correct direction to optimize in, and then, eventually, all hay breaks loose.
The free market got things optimized, but when it runs for too long. The issue with capitalism and the issue with AI have kind of merged now.
The Moloch I talked about is exactly the capitalist Moloch. We have built an economy that has optimized for only one thing, profit. And that worked great back when things were very inefficient and now things are getting done better and as long as the companies were small enough that they couldn’t capture the regulators. But that is not true anymore, but they keep optimizing.
So they realize that these companies can make even more profit by building ever more profitable AI even if it is reckless. So this is Moloch again showing up.
Anyone who has any concerns about late stage capitalism having gone a little too far1, you should worry about super intelligence cause it is the same villain in both cases. It is Moloch.
We have to pause from time to time, look into our hearts, and ask, why am I doing this. . .maybe we should change direction.
[a discussion about why six months which is only a step]
It will get better and better and better and boom then we all lose.
. . .
i.e. Humanity will have invented a technological species which out competes us on planet earth. I find Tegmark very convincing if for no other reason that he interpolates capitalism into his analysis. Anyone who can see that capitalism has reached a natural end in that it will only do more and more damage, “make things really, really bad,” as it plays out gets my attention.
What say you to this proposition? I have to take warning from the greatest minds working on this technology seriously. This week, Geoffrey Hinton, has quit his job at Google in order to be free to warn of AI’s dangers.
We all need to have this discussion, now, not later.
To sign the letter asking for a 6 month moratorium on the development of high level AI: futureoflife.org/…
Pause Giant AI Experiments: An Open Letter
We call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. time.com/…
. . .
Max Tegmark’s website at MIT: space.mit.edu/…
Introducing Chat GPT
GPT-4
Addendum:
Chat GPT is available for free. It seems as if the general public is being given access to it in order to test out its capacities at the same time our input builds those capacities.
In order to illustrate what answers we might get from GPT-4, I asked it to summarize Karl Marx’s theories. Here is the result:
Karl Marx was a German philosopher, economist, and political theorist who developed a comprehensive analysis of capitalism and its impact on society. His theories were influential in the development of socialism, communism, and labor movements worldwide. Here is a brief summary of some of his key ideas:
- Historical Materialism: Marx believed that the history of humanity is characterized by class struggles. He argued that economic and social relations determine the development of society, and that changes in economic systems lead to changes in social structures.
- Labor Theory of Value: Marx believed that the value of a product is determined by the amount of labor that goes into producing it. He argued that the capitalist system exploits workers by paying them less than the value of their labor.
- Surplus Value: Marx believed that capitalists extract surplus value from workers by paying them less than the full value of their labor. This surplus value is then used to generate profits for the capitalist class.
- Alienation: Marx believed that capitalism alienates workers from their work, from each other, and from their own humanity. He argued that under capitalism, workers are reduced to mere cogs in a machine, with no control over the products of their labor.
- Class Struggle: Marx believed that class struggle is the driving force behind historical change. He argued that the interests of the working class are fundamentally opposed to those of the capitalist class, and that this conflict will ultimately lead to the overthrow of capitalism.
- Communism: Marx envisioned a classless society in which the means of production are collectively owned and controlled by the workers. He believed that such a society would be free from exploitation, alienation, and oppression.
What say all of you about this summary that AI produced in seconds?
. . .
The complete interview with Tegmark which I recommend when you have time is here:
Via Apple: podcasts.apple.com/…
***
1 Admittedly, most of us thought early stage capitalism went too far!!