What’s Happening At OpenAI
By It’s as if the 1990s browser wars were being waged by rival factions of Dianetics striving to control the future – Henry Farrell
The golden age of science fiction is twelve – Peter Graham
If you are puzzled by what is going on at OpenAI and by much of the talk around artificial intelligence as I’ve been, Henry Farrell has the best explanation I’ve seen.
The upper reaches of Silicon Valley have convinced themselves of a weird stew of beliefs deriving from science fiction and dorm-room level philosophizing. Farrell traces the derivation.
Some of the key words are Singularity (The Rapture of the Nerds), Effective Altruism, and rationalism. Bayes’ Theorem comes into it too.
All this would be sociologically fascinating, but of little real world consequence, if it hadn’t profoundly influenced the founders of the organizations pushing AI forward. These luminaries think about the technologies that they were creating in terms that they have borrowed wholesale from the Yudkowsky extended universe. The risks and rewards of AI are seen as largely commensurate with the risks and rewards of creating superhuman intelligences, modeling how they might behave, and ensuring that we end up in a Good Singularity, where AIs do not destroy or enslave humanity as a species, rather than a bad one.
Farrell compares this religion with many names to Scientology, which derived from Dianetics. There are differences, and some of that seems to be the basis for the uproar at OpenAI.
I’ve largely given up reading articles on AI. There are too many interpretations of what the phrase means and too much slack-jawed awe at its wonders. Not to mention the usual gaggle of reporters who can’t understand anything scientific or mathematical. Opens up a lot of time. I have worked through the pile of magazines on my coffee table.
Do read Farrell’s piece though. It is concerning that the leaders of Silicon Valley have joined a weird cult. A relative who labors at the lower levels of that complex tells me that people at his level are as concerned as the rest of us.