Inside the AI Party at the End of the World

A group of AI researchers, philosophers, and technologists gathered in a$ 30 million mansion perched on a cliff overlooking the Golden Gate Bridge to talk about the end of humanity.

The symposium’s theme,” Worthy Successor,” was based on an incisive statement from businessman Daniel Faggella:” You would gladly prefer that it ( not humanity ) determine the future course of life itself.

In his offer, Faggella made the concept obvious. He wrote to me via X DMs that” This celebration is very much focused on posthuman transition.” No on AGI, which is a tool for humanity forever.

It might be referred to as market at a party filled with modern fantasies where guests discuss the end of humanity as a logistical issue rather than a symbolic one. This is a usual Sunday if you live in San Francisco and function in AI.

Before reuniting to speak three talks on the future of brains, about 100 visitors nursed nonalcoholic drinks and gnawed on cheese plates close to floor-to-ceiling windows facing the Pacific Ocean. One participant wore a sweater that read,” Kurzweil was right,” making it seem as though Ray Kurzweil, the revolutionary, had predicted that machines would outsmart humans in the near future. Another wore a shirt that read,” Does this help us get to safe AG I”? accompanied by an icon for” thinking face”

According to Faggella,” the great labs, the people who know that AGI is likely to stop humanity, don’t talk about it because the incentives don’t allow it,” and he made reference to early comments from tech leaders like Elon Musk, Sam Altman, and Demis Hassabis, who “were all very coy about the possibility of AGI killing us all.” He claims that now that the incentives are to compete,” they’re all racing complete bore to develop it.” ( To be fair, Musk still talks about the dangers of developing advanced AI, but this hasn’t prevented him from going further. )

On Linked In, Faggella boasted a star-studded host list that included “most of the essential philosophical intellectuals on AGI,” experts from all the major Western AI labs, and AI founders from all the major Western AI startups.

Ginevera Davis, a writer based in New York, made the warning that it might be difficult for individual principles to be translated into AI. She said that trying to hard-code individual preferences into potential systems may be foolish because machines may not understand what it’s like to be aware. She rather suggested creating Artificial that you search for deeper, more universal values that haven’t yet been discovered under the pseudonym” cosmic alignment,” or” celestial alignment.” A group of people gathered on a grass hill with a modern city in the distance frequently appeared in her presentations, which appeared to be AI-generated.

Big language models are just random parrots, as opposed to the notion that machine awareness is a construct created by a group of researchers, some of whom worked at Google, who claimed in a well-known paper that LLMs are merely stochastic machines. However, that discussion was not a part of the symposium, where speakers assumed that superintelligence was emerging quickly and as a given.

By the second talk, everyone in the audience was fully engaged. The participants scribbled notes while standing cross-legged on the wood floor. We all have an intuition that radical technological change is about to occur, but we lack a grounded framework for dealing with it, especially in terms of human values, according to a philosopher named Michael Edward Johnson. He claimed that developing AI without fully understanding consciousness is a risky gamble if consciousness is” the home of value.” We run the risk of enslaving something that can suffer or trusting something incapable. This idea is heavily contested, and it also relies on a similar premise to machine consciousness. He suggested a more ambitious goal: teaching both humans and our machines to pursue” the good” rather than forcing AI to follow human instructions forever. He didn’t give a precise definition of” the good,” but he said it wasn’t mystical and that it could be described as such.

Faggella finally took the stage. He thinks that humanity won’t last forever in its present form and that we have a responsibility to design a successor that can provide both new and lasting meaning and value. He cited the ability to “autopoiesis” and “autopoiesis,” which states that this successor must possess. He cited philosophers like Baruch Spinoza and Friedrich Nietzsche as saying that the majority of the universe’s value is still unexplored and that our task is not to cling to the past but to create something capable of revealing what lies ahead.

This, he claimed, is at the core of what he called” axiological cosmism,” a worldview in which intelligence is meant to expand the scope of what is possible and valuable rather than just fulfill human needs. He warned that humanity may not be ready for what it is building because the AGI race of today is reckless. However, he claimed that AI won’t just inherit the Earth; it might also inherit the potential for meaning of the universe.

Clusters of guests debbered on topics like the AI race between the US and China during a break between panels and the Q&amp, A. I had a conversation with the CEO of an AI startup and argued that there are other kinds of intelligence in the galaxy. What we’re constructing here is trivial in comparison to what must already exist somewhere other than the Milky Way.

At the conclusion of the event, some attendees poured into Ubers and Waymos while others stayed around to talk. This is not a group that advocates the demonization of man, Faggella said to me. This group is promoting the slowdown of AI development, if anything, to ensure that we are heading in the right direction.