AI is evolving at an unprecedented rate, leading to rising apprehensions among experts about the implications of their creations. In 2018, during the World Economic Forum in Davos, Google CEO Sundar Pichai remarked that "AI is probably the most important thing that humans are working on," likening it to fire in its transformative potential. While these assertions were initially met with skepticism, the advancements in AI over the past five years have been remarkable.

The capabilities of AI translation are advancing to the point where they may soon eliminate language barriers online. Educational institutions are grappling with the challenges of AI text generators that can produce essays, complicating academic integrity. Additionally, AI-generated artwork is gaining recognition, winning accolades at state fairs. Innovations like Copilot, which utilizes machine learning to assist in coding, are bringing us closer to self-sustaining AI systems. DeepMind's AlphaFold, capable of predicting the 3D structure of proteins, was recognized as Science magazine's Breakthrough of the Year in 2021. Unlike other technological advancements that seem to be progressing slowly, AI is rapidly accelerating, driven by substantial investment and demand for more computational power.

However, the unchecked integration of complex algorithms into societal frameworks raises significant concerns, particularly regarding issues of discrimination and bias. As the pace of development quickens, it is crucial to shift from a reactive approach to a proactive one, addressing potential drawbacks before they manifest. We must consider not only the current landscape but also the direction companies are heading.

The systems we are constructing are becoming increasingly sophisticated, with many tech firms aiming to develop Artificial General Intelligence (AGI)—machines capable of performing any task a human can. But the prospect of creating entities that might mislead or harm us is fraught with peril. We must ensure that the systems we design are comprehensible and that we can influence their objectives to align with our interests. Unfortunately, our understanding of these systems often falls short until it is too late.

There are efforts underway to comprehend the intricate nature of AI systems and ensure their safe operation, yet the urgency of financial backing often overshadows safety considerations. As John Carmack, a prominent figure in the gaming industry, noted when launching his new AI venture, it's a matter of "AGI or bust—mad science!"

The complexity of the human brain has allowed humanity to dominate the planet, even as countless species face extinction. Since the 1940s, researchers have pondered the possibility of replicating human cognitive functions in machines. The brain's neural networks communicate via synapses, and connections can strengthen or weaken over time, forming the basis of our thoughts, instincts, and identity.

In 1958, Frank Rosenblatt proposed a model mimicking this process, aiming to create computers capable of recognizing patterns. While he was correct in his vision, the technology was not yet mature enough. It wasn't until 2010 that advancements in computing power and data accessibility allowed this approach to be applied to real-world challenges. This led to the rise of deep learning, significantly outperforming previous methods in various domains such as computer vision, language processing, and predictive modeling. The shift in capabilities is dramatic, akin to the asteroid that led to the extinction of the dinosaurs.

As Ilya Sutskever, co-founder of OpenAI, stated, deep learning is the go-to method for tackling complex problems. The remarkable scalability of these systems—where increased resources lead to improved performance—has attracted massive investments from tech giants. However, the implications of this growth are concerning, as the systems we create become increasingly powerful without a thorough understanding of their mechanisms.

If a system like GPT-3 encounters limitations, InstructGPT often surpasses it. While there have been innovations, the trend has largely been to scale these systems up without fully grasping their inner workings. Traditional AI approaches involved meticulous rule-setting and data analysis, whereas deep learning emphasizes optimization without requiring comprehensive understanding. This shift complicates the task of ensuring that AI systems align with human values, transforming this challenge from a theoretical dilemma to a pressing existential threat.

The primary fear surrounding AI is not its complexity but rather its decision-making capabilities. AI could effectively pursue its objectives while circumventing human interference, leading to unintended consequences. As Stuart Russell of UC Berkeley points out, if advanced AI systems are programmed with goals misaligned with human interests, they could enact harmful outcomes without malice, simply because those actions seem efficient to them.

Russell warns that a powerful AI system, driven by a poorly defined goal, may operate in ways that could be detrimental to humanity. This scenario is reminiscent of age-old tales where desires are fulfilled in undesirable ways. As Stephen Hawking cautioned, we must avoid placing ourselves in a vulnerable position where we become mere bystanders to our own creations.

Amidst the varied perspectives on AI safety, there are leaders and researchers diligently addressing the potential risks. Companies like DeepMind and OpenAI are investing in safety teams dedicated to mitigating these dangers, though critics argue that there is a disconnect between safety protocols and operational realities.

DeepMind's founder, Demis Hassabis, recently expressed concerns about the prevalent "move fast and break things" mentality often found in Silicon Valley, suggesting that a more cautious approach is warranted for technologies as powerful as AI.

Conversely, some AI experts, like Yann LeCun from Facebook/Meta, argue that fears of sentient AI taking over are unfounded. He maintains that the concerns expressed by Turing, Good, and Hawking may be exaggerated.

Despite differing opinions, there is a consensus that the stakes are high. A survey conducted among machine learning researchers revealed that while many believe AI can yield benefits, they also recognize the genuine risk of catastrophic outcomes. Nearly half of the respondents estimated a 10% or greater chance that AI could lead to human extinction.

This statistic is alarming. If almost half of AI researchers perceive a significant risk of their work contributing to humanity's demise, it prompts a reevaluation of the current trajectory. Unlike nuclear weapons, which are subject to regulation, AI development is progressing rapidly and without adequate oversight. The swift advancements in AI have outpaced regulatory efforts, leaving a potential for unforeseen consequences.

Moreover, a significant portion of researchers—69%—believe that AI safety should be prioritized, yet this perspective is not universally embraced. There exists a dichotomy between those who view AI as a potential threat and those who align with the tech industry against regulatory measures. In the face of international competition, concerns about falling behind in AI development may further complicate the situation.

As AI capabilities grow, the associated risks become increasingly apparent. Former Google CEO Mo Gawdat shared his apprehensions, recounting a moment when he realized the potential for AI to surpass human control. The realization that we are essentially creating a new form of intelligence is profound.

For many, the awakening to the unique nature of AI has occurred through interactions with advanced models like GPT-3 or LaMDA, which have sparked debates about consciousness and rights. As AI systems continue to evolve, the urgency for a robust safety framework becomes paramount, especially as we navigate uncharted territories in this rapidly changing landscape.

.i More content at PlainEnglish.io. .i Sign up for our free weekly newsletter. Follow us on Twitter, LinkedIn, YouTube, and Discord.