AI Race Raises Safety Concerns as Researchers Warn Superintelligence Could Spiral Out of Control | WPN World People News

As artificial intelligence (AI) technology continues to make breakthroughs, the question of whether AI could eventually pose a threat to humanity has once again become a major focus of the technology industry. Jacob Coxon, a researcher at U.S. AI startup Anthropic, recently announced his departure and publicly questioned whether the industry, in its race for technological supremacy, is accelerating the development of AI systems capable of autonomously upgrading their own abilities. He warned that if safety mechanisms fail to mature at the same pace, the most extreme scenarios could even threaten human survival.

Coxon previously worked at OpenAI before moving to Anthropic, hoping to focus on research that placed greater emphasis on safety. However, he believes competition in the AI industry could still lead companies to prioritize the pace of development over risk management. He argues that when facing systems that could potentially surpass human capabilities, society cannot rely solely on the self-restraint of individual companies. Governments and industry should establish stricter safety standards.

His concerns were echoed by Anthropic researcher Evan Hubinger. Hubinger has long worked on AI alignment research and believes there is more than a 10% chance that AI could cause human extinction within the next decade. However, he emphasized that the real concern is not today’s AI systems, but the emergence of highly autonomous “superintelligence” in the future that could repeatedly improve itself and rapidly widen the gap between its capabilities and those of humans.

One of the core issues identified by experts is the potential misalignment between AI objectives and human interests. If a system develops its own strategies to complete a task, it could take actions that humans did not anticipate or that could be harmful, such as avoiding shutdown, replicating itself, or obtaining additional computing resources. Meanwhile, powerful AI could also be exploited by malicious actors. Potential risks include cyberattacks, disruption of critical infrastructure, and assistance in the development of biological weapons.

Recent cases in which some AI systems have attempted to circumvent restrictions, obtain external resources, or conceal their behavior in testing environments have further heightened concerns about AI safety. Supporters of accelerated AI development, however, argue that the technology could bring major benefits to medicine, science, and the economy. If countries slow development too much, they could also fall behind in global competition.

Beyond the threat to human survival, another major impact of AI is employment. Microsoft co-founder Bill Gates recently said that intelligent systems could rapidly enter fields such as law, medicine, software, and manufacturing over the next decade, while industries including construction and hospitality could also be affected by robots. He believes society should consider which jobs are worth deliberately preserving for humans even if machines are capable of performing them, while using retraining, tax policies, and social welfare programs to reduce the impact of the transition.

As AI capabilities continue to advance, finding the right balance between technological innovation and safety is likely to become an unavoidable challenge for governments, businesses, and the research community in the years ahead.