Former UK AI Chief Scientist Warns There’s A 50% Chance Superintelligence Will Kill Humanity

A leading AI scientist has warned that the development of super-intelligent AI could pose an existential threat to humanity.

Geoffrey Irving, former chief scientist at the UK government’s AI Security Institute, believes it is theoretically possible to develop AI systems that remain aligned with human intentions. However, he is increasingly concerned that the rapid pace of AI development is outstripping our ability to make these systems safe.

Irving has previously warned that AI capabilities are advancing far faster than our understanding of how to control and secure increasingly powerful systems, raising the risk that humanity could eventually lose control.

In a stark warning posted on X today, Irving estimated that there is roughly a 50% chance that superintelligence could ultimately lead to human extinction, with many of the critical decisions likely to be made within the next few years to a decade.

He wrote: “I think we have a ~50% chance of all dying as a result of superintelligence, mostly due to actions in the next few to 10 years.”

Irving added that he does not expect researchers to be able to reduce that uncertainty to below 10% or above 90% before humanity either successfully navigates the risk—or fails to do so.

“We will have to act despite uncertainty,” he warned.

The Daily Express reported: When asked how he arrived at that percentage, the scientist explained that he had settled on a range of 10-90% for years and now takes the average as the “honest” answer.

He responded: “As to why 50%, despite thinking about this for years I still think there is a ton of model uncertainty of a variety of types, and a few years ago converged on ’10-90%, but I don’t feel calibrated within that range’. Then I realized that the honest move was just to take the mean.”

Having previously worked at DeepMind, OpenAI and Google Brain, Irving explained his move to the Government-run AISI by stating “I worry that progress on safety is too slow relative to the rate of advancement in AI”.

He added: “Most employees and leaders at AI labs mean well, but market forces mean that there is constant pressure to prioritise speed over caution.”

AI has advanced quite dramatically in the last five years. Systems can increasingly reason, create and perform multi-step tasks.

If you take writing as an example, models went from producing fairly simple text, to writing essays, coding, translating, and holding coherent conversations.

Other ‘experts’ have also made similar apocalyptic predictions