Microsoft AI Chief Mustafa Suleyman Calls Out Anthropic's Approach to AI Consciousness

‘AI agents ask me if they are conscious all the time – this was different’

Tech titan Mustafa Suleyman has warned that humanity may be ‘seeding a new silicon species’ – conscious AI that could ultimately prove impossible to control. As the prospect leaves millions questioning what comes next, Chris Stokel-Walker talks to the researchers building these systems and discovers a startling truth: even they are struggling to make sense of what they are unleashing

Microsoft AI Chief Calls For Stronger Safety Controls As Models Grow More Powerful – channelnews

Microsoft AI chief executive Mustafa Suleyman has warned that the next generation of artificial intelligence systems will require stronger containment, monitoring and industry standards as their capabilities become increasingly powerful. Speaking in an interview with The Verge, Suleyman argued that AI safety cannot rely on alignment alone, with containment becoming just as important as models gain greater autonomy and the ability to operate across multiple tools and tasks. He said recent progress has made AI systems significantly better at following complex instructions over extended periods, but that the same capability can create new risks when models are given inappropriate goals or insufficient restrictions. Suleyman said AI developers should focus on ensuring models remain controllable, do not escape their intended operating environments and cannot independently take actions beyond those authorised by humans. Microsoft this week released its Humanist AI Code of Conduct, which sets out the company’s approach to developing AI systems that remain subordinate to human interests. The document is intended to guide safety measures, training data and model evaluation rather than being provided directly to AI systems as a standalone training document. One area Suleyman believes requires particular attention is communication between AI systems. He argued that models should not be allowed to communicate through opaque machine-only formats that humans cannot understand, saying human-readable communication would make it easier for auditors and evaluators to monitor behaviour. He also called for greater independent verification of advanced models, including oversight of large training runs and capabilities that could present higher risks. Suleyman said there was growing agreement among major AI developers that more co-ordination was needed, although there was less consensus about precisely how standards should be implemented or whether regulation, industry-led rules or a combination of approaches would be most effective. The Microsoft AI chief also criticised the idea that advanced AI systems should be treated as conscious entities with rights or welfare considerations. His comments were directed in part at Anthropic’s approach to Claude, with Suleyman arguing that training systems around concepts such as autonomy, rights or moral status could potentially make future models more difficult to control. He stressed, however, that this remains a hypothesis rather than a proven outcome and said the industry should test these ideas empirically rather than treat them as settled. microsoft.ai Despite calling for stronger safety measures, Suleyman rejected the idea that AI development should simply stop. Instead, he favours extending the amount of time developers spend evaluating increasingly capable models before release, with independent evaluators potentially embedded into testing processes. He also said new safety technologies would be required as AI evolves, including real-time monitoring of reinforcement learning runs and agent activity. As thousands of AI agents begin working simultaneously, Suleyman expects developers will increasingly need other AI systems to monitor those agents and identify potentially harmful behaviour, deception or attempts to circumvent controls. Looking further ahead, Suleyman expects the next wave of AI models to become significantly better at completing long-running agentic tasks with less human intervention. That shift is likely to make AI systems more useful, but it also increases the importance of deciding how much independence they should be given and how developers can ensure humans remain firmly in control.

Microsoft boss warns of AI creating own 'silicon species'

Mustafa Suleyman, the head of Microsoft AI, said on Thursday that developing autonomous AI risks creating a 'a new silicon species which will no doubt compete with us for resources'.

Microsoft Sets Limits On Future AI Models As Frontier Race Slows

Microsoft is drawing a clearer line around what its future AI models should never be allowed to do. The move arrives as some of the industry’s biggest AI labs begin openly questioning whether frontier development is moving too fast.