Safe Override Mode in AI systems supporting critical national infrastructure is essential to ensure ultimate human control.
O Gartner, Inc..A business insights and technology company predicts that, by 2028, misconfigured Artificial Intelligence (AI) in cyber-physical systems (CPS) will cause shutdowns. national critical infrastructure in a G20 country.
Gartner defines CPS as engineered systems that orchestrate sensing, computing, control, networking, and analytics to interact with the physical world (including humans). CPS is the umbrella term that encompasses operational technology (OT), industrial control systems (ICS), industrial automation and control systems (IACS), Industrial Internet of Things (IIoT), robots, drones or Industry 4.0.
“"The next major infrastructure failure may not be caused by hackers or natural disasters, but rather by a well-intentioned engineer, a script "It could be due to a faulty update or an incorrectly positioned decimal," he says. Wam Voster, Vice President Analyst at Gartner. “A secure emergency shutdown mechanism (kill switch) or override mode accessible only to authorized operators is essential to protect the national infrastructure from unintentional shutdowns caused by misconfiguration of AI.”
THE Artificial intelligence A poorly configured system can autonomously shut down vital services, misinterpret sensor data, or trigger unsafe actions. This can result in physical damage or large-scale service disruptions, posing direct threats to public safety and economic stability by compromising the control of essential systems such as power grids or industrial plants.
Modern power grids, for example, rely on AI to balance generation and consumption in real time. A poorly configured predictive model can misinterpret demand as instability, unnecessarily triggering grid isolation or load shedding in entire regions or even countries.
“Modern AI models are so complex that they often resemble ‘black boxes’,” says Voster. “Even developers can’t always predict how small changes in configuration will affect the model’s emergent behavior. The more opaque these systems become, the greater the risk posed by a misconfiguration. Therefore, it’s even more important that humans can intervene when necessary.”
To mitigate the risks, Gartner recommends that Chief Information Security Officer (CISOs) must:
– Implement safe overlay modes: For all critical CPS infrastructures, include a safe emergency shutdown mechanism (kill switch) or other override mechanisms accessible only to authorized operators, so that humans retain ultimate control, even during full autonomy.
Digital twins: To develop a full-scale digital twin of the systems that support critical infrastructure for realistic testing of upgrades and configuration changes before deployment.
– Real-time monitoring: Require real-time monitoring with rollback mechanisms for changes made to Artificial Intelligence in CPS, also ensuring the creation of national AI incident response teams.
Gartner customers can learn more at “Predicts 2026: Emergent Critical Risks of AI in CPS Security”Learn how to navigate the path to infrastructure modernization in roadmap Gartner's free guide,“Modern Infrastructure: Built for Tomorrow's Business Demands”.
Gartner is the world authority on AI.
Gartner is the indispensable partner for C-level executives and technology vendors as they implement AI strategies to achieve their mission-critical priorities. The independence and objectivity of Gartner's insights provide clients with the confidence needed to make well-informed decisions and unlock the full potential of AI. C-level clients are using Gartner's AI tool. AskGartner to define how to leverage AI in your business. With over 2,500 business and technology experts, 6,000 written insights, plus more than 1,000 AI use cases and case studies, Gartner is the world authority on AI. More information can be found [here/at this link]. on here.
About the Gartner Security & Risk Management Conference
Gartner analysts will present the latest insights for security and risk management leaders during the Gartner Security & Risk Management Conferences, which will be held on March 9th and 10th in [location missing from original text]. Mumbai (India), March 16 and 17 Sydney (Australia), June 1-3 National Harbor (United States), July 22-24 Tokyo (Japan), August 4 and 5 Sao Paulo (Brazil) and September 22-24 in London (United Kingdom). Follow the news and updates from the conferences at X and LinkedIn using #GartnerSEC.
About Gartner for Cybersecurity Leaders
Gartner for Cybersecurity Leaders equips security leaders with the tools to help reshape roles, align security strategy with business objectives, and build programs to balance protection with business needs. Additional information is available at https://www.gartner.com/en/cybersecurity/products/gartner-for-cisos.Follow the news and updates from Gartner for Cybersecurity Leaders on X and LinkedInusing #GartnerSEC.
About Gartner
O Gartner, Inc. delivers objective, actionable insights that drive smarter decisions and better performance for enterprises’ mission-critical priorities. To learn more, visit www.gartner.com.
