Title:

When AI Systems Begin to Act: Authority, Control and Accountability in Critical Infrastructure

Keynote speaker:

Patrick Upmann, Founder & Research Lead | AIGN Research, AI Governance Advisor

Patrick Upmann is Founder and Research Lead of AIGN Research and works on AI governance, machine authority and the governance of AI-enabled systems in high-consequence environments. His research focuses on how authority delegated to AI-enabled systems can be defined, bounded, controlled at runtime and made reconstructable, particularly in critical infrastructure. He brings more than 20 years of experience across technology, transformation, data and governance, with work spanning sectors including energy, financial services, telecommunications, mobility and other critical environments. In September 2026, he delivered a keynote at a NATO-supported Advanced Research Workshop in Yerevan on governance frameworks for AI in critical sectors, including AI-enabled monitoring and early-warning systems. His current research examines the relationship between machine authority, human accountability, runtime control and evidence in increasingly autonomous and agentic systems. He is the author of several publications on AI governance, including *Defensible Autonomy* and *Sovereign Autonomy – The Authority Supply Chain of the Agentic Age*.

Abstract: 

AI-enabled systems are increasingly moving from analysis and decision support towards systems capable of initiating actions, invoking tools, interacting with other systems and operating with varying degrees of autonomy.

In critical infrastructure, this development raises a question that extends beyond model performance and conventional AI risk management:

Under what conditions should an AI-enabled system be permitted to act without real-time human confirmation, and how can accountability remain identifiable when authority is delegated to software?

The keynote examines this question through the concept of machine authority: the practical ability of an AI-enabled system to initiate actions that may have operational consequences.

It distinguishes between technical capability and authorised action and considers how authority can be specified, bounded and controlled across data access, tools, communications, transactions and physical or digital interventions.

The talk will examine several interconnected governance and dependability challenges: how operational authority is defined; where human confirmation remains necessary; how authority boundaries can be enforced at runtime; what happens when systems encounter conditions outside their authorised scope; and what evidence is required to reconstruct consequential actions afterwards.

Critical infrastructure provides a particularly relevant setting for these questions. Energy, water, transport, telecommunications and other essential systems combine requirements for availability, resilience and rapid response with potentially significant consequences of incorrect or unauthorised actions.

The keynote does not assume that greater autonomy or greater human intervention is inherently preferable. Instead, it proposes that the relevant question is how different levels of delegated authority can be designed and assessed according to context, consequence, time criticality and the ability to maintain effective control and accountability.

The objective is to contribute to a broader discussion on dependable AI: moving from the question of whether an AI system can produce a reliable output towards the additional question of when it should be authorised to act upon it.