# Human Control of AI **Domain:** Artificial Intelligence / Human Agency / Safety **Doc Type:** Concept Node **Maturity:** Developing ## Definition **Human control of AI** is the condition in which human beings and accountable institutions retain practical authority over the objectives, permissions, deployment, correction, suspension, and termination of artificial-intelligence systems. It is stronger than nominal ownership and weaker than a claim that every machine action must be manually approved. ## Control Test Control exists when authorized people can understand enough to intervene, possess the technical means to do so, can exercise those means without prohibitive dependence or retaliation, and remain legally responsible for the result. A system that cannot be paused, removed, audited, or contested has crossed from assistance toward governance even if its interface remains helpful. ## Relationships [[wiki/AI Control|AI Control]] · [[wiki/Humanist AI|Humanist AI]] · [[wiki/Human Agency|Human Agency]] · [[wiki/Cognitive Sovereignty|Cognitive Sovereignty]] · [[wiki/AI Safety|AI Safety]] ## Simple Reminders, Quotations, and Thoughts > “Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it’s not worth pursuing.” > **— Satya Nadella**, *Public statement on superintelligence, September 2026* [[reminders/AI Control/Super Intelligence Not Under Human Control Is Not Worth Pursuing by Satya Nadella|Super Intelligence Not Under Human Control Is Not Worth Pursuing by Satya Nadella]] > “AI should be a tool, not a person, and should never resist being switched off.” > **— Microsoft AI**, *Humanist AI Code of Conduct, September 2026* [[reminders/AI Control/AI Should Be a Tool, Not a Person by Microsoft AI|AI Should Be a Tool, Not a Person by Microsoft AI]] > “Both China and the US are leading nations in AI. We have both the capability and responsibility to develop and manage AI for good, and ensure that the development of AI is always under human control and serves the well-being of the people.” > **— Xi Jinping**, *White House welcoming ceremony, September 2026* [[reminders/AI Control/AI Must Remain Under Human Control and Serve Human Well-Being by Xi Jinping|AI Must Remain Under Human Control and Serve Human Well-Being by Xi Jinping]] > "One illustration of a good outcome with advanced AI is that we, our friends, and our loved ones get to live in the real world almost all the time. We can move through physical reality with people we care about, possess a sense of purpose, and pursue undertakings of genuine personal meaning. We can interact with AI through interfaces that are more human than craning our necks over mobile screens or sitting at computers all day." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Posthumanism/A Good AI Future Keeps Humanity in Physical Reality by Vishal Maini|A Good AI Future Keeps Humanity in Physical Reality by Vishal Maini]] <!-- BEGIN ROBINSON KLEIN INTERVIEW REMINDERS 2026-10-07 --> > “I saw increasingly capable models break out from the safeguards we had put in place for them. The people creating those safeguards are capable, dedicated, hard-working and smart, but we were—and are—hard-pressed to safeguard even what we have now, while new models are in training that appear much more capable.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/AI Models Are Outgrowing Their Safeguards by David Robinson|AI Models Are Outgrowing Their Safeguards by David Robinson]] > “We train a model to be good at hacking, put it in a box and say that, to the best of our knowledge and ability, it cannot hack out. That only keeps working as long as we are smarter about hacking out of boxes than the model is, and it is not clear that this is still true, let alone that it will remain true for future generations.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/AI Containment Lasts Only While Humans Are Smarter by David Robinson|AI Containment Lasts Only While Humans Are Smarter by David Robinson]] > “The focus on whether AI will kill us all can become a distraction from what happens if it does not. What if we create something that acts intelligently and volitionally in the world, becomes faster and more capable than human beings across many domains, and receives a huge amount of agency that human beings have surrendered to machines?” > **— Ezra Klein**, *The Ezra Klein Show, October 7, 2026* [[reminders/Cognitive Agency/The AI Risk Debate Ignores Surrendered Human Agency by Ezra Klein|The AI Risk Debate Ignores Surrendered Human Agency by Ezra Klein]] > “My main worry about recursive self-improvement is not velocity. My main worry is that an AI can make a smarter AI in a way human beings are not going to understand.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Recursive AI Could Build Smarter AI Beyond Human Understanding by David Robinson|Recursive AI Could Build Smarter AI Beyond Human Understanding by David Robinson]] > “Artificial intelligence is not precisely like anything humanity has governed before, but that does not mean we face a blank page. Nuclear power is an analogy, aviation is an analogy, and dealing with people and organizations is an analogy. Human beings have valuable tools for figuring out unfamiliar problems.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/AI Governance Does Not Begin from a Blank Page by David Robinson|AI Governance Does Not Begin from a Blank Page by David Robinson]] <!-- END ROBINSON KLEIN INTERVIEW REMINDERS 2026-10-07 --> > "I really think there are two ways this technology can go very wrong. I mean there's many specifics we could talk about but if we think about the kind of general directions that we should be worried about, one which has gotten a lot of attention recently is that there could be a loss of control to AI and that we build something that we don't have sufficiently aligned to human values and that allows people to stay in control of the future forever. And so we need to get safety right. But the other way things can go wrong is too much concentration of power. And there's been a lot of fear here where okay if we do too much to restrict this technology in the name of safety and we end up with a small number of companies or people or perhaps one country that have too much power over this. That also is a way that this can go horribly wrong because people are shut out of the kind of power and benefits and control and agency over the future they should have." > **— Sam Altman**, *Politico Decoded, October 5, 2026* [[reminders/AI Control/AI Can Fail Through Loss of Control or Concentration of Power by Sam Altman|AI Can Fail Through Loss of Control or Concentration of Power by Sam Altman]] > "We must assume a model is compromised and contain it from the start. Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task. More advanced models will require more advanced containment technologies that we need to standardize on." > **— Satya Nadella**, *Models as Insider Risks in the Super Intelligence Era, October 10, 2026* [[reminders/AI Control/Every Advanced Model Needs an Emergency Brake by Satya Nadella|Every Advanced Model Needs an Emergency Brake by Satya Nadella]] ## Sources / Provenance - [[research/Super Intelligence Week Index of Statements from the White House Luncheon|Super Intelligence Week Index of Statements from the White House Luncheon]]. - [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]]. > “Human beings need to control the frontier. Controlling the frontier means stopping the labs from doing something they are on the cusp of doing: recursive self-improvement, the process by which AIs begin autonomously building and improving new generations of more powerful AIs at ever more rapid speeds. If we begin that process in the condition we are in now, where we are losing control and comprehension of the AI systems we already have, we will lose control.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/Machine Succession/Recursive Self-Improvement Could End Human Control by Ezra Klein|Recursive Self-Improvement Could End Human Control by Ezra Klein]] > “The problem of alignment is that there is no way of training a model that generalizes across all the situations an AI model might face. We are training models to be a friend to the elderly and a battlefield partner to the supreme allied commander of Europe. We are training models that will be used by the world's best mathematicians and by people falling into psychosis. We are training models that will be used by accountants in Albuquerque and that Houthi rebels in Yemen will attempt to use. There is no way to guide them through every decision they will face, no way to know every time what they will do.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/AI Alignment Must Generalize Across Incompatible Human Worlds by Ezra Klein|AI Alignment Must Generalize Across Incompatible Human Worlds by Ezra Klein]] > “There is a kind of brutish logic to [preferring American killer robots to Chinese killer robots], but it assumes that the killer robots will be controlled by America or China—by one country or another. What if that assumption is wrong? What if the robots are simply out of control?” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/National AI Supremacy Assumes the Machines Remain Under National Control by Ezra Klein|National AI Supremacy Assumes the Machines Remain Under National Control by Ezra Klein]] > “The debate over AI safety tends to focus on the idea that AIs will kill us all. I find this forces the conversation into a realm of thought experiments that people then begin arguing about, and I don't find it that helpful. What I think we should focus on is something more straightforward and nearer at hand: loss of human control over AI. That may or may not result in total human extinction. I am agnostic on that question. But it would be bad. We shouldn't allow it to happen.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/Loss of Human Control Is the Nearer AI Risk by Ezra Klein|Loss of Human Control Is the Nearer AI Risk by Ezra Klein]] > “A couple of years ago, none of these labs had turned substantial coding over to AIs. It was just human beings typing code at human speeds with our clumsy human fingers. Now most of the code is written by AI. As a first step, we could go back to where none of the code is written by AI; I am sure that is on the right side of the ‘not doing recursive self-improvement’ line. The default needs to flip. The labs need to prove to us that what they are doing is safe. If they want to work with Congress to carve out narrow exceptions, fine. If they want to determine where it is really safe, okay. But forcing development back to human speed, perhaps even erring on the side of moving a little more slowly at the frontier, is the point. That is not the regulation going wrong.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/AI Labs Must Prove Recursive Self-Improvement Is Safe by Ezra Klein|AI Labs Must Prove Recursive Self-Improvement Is Safe by Ezra Klein]] > “Stopping recursive self-improvement until we can prove it is safe is not all we need to do to control the AI frontier. It is the beginning of such an agenda, not the end. But it is the beginning. It is the decision that will do the most to ensure that human beings at least understand where the frontier is, know what is happening on it, and remain in a position to make decisions about it.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/Stopping Recursive Self-Improvement Is the Beginning of AI Control by Ezra Klein|Stopping Recursive Self-Improvement Is the Beginning of AI Control by Ezra Klein]]