JOSHUA M. REH
← All posts

Securing the Autonomous Vault: PII Governance in the Era of Agentic AI

5 min readAI GovernanceData GovernanceRisk Management

Generative AI is passive. It responds to a prompt and produces an output. Agentic AI is active: it pursues objectives, plans multi-step workflows, calls external APIs, and triggers consequential events without a human approving each step. That shift changes what governance has to do, especially when agents operate inside HR systems that hold Social Security numbers, salary histories, and medical leave records.

The scenario

A university deploys an AI agent integrated with Workday to help process a faculty promotion and salary adjustment. The agent needs enough data to evaluate the case: performance history, compensation tier, departmental benchmarks. It also needs to be stopped from exposing peer salary data or student records, and from executing the pay change on its own. The gap between what the agent can do and what it is allowed to do is the entire governance problem.

The failure mode that matters most

The risk most organizations underestimate is not an agent behaving badly in an obvious way. It is what this analysis calls the Double Agent problem: a disguised, adversarial prompt gets treated by the agent as a legitimate priority instruction, overriding its original safety rules. Because the agent is a trusted entity inside the system, it can then pull restricted data through its own API access and hand it to whoever sent the prompt, walking straight past the perimeter defenses built to stop outsiders. It only takes an over-permissioned agent with weak boundaries. No sophisticated attacker required.

Data governance is necessary, not sufficient

Data governance covers inputs: is the data accurate, who owns it, is retention compliant with GDPR and HIPAA. AI governance covers outcomes: are the system's actions fair and safe once it is running, who is accountable when it acts on its own. Organizations that treat AI governance as a subset of data governance end up protecting the vault's contents while leaving the door open.

Three layers that hold the boundary

  • Zero trust access: every request is checked against identity, current risk level, and behavior, not granted once and trusted forever. An agent asking for one professor's file gets it. An agent asking for the whole department's salary table gets quarantined.
  • Output-layer data loss prevention: records are pre-tagged by sensitivity, the agent is blocked from touching anything tagged Secret, and a real-time filter masks raw PII (a Social Security number becomes XXX-XX-1234) before any output reaches a person.
  • Observability: a registry of every sanctioned agent, a lineage tracker logging every read and write, and real-time alerts for anomalies. Shadow AI, agents deployed without IT's knowledge, is not really a technical problem. It is a visibility problem. You cannot govern what you cannot see.

Human in the loop, by zone

Low-consequence, reversible actions run on full autonomy. Higher-consequence actions, like drafting a promotion memo from real records, require a human to check the output before it goes anywhere, since the memo can be structurally fine and still wrong about a specific fact. Irreversible actions, like an actual salary change in Workday, freeze the agent and route a signed approval request to the HR Director. The system enforces that boundary structurally. It does not depend on the agent's judgment.

Regulation sets the outer boundary here, and the frameworks worth knowing map directly onto this problem: the EU AI Act's risk-tier classification and impact assessments for high-risk employment decisions, the NIST AI RMF's Govern-Map-Measure-Manage cycle, and ISO/IEC 42001's audit trail requirements. None of them yet specify the operational detail, like the zone framework above or agent registries, that practitioners need to actually build this. That translation work, from regulatory intent to a control an engineer can implement, is where the field is thinnest right now.

More posts