Skip to main content

Oct 25, 2026

How to Implement AI Governance: A Practical Guide

An AI governance strategy only works once it runs day-to-day. Learn how to roll it out in phases, from discovery and policy to enforcement.

Key Insights

NIST's AI RMF Playbook warns that partial inventories may not provide the value of a full inventory.

NIST AI RMF 1.0 does not prescribe risk tolerance, leaving tier definitions to each organization.

Joint CISA and NSA agentic guidance recommends logging identity and privilege changes and auditing for drift.

ISO/IEC 42001:2023 remains in its first edition for organizations pursuing certification.

On 1 May 2026, CISA, NSA and partner agencies published the joint guide Careful Adoption of Agentic AI Services, which recommends logging identity and privilege changes and auditing regularly for drift, impersonation or misconfiguration. Acting on that advice depends on something earlier, which is knowing what AI is running in the first place.

An AI governance strategy defines how an organization intends to use AI safely. Implementing AI governance means running that intent in phases, each with one owner and one output the next phase depends on.

This guide covers that sequence, typical 90-day timing, what to do when a team bypasses an approval gate, and how to keep sanctioned AI use moving while controls take hold.

What Does It Take to Turn an AI Governance Strategy Into Practice?

Implementing AI governance takes visibility, policy and enforcement running together. Visibility shows what AI is in use, policy defines what is allowed, and enforcement makes the policy hold at the point where people and agents act.

Without an inventory, policy writers set rules without knowing which tools employees use. Without policy, security teams block work nobody has reviewed, and the exception requests that follow expose both problems at once.

Strategy and implementation answer different questions. An overview of AI governance covers what the discipline includes, while implementation is the operating layer underneath it, meaning who runs discovery each week, who approves a new tool, and who answers the alert when an agent's permissions change.

Because AI now arrives as new features inside existing SaaS suites, often without a separate procurement event, security operations has to track those changes continuously.

What Needs to Be in Place Before You Start?

Before Phase 1, a rollout needs an executive sponsor, an agreed scope, a baseline count of current AI use, and the control frameworks already in place. Each one settles a question that would otherwise resurface as a blocker during policy or enforcement:

  • Executive Sponsor: A CISO or equivalent executive owns the program, settles disputes between security and business units, and accepts residual risk.

  • Agreed Scope: The program states what it covers, such as public AI tools, AI features in sanctioned SaaS, internally built models and autonomous agents.

  • Baseline of Current AI Use: A rough first count from SSO logs and expense data gives the program a number to measure progress against.

  • Existing Control Frameworks: Extend identity, DLP, vendor risk and incident response processes to AI rather than rebuilding them in parallel.

Naming the sponsor is the one that cannot wait, because the other three need someone with authority to settle them, and the split of AI governance roles and responsibilities follows from that appointment.

The Phases of an AI Governance Rollout

Implementation runs in order, and each phase has one accountable owner. Security operations runs discovery, security and risk assign tiers, the CISO publishes policy with legal, security engineering builds enforcement, the SOC connects governance to detection, and the CISO reports to the board. Every phase hands the next one an output it cannot start without, which is why discovery comes first.

AI governance rollout in six phases: discover, classify, publish policy, enforce, connect to security operations, and report and review, each with its owning team.

Phase 1: Discover AI Use Across Tools, Agents and Data Flows

Owner: security operations.

Output: an AI inventory of sanctioned tools, unsanctioned shadow SaaS tools, AI features inside approved apps, agents, and the data each one touches.

Security operations can start from telemetry it already holds. SSO and OAuth consent logs show which AI apps are connected to corporate identities and what scopes employees granted, while proxy, DNS and browser telemetry show traffic and uploads to AI services. Agents need extra fields for a human owner, the credentials they hold and the systems they can reach.

Coverage is what separates a useful inventory from a misleading one. NIST's AI RMF Playbook warns that "partial inventories may not provide the value of a full inventory," and counting only sanctioned accounts is the usual way a partial inventory happens. Abnormal's AI Governance is designed to discover approved and shadow AI tools, attribute ownership and track compliance gaps.

Phase 2: Classify and Prioritize AI Use by Risk

Owner: security and risk management.

Output: a risk tier for every inventoried tool and use case, each tied to a defined depth of review.

Most of the early tiering work rests on three readings taken for each tool:

  • Data Sensitivity: What the tool receives, from public marketing copy through to regulated customer records.

  • Permissions: What it holds, from chat access alone through to OAuth scopes over mail, files and calendars.

  • Vendor Trust: What the contract says about retention, training on inputs and who else processes the data.

The NIST AI Risk Management Framework (AI RMF 1.0) does not prescribe risk tolerance. Its Playbook suggests policies "define how to assign AI systems to established risk tolerance levels by combining system impact assessments with the likelihood that an impact occurs." NIST states that AI RMF 1.0 is being revised, so recheck tier definitions when the update is published.

Reviewing every tool at the same depth is a common failure, because it can turn a low-risk writing assistant into a weeks-long ticket.

Phase 3: Publish Policy and Approval Paths

Owner: the CISO, with legal and privacy.

Output: a published AI policy, an approval path for new tools and release gates for internally built AI.

The policy states which tiers employees can use freely, which need approval and which are off-limits. The approval path names where requests go and how fast decisions should arrive. Release gates apply to what the organization builds, with one before production, one before an agent receives write permissions and one before customer-facing use. The policy wording follows the guide to writing an AI security policy.

An approval path without a service-level target invites teams to route around governance, and security engineering inherits that backlog in the next phase.

Phase 4: Enforce Policy at the Point of Use

Owner: security engineering.

Output: controls that apply policy where employees and agents act, plus a defined response to bypass.

Controls work in layers, and the first layer is identity. Security engineering should check whether identity settings can restrict OAuth consent to unsanctioned AI apps, whether DLP rules can inspect prompts and uploads bound for approved tools, and whether proxies or AI gateways support tier rules. Signup controls can then block, warn or allow, naming the approved alternative in every warning.

When a team bypasses an approval gate, the response should be procedural. Discovery telemetry flags the tool, security engineering revokes the OAuth grant or rotates any agent credential, and the tool enters expedited review. Any interim exception gets an owner and an expiry date, and repeat bypasses escalate to the executive sponsor.

Hard blocking before alternatives exist backfires, because employees move to personal accounts and the use becomes harder to monitor.

Phase 5: Connect Governance to Security Operations

Owner: the SOC.

Output: AI threat models, governance events in the SIEM and SOAR, and an AI incident response playbook.

Threat modeling gives detections a target, and the OWASP Top 10 for Agentic Applications, in its 2026 edition, names risks such as agent goal hijack and identity and privilege abuse.

New AI tools, governance status changes and new OAuth grants belong in the SIEM beside identity events. The joint guidance cited earlier recommends logging identity and privilege changes and regularly auditing for drift, impersonation, or misconfiguration, and incident plans need scenarios that reflect how AI systems fail.

Logging agents as generic service accounts defeats all of it, because nothing then shows which actions an AI took.

Phase 6: Report to Leadership and Review the Program

Owner: the CISO.

Output: a recurring leadership and board report, plus a review cycle for updating tiers, policy and controls.

Board metrics should show coverage and outcomes, among them the share of discovered tools inventoried and tiered, sanctioned versus unsanctioned use, policy acknowledgment, median time to approve a request, AI incidents escalated and remediated, and spend by tool and team. Organizations pursuing certification can align with ISO/IEC 42001:2023, still in its first edition.

The CISO uses each review to guide Phase 1, since teams adopt new tools and agents between cycles.

How Long Does AI Governance Implementation Take?

A first working version typically takes about 90 days, though scope, tenant count and regulated data can stretch it. That timing is a common pattern rather than a guarantee, and the work usually divides by month:

 

  • Days 1 to 30: The sponsor confirms scope while security operations runs discovery and builds the first inventory, including agents and OAuth grants.

  • Days 31 to 60: Security and risk assign tiers, and the CISO publishes policy and the approval path with legal sign-off.

  • Days 61 to 90: Security engineering turns on warn-first enforcement, and the SOC routes governance events to the SIEM and drafts the AI incident playbook.

Board reporting usually begins at the first quarterly cycle after day 90, when the rollout becomes a continuing review loop.

How Do You Bring Employees Along Without Blocking AI Use?

Employees follow governance when the approved path is faster than the workaround. A ban does not remove the need that drove someone to the tool, so it tends to move the use somewhere nobody can see it. Each of these practices shortens the distance between a need and a sanctioned option:

  • Approved Alternatives: Every unsanctioned tool maps to a sanctioned option employees can use the same day.

  • Fast Approval: Low-tier requests get decisions within a published target.

  • Coaching at the Moment of Use: A warning at signup or upload names the rule and the approved alternative.

  • Clear Communication: Employees learn what is allowed and how to ask for more, and acceptable use policy guidance helps set that language.

What Mistakes Slow AI Governance Down?

Most delays come from sequencing errors rather than missing tools, and the same ones recur:

  • Writing Policy Before Discovery: Policy writers who skip the inventory miss embedded AI features and agents, then need to rewrite the rules.

  • Banning Without Alternatives: Employees may move work to personal accounts when security blocks tools without a sanctioned option, making use harder to monitor.

  • Building Gates With No Enforcement: Teams can skip approval when nobody checks whether they followed the path.

  • Keeping Governance Apart From the SOC: Without AI events in the SIEM, an agent with drifting permissions looks like routine service-account activity.

Writing policy before discovery is the hardest to reverse, because a published policy written against the wrong estate must be withdrawn before it can be replaced.

Making AI Governance an Operating Practice

What separates a working program from a published policy is its cadence. Security and risk turn discovery findings into tiers and approval depth, security engineering applies those decisions at the point of use, and the SOC and CISO use monitoring and board reporting to aim the next round of discovery. That review cycle is what keeps tiers, policy and controls current as the estate changes.

Discovery is where rollouts tend to stall, and every later phase inherits whatever Phase 1 missed. Abnormal's AI Governance, part of the AI Security suite, surfaces approved and shadow AI tools and assigns each to a named owner. Explore AI Governance to see what discovery surfaces before policy is written.

Frequently Asked Questions

Where should AI governance implementation start?

Start with discovery, because an inventory of sanctioned tools, shadow AI, embedded AI features, and agents shows what the program needs to govern. Choosing a framework or drafting policy first tends to produce rules that miss real usage, and most teams run discovery in the first 30 days.

What stretches a rollout past 90 days?

Multiple tenants, regulated data and internally built models are the usual causes, because each adds a review that cannot run in parallel with the others. A large estate of agents does the same, since every agent needs an owner and a credential review before enforcement can be switched on safely.

Who should lead AI governance implementation?

A CISO or equivalent executive usually sponsors the program and accepts residual risk, while daily ownership is cross-functional. Security operations runs discovery, security and risk assign tiers, legal and privacy co-own policy, security engineering builds enforcement, and the SOC handles AI detections and incident response.

Do you need a dedicated AI governance tool?

Many teams start with SSO logs, OAuth grants, proxy data and spreadsheets. A dedicated tool becomes worth evaluating when manual inventory stops scaling, usually when new AI tools appear faster than anyone can record them and agents multiply between review cycles.

How do you enforce AI policy without blocking employees?

Lead with warnings instead of blocks for most tiers, and pair every restriction with an approved alternative. Publish a fast approval path with a decision target for low-risk requests. Reserve hard blocks for the highest-risk tools and data, and coach users at signup or upload.

Protect Against Evolving Email Threats

See how behavioral AI detects attacks that legacy defenses miss.