The promise of artificial intelligence in cybersecurity comes with a sobering reality: when AI systems make mistakes, organizations often struggle to answer a fundamental question—who's responsible? A healthcare organization that deployed an AI system for treatment recommendations discovered the model exhibited bias affecting patient care—the technical failure was clear, but the accountability structure was anything but. This scenario plays out across industries as organizations race to adopt AI-powered security tools without establishing clear governance frameworks, creating a dangerous gap between technological capability and organizational readiness.

This article draws from insights shared in the Convergence Series panel discussion on AI and cybersecurity policy. Watch the full webinar to hear directly from former White House cybersecurity adviser Michael Daniel and other industry experts.

• AI governance requires a formal framework that assigns clear accountability across security, legal, data science, and executive leadership functions

• The tension between innovation speed and governance requirements demands deliberate balance—organizations cannot afford recklessness when sensitive data and critical systems are at stake

• Explainability and transparency are foundational to building trust in AI systems and meeting emerging regulatory expectations

• Regulatory landscapes vary significantly across jurisdictions, requiring organizations to build adaptable governance structures

AI governance encompasses the framework of policies, processes, and accountabilities that ensure AI systems are developed and deployed responsibly within an organization. For security leaders, this means establishing oversight mechanisms that address both the opportunities and risks AI introduces to cybersecurity operations.

The scope of AI governance extends beyond technical controls. It includes decision rights, risk management protocols, compliance requirements, and ethical considerations that guide how organizations build, procure, and operate AI systems. For cybersecurity professionals specifically, governance must address how AI tools handle sensitive data, make security decisions, and interact with existing security architectures.

Michael Daniel, President and CEO of the Cyber Threat Alliance and former White House cybersecurity adviser, framed the current landscape during the webinar: "Governments are really looking at this issue of how organizations are using AI. And that applies to its use in the cybersecurity area as well." He noted that "many governments, including the US government, are still trying to figure out exactly how they want to approach this topic."

This regulatory uncertainty creates both challenges and opportunities for security leaders. Organizations that establish robust internal governance now position themselves to adapt more readily as external requirements crystallize.

Governance requirements also differ based on AI type. Daniel drew an important distinction during the webinar between classification AI—which cybersecurity tools have leveraged for over a decade—and newer generative or agentic AI systems. Classification models that categorize threats require different governance controls than generative systems that create content or take autonomous actions. Effective frameworks must account for these differences rather than treating all AI governance as monolithic.

The healthcare AI bias incident mentioned earlier illustrates a critical reality: when AI governance responsibilities remain undefined, organizations face compounding risks—technical failures become compliance failures, which become reputational crises. The scale of this governance gap is striking: only 32% of organizations use AI extensively in their security programs, and nearly two-thirds lack an AI governance policy altogether, according to recent research.

James Yeager, who leads public sector operations at Abnormal AI, captured this tension in the webinar: "AI brings both a tremendous amount of promise to the table, but dragging along with it is a fair amount of peril."

The stakes are particularly high in security contexts. AI-powered tools increasingly make decisions about threat detection, access control, and incident response. Email security represents one of the highest-stakes governance domains—email was identified as the attack vector in 27% of breaches, making AI-powered email security tools precisely where explainability and bias testing matter most. When these decisions go wrong—whether due to model drift, adversarial manipulation, or training data bias—unclear accountability structures leave organizations scrambling.

Yeager emphasized the need for deliberate governance despite the pressure to innovate: "We do need to avoid certain impulses. We can't be reckless about it. We have lives at stake." This applies equally to healthcare organizations handling patient data and financial institutions protecting customer assets.

The compliance dimension adds urgency. Regulatory bodies worldwide are developing AI-specific requirements, and organizations without documented governance structures will struggle to demonstrate compliance when auditors come calling.

Establishing clear AI governance responsibilities requires mapping accountabilities across multiple organizational functions. The RACI matrix—defining who is Responsible, Accountable, Consulted, and Informed—provides a practical framework for this exercise.

A fundamental tension exists in many organizations: should the CISO own AI governance, or does this responsibility belong to a Chief AI Officer or similar role? The answer depends on context, but security leaders must recognize that AI governance touches their domain regardless of formal reporting structures.

Daniel offered perspective on navigating this tension: "At a certain point, somebody had to say, okay, yes, I understand that there is some risk. We're going to take it because the benefits could easily outweigh this." This decision authority must be clearly assigned within governance frameworks.

The CISO bears primary accountability for AI security risks, including threats introduced by AI tools and vulnerabilities in AI-powered defenses. This encompasses data protection, model security, and integration with existing security architectures.

Cross-functional coordination with legal, compliance, and business leaders ensures governance addresses regulatory requirements, contractual obligations, and strategic objectives. Executive sponsors must authorize risk acceptance decisions that technical teams cannot make independently.

Security engineers and SOC teams carry operational responsibility for AI tool oversight. This includes monitoring AI system performance, validating outputs, and escalating anomalies that may indicate model degradation or adversarial activity.

Yeager highlighted the opportunity this creates: "We can optimize SOC teams and infosec professionals, giving them more exciting things to work on." Effective governance enables this optimization by establishing clear boundaries and escalation paths.

Comprehensive AI governance addresses twelve critical functions that span the AI lifecycle:

Model validation ensures AI systems perform as intended before deployment and during operation. This includes testing against diverse scenarios and monitoring for performance degradation over time.

Bias testing identifies and mitigates discriminatory patterns in AI decision-making. For security tools, this means ensuring detection capabilities work equally well across different user populations and threat types.

Incident response protocols address AI-specific failure modes, including model compromise, data poisoning, and adversarial attacks targeting AI systems.

Data governance establishes controls over training data, operational data, and AI outputs. This includes data quality standards, retention policies, and access controls.

Explainability requirements determine how AI decisions must be documented and communicated. Yeager described Abnormal's approach: "When we render a verdict about specific threat related activity, we do our best to inform the customers about the signaling that's allowed us to arrive at that conclusion." He emphasized that Abnormal doesn't want customers to "just take our word for it" and aims to be "educators as well"—a philosophy that transforms explainability from mere compliance checkbox into genuine partnership with security teams.

Transparency builds stakeholder trust through clear communication about AI capabilities and limitations. As Yeager noted, "That transparency, that's how you build trust. That's how you build confidence."

These functions must map to emerging cybersecurity frameworks including the EU AI Act and NIST AI RMF, which provide structured approaches for categorizing AI systems by risk level and establishing appropriate controls.

The regulatory landscape for AI governance remains fragmented and evolving. Daniel described the complexity: "Differences between, say, the US and the EU, but also other jurisdictions—Japan, China, Korea, Australia."

Organizations operating internationally face the challenge of building governance structures that accommodate varying requirements. The EU AI Act establishes risk-based classifications with specific obligations for high-risk systems. NIST AI RMF provides a voluntary framework emphasizing risk management throughout the AI lifecycle.

Daniel characterized the current state: "I would consider us in very early days." However, he offered a clear warning: "Any technology that can impose harms on society will eventually have some form of government oversight and regulation."

This reality demands proactive governance. Organizations that wait for concrete regulations before establishing accountability structures will find themselves scrambling to retrofit controls onto deployed systems.

Daniel also highlighted the importance of pushing for international regulatory harmonization—working toward frameworks where jurisdictional differences represent "ten percent of the compliance burden as opposed to eighty percent." For organizations building governance structures today, this means designing adaptable frameworks with modular controls that can accommodate regional variations without requiring complete rebuilds as regulations converge.

Government expectations extend specifically to security tools. Regulators increasingly expect organizations to demonstrate oversight of AI-powered defenses, including documentation of decision-making processes and evidence of ongoing monitoring.

Many AI security tools operate as what Yeager called a "black box... we're told as security professionals, just trust us." This opacity undermines governance by making it impossible to validate AI behavior or assign meaningful accountability.

The challenge, as Yeager framed it: "It's really not so much what it does, but it's really like, how is it doing it?" Governance frameworks must include requirements for vendor transparency and internal documentation of AI decision processes.

AI capabilities evolve faster than governance structures can adapt. Daniel acknowledged this reality: "The technology can change, like, people can only adopt the technology so fast. Organizations can only adopt it so fast."

Effective governance requires flexibility—principles-based approaches that can accommodate new AI capabilities without requiring complete framework overhauls.

Some organizations assume AI moves too fast for meaningful governance. Daniel pushed back on this assumption: "It's a misconception in my view that the government is completely incapable of ever saying anything useful about AI." Organizations that delay governance based on this misconception will find themselves unprepared when regulations materialize.

Start by mapping current AI usage across security operations. Many organizations have more AI-powered tools than leadership realizes, from email security platforms to threat intelligence feeds.

Define clear use cases for each AI tool. Daniel emphasized this approach: "You should know what you're trying to achieve with that tool... that's how you make that balance." Use cases provide the foundation for appropriate governance controls.

Establish documentation requirements that enable accountability. This includes decision logs, performance metrics, and incident records that demonstrate governance in action.

Connect governance to outcomes. As Yeager noted: "AI is cool and all, but we're not adopting AI just because it's the latest RSA buzzword." Governance should enable rather than impede the security improvements AI promises.

Finally, build in oversight mechanisms. Yeager's summary captures the imperative: "Finding that balance there between what type of costs can we avoid, what type of operational efficiencies can we gain, that's fantastic, but we need to have some oversight."

Establishing clear AI governance responsibilities is no longer optional for security leaders. The convergence of regulatory pressure, sophisticated threats, and organizational AI adoption demands formal accountability structures.

Start with the fundamentals: document what AI you have, define who owns each governance function, and establish mechanisms for oversight and continuous improvement. Organizations that build these foundations now will navigate the evolving regulatory landscape with confidence.

For security teams looking to evaluate how their current email security tools handle transparency and explainability, a risk assessment can reveal gaps between existing defenses and emerging governance requirements.