Key Insights
A 2025 Gartner survey of cybersecurity leaders found that 69% of organizations suspect or have evidence that employees use prohibited public generative AI (GenAI) tools. Those tools typically bypass AI risk assessments, the structured process that identifies potential failures or misuse, judges each scenario's likelihood and harm, and decides whether a system should run.
This guide structures an AI risk assessment around evidence to decide whether to approve, require treatment, accept residual risk, or halt use. It also follows an illustrative customer support chatbot from scoping through reassessment. Each step ties back to the National Institute of Standards and Technology (NIST) AI Risk Management Framework (AI RMF), ISO standards, and the EU AI Act, and separates what those frameworks expect from what each organization decides.
What Is an AI Risk Assessment?
An AI risk assessment is a documented judgment about whether a specific AI system, in a specific deployment, stays within the organization's risk tolerance. AI complicates that judgment because the same input can produce different outputs, and a deployed system may behave in ways testing never showed. Privacy, fairness, reliability, and societal effects belong in the assessment alongside security. Legal, privacy, and business owners hold the judgments security teams cannot make alone.
Why AI Risk Assessment Needs Structure and Ownership
AI risk assessment needs structure and ownership because ad hoc reviews miss AI systems that change after approval, appear without it, or arrive inside vendor products. A chatbot's launch review says nothing about a retrieval source connected three months later, and third-party models create a second blind spot, since a provider can change versions underneath a deployment.
Shadow AI is the third blind spot, because employees adopt public tools that never reach a review. Abnormal's AI Governance can help bring those tools into review by discovering approved and shadow AI tools and attributing an owner to each.
A structured AI governance program closes all three blind spots by tying each system to named owners. A system owner maintains the inventory entry and its evidence, a risk owner accepts or refuses residual risk against the documented tolerance, and a named person holds halt authority.
AI Risk Assessment Frameworks and Legal Requirements
Frameworks set what an AI risk assessment should document, and laws decide which duties apply. The AI RMF describes good documentation, ISO/IEC 42001's management system standard adds a risk process from assessment to treatment, and the EU AI Act ties legal duties to how an organization uses a system. The NIST AI RMF is voluntary, and the AI RMF's four core functions each call for records an assessor can produce on request:
Govern: Organizations maintain inventory mechanisms, scale effort to their risk tolerance, and review periodically with defined roles.
Map: Teams document purpose, setting, users, and limitations, then judge likelihood and magnitude.
Measure: Teams document security and resilience evaluations and periodically check control effectiveness.
Manage: Teams decide whether to proceed with deployment and document a response to each risk.
The functions are not an ordered checklist, and the framework expects most users to start with Map.
The EU AI Act is law and classifies systems by intended purpose, so uses such as determining eligibility for loans or employment are high-risk. That legal tier differs from an enterprise risk score, and its obligations depend on the organization's role. Providers of high-risk systems must comply with current EU AI Act requirements, including risk mitigation, logging, human oversight, and cybersecurity.
Certain deployers, including public bodies and operators of high-risk credit or insurance systems, must complete a fundamental rights impact assessment before deployment. For a high-risk system in the EU, either role turns parts of the assessment into a legal duty.
The six assessment steps that follow, an editorial synthesis rather than a prescribed method, turn these frameworks into one working process. Each uses an illustrative customer support chatbot that answers billing and product questions using retrieval-augmented generation over help-center articles and past tickets.
How to Scope an AI Risk Assessment and Inventory the System
Scoping an AI risk assessment means defining what the system does, what it can reach, and who decides, then inventorying the models, data, and provider evidence behind it. Every later likelihood and impact rating depends on the scope record.
Step 1: Define the Use Case, Boundaries, and Decision Owners
The scope record answers who uses the system, for what, with which data, and under whose authority. For the chatbot, that means the intended use (billing and product questions, not account changes), affected customers and escalation agents, reachable data, and a live-agent handoff when it cannot answer. The record also notes whether the system supports or replaces a human decision. This chatbot replaces one because customers may act on a refund policy it states.
The scope record also holds risk tolerance and approval authority, such as a statement that bars any public-facing system from exposing one customer's personal data to another. For the chatbot, the support operations director owns the system, and the head of customer operations owns the risk and approves deployment. Naming both roles before testing starts keeps the team that built the system from approving its own residual risk.
What the chatbot can read and do sets the boundary, and indexing past tickets brings personal data into retrieval, so data disclosure enters scope. If the product team later adds a refund tool, the boundary expands to actions on customer accounts and agent misuse a read-only chatbot could never cause.
Step 2: Record Systems, Dependencies, and Provider Evidence
Each entry in the organization's AI inventory gives a later reviewer what they need to retrace the assessment:
Purpose and Ownership: The entry records the system's purpose and a named owner.
Model and Lifecycle: The entry records the model version and lifecycle stage.
Data and Dependencies: The entry records data provenance and dependencies.
Monitoring and Review: The entry records the monitoring plan and the date of the last assessment.
The chatbot depends on the hosted model API, a vector database of indexed content, and the pipeline that pulls tickets into it. Each dependency can change without notice. The inventory must also detect unregistered AI through continuous usage monitoring, including AI features vendors enable within approved software.
Third-party AI components fall under the same third-party risk management discipline as any other supplier, and they need provider evidence, such as model or system cards, training data provenance, testing documentation, and, where available, an AI bill of materials. For the chatbot, the provider controls model behavior, and the organization controls retrieval content, prompts, and permissions. The entry should also name a fallback, such as routing every conversation to live agents if the provider's API fails or it retires a model version.
How to Identify AI Threats and Rank Risks with Evidence
Identifying AI threats turns each inventory entry into specific risk scenarios that testing can confirm or rule out. Ranking them against preset bands ties each rating to evidence, not intuition.
Step 3: Build Risk Scenarios and Test Their Assumptions
Risk scenarios describe how a specific threat could reach the system. Published attack taxonomies give the team a starting list. The NIST adversarial attack taxonomy sorts generative AI attacks by attacker goals, including availability and integrity violations, privacy compromises, misuse, and supply chain attacks.
The chatbot's illustrative scenario is indirect prompt injection through retrieved documents. A customer submits a ticket containing hidden instructions. Once indexed, the ticket surfaces in a later conversation, and the chatbot follows the planted text by revealing another customer's details or stating a false refund policy. A second scenario needs no attacker, because the chatbot can invent a refund rule that no help-center article contains, and sampled conversations show how often that happens.
Every scenario rests on assumptions that need evidence. Testers can plant injected instructions in a staging copy of the index and check whether outputs change, and the team can confirm who can write content that reaches retrieval.
If the chatbot later gains the refund tool from Step 1, testers also check whether injected text can trigger a refund or poison memory across sessions. Public or internal incident reports involving similar retrieval assistants count as evidence for the likelihood rating.
Step 4: Assess Likelihood, Impact, and Tolerance
Likelihood and impact ratings place each scenario against the organization's tolerance bands. The AI RMF calls for likelihood and magnitude judgments but gives no numeric scale. When evidence is thin, the team records low confidence and either assigns the rating to the higher band or tests further. An illustrative rubric ties each band to the evidence that puts a scenario there:
Band | Likelihood Evidence | Impact Evidence |
|---|---|---|
Low | Path requires privileged access, and testing failed to exploit it | No personal or confidential data exposed, and errors are easy to correct |
Medium | Path reachable by authenticated users, or exploited only under contrived conditions | Limited internal data exposed, or a few users misled |
High | Path reachable by anyone, and exploited under realistic conditions | Personal data exposed across customers, or financial or regulatory harm |
For the chatbot, any high-impact scenario needs treatment before launch, and the injection scenario rates high on both. Its register entry records the basis for that rating:
Evidence: Staging injection tests succeeded, any customer can create content that reaches retrieval, and indexed tickets hold personal data.
Owners: The support operations director owns the entry, and the head of customer operations owns the risk decision.
Treatment and Review: The team will restrict retrieval to reviewed help-center articles, keep the rating high until retesting, and review the entry when sources or tools change.
With those fields, a later reviewer can see why the rating stands, who owns it, and when it returns for review.
How to Treat AI Risks and Keep Deployment Decisions Current
Treating AI risks means choosing a response for each ranked scenario and naming who accepts what remains. Monitoring keeps that approval current, and any system change reopens the assessment.
Step 5: Select Controls and Review Residual Risk
Every risk scenario ranked in Step 4 needs a recorded response, chosen from four options:
Accept: The team keeps the risk as it stands, with reporting and monitoring.
Avoid: The team removes the feature or data that creates the risk.
Transfer: The team shifts part of the risk through contracts.
Mitigate: The team applies controls that reduce likelihood or impact.
Before adding controls, the team checks whether existing safeguards keep exposure within tolerance. For the chatbot, removing raw tickets from the index avoids the cross-customer disclosure path. Limiting edit rights on help-center articles, treating retrieved text as untrusted input, and routing account-specific questions to a live agent mitigate what remains. Red teaming here means rerunning the staging injection tests against the treated configuration.
The rerun results become the control-effectiveness record, so if injection attempts no longer change outputs and no personal data remains reachable, the team has evidence to re-rate the scenario as low likelihood and medium impact. It then documents the remaining exposure, such as misleading answers from a tampered help-center page. The head of customer operations, not the team that built the chatbot, accepts that residual risk or withholds approval.
Step 6: Monitor Changes and Reassess Approval
Monitoring keeps an approval valid by checking that the deployed system still matches what the team assessed. For the chatbot, the monitoring plan names the support operations director as its owner, the evidence to collect (logs of retrieved sources, sampled conversations, handoff rates, and repeated injection test results), and the organization's thresholds. An illustrative threshold would send any confirmed injected instruction in indexed content straight to the risk owner. The on-call support manager holds halt authority, the documented right to switch off the chatbot or revoke a tool without waiting for a committee.
Model or prompt changes, new data sources or tools, system incidents or similar events, and control failures should reopen the assessment immediately. No widely adopted framework prescribes a universal review cadence, so scheduled reviews are only a backstop.
Suppose the product team proposes indexing a community forum or giving the chatbot a refund tool. Both changes alter the Step 1 boundary, so the assessment restarts there with new scope, inventory fields, scenarios such as tool misuse, and a fresh approval decision. Treating those proposals as new assessments keeps the original approval from covering untested risks.
Make Every Deployment Decision Traceable
A sound AI risk assessment traces each deployment decision back through scope, inventory, tested scenarios, treatment records, and monitoring evidence. An approval expires the moment its evidence does, which is why the monitoring plan, rather than the sign-off date, sets how long it stands.
AI Governance, generally available in Abnormal's AI Security suite, is designed to discover approved and shadow AI tools and track compliance gaps. Explore the AI Security platform to see how Abnormal helps surface unregistered AI before an assessment.
Frequently Asked Questions
Is an AI impact assessment the same as an AI risk assessment?
No, a risk assessment asks how uncertainty threatens the organization's objectives, and an impact assessment asks how the system affects people and society. Shared records, such as the system description, data provenance, and stakeholder list, can support both. A completed risk assessment does not satisfy an impact assessment requirement where a law or standard specifies what that assessment must contain.
Can a general-purpose AI tool complete an AI risk assessment?
No, a general-purpose AI tool can draft a starting point, such as a risk-register template or candidate scenarios, but it lacks verified evidence about the specific system and cannot test the deployment. A named owner owns likelihood and impact judgments, and the accountable risk owner approves deployment.
Can an AI risk assessment work without access to model internals?
Yes, provider evidence, including model cards, testing documentation, and stated limitations, describes how the provider built the model and where it falls short. Service-level testing queries the deployed system and shows how it behaves with the organization's own data and configuration. Documented limitations then enter the register as assumptions the team monitors.