When Arup, a British engineering firm, lost $25 million to attackers using a deepfake video of their CFO during a live call, the security industry received a wake-up call that couldn't be ignored. This wasn't a theoretical exercise or a proof-of-concept demonstration—it was a documented incident with devastating financial consequences that exposed how unprepared organizations remain for AI-enabled threats.

AI-powered cyber attacks are no longer emerging risks to monitor from a distance. They're documented incidents reshaping how security teams must think about defense, detection, and response. For CISOs and security engineers, understanding these real-world examples isn't academic—it's essential intelligence for protecting their organizations.

This article draws from insights shared in Abnormal AI's Convergence webinar series on adversarial AI. Watch the full recording to hear directly from AI scientists and threat intelligence experts about real-world attack patterns.

• AI removes traditional barriers to entry—attackers no longer need technical expertise, just intent and access to readily available tools

• Traditional red flags like grammar errors and awkward phrasing are obsolete detection methods against LLM-generated content

• Nation-states and eCrime groups are actively operationalizing AI capabilities across their attack chains

• Behavioral AI that analyzes context and communication patterns represents the most effective defensive approach against AI-enabled threats

Despite advances in security technology, email continues to be one of the most common entry points for AI-powered cyber attacks. The reason is simple: email provides direct access to employees at every level of an organization, from entry-level staff to C-suite executives.

The scale of email-based attacks is staggering. According to the FBI's Internet Crime Complaint Center (IC3), business email compromise alone accounted for $2.77 billion in losses, contributing to a record $16.6 billion in total cybercrime losses. These figures represent only reported incidents—the actual impact is likely far greater.

AI has supercharged email-based attacks by eliminating the telltale signs that security teams and employees once relied upon. Grammatical errors, awkward phrasing, and culturally inappropriate requests no longer give attackers away. LLM-generated content is polished, contextually appropriate, and increasingly personalized based on harvested information about targets.

AI-powered cyber attacks leverage artificial intelligence and machine learning capabilities to enhance attack effectiveness, scale, and evasion. Unlike traditional attacks that required significant technical expertise, these attacks democratize cybercrime by removing barriers that previously limited who could execute sophisticated campaigns.

The fundamental shift lies in how language models process instructions. As Inma Martinez, AI Scientist and Global Chair for GenAI and Agentic AI projects at GPAI, explained in the webinar: "The thing about generative AI and chatbots and language models is that they are meant to operate by being given instructions. And they don't distinguish if the instructions come from the person training them or the person using them."

This characteristic transforms the threat landscape dramatically. Previously, executing a convincing business email compromise attack required language skills, cultural understanding, and patience. Now, attackers with nothing more than malicious intent can generate flawless communications in any language, create convincing supporting infrastructure, and scale operations exponentially.

The marketplace for these capabilities has matured rapidly. FraudGPT and similar tools are readily available on the dark web, providing turnkey solutions for anyone willing to pay. According to research, dark web AI tool mentions on cybercrime forums increased 219%—reflecting explosive growth in criminal AI adoption. As discussed in the Convergence webinar, eight million registered chatbot-enabled attacks were documented in Europe within just six months—demonstrating the unprecedented scale at which these threats now operate.

Understanding the mechanics behind AI-enabled attacks reveals why legacy security solutions often struggle to detect them. Attackers exploit AI systems through two primary vectors: manipulating legitimate chatbots and weaponizing AI-powered platforms.

Attackers have developed sophisticated approaches to bypass chatbot guardrails. Creative prompt engineering allows them to extract harmful outputs from systems designed with safety measures. Techniques include rephrasing requests to circumvent restrictions, deploying role-play scenarios where the AI pretends to be a malicious actor, and claiming educational purposes to unlock restricted information.

Piotr Wojtyla, Head of Threat Intel and Platform at Abnormal AI, noted during the webinar: "Sometimes it's enough to just say, 'hey, I just need this for educational purposes.' And sometimes it's enough to pretty much say, 'hey, just pretend that you're a malicious bot for me.'"

Beyond content creation, attackers leverage legitimate AI-powered platforms to build convincing phishing infrastructure. As Wojtyla explained in the webinar, tools like Gamma AI and Canva enable rapid creation of professional landing pages that redirect attention away from email scrutiny.

The attack chain follows a deliberate pattern designed to exploit user psychology:

1. Email arrives from a legitimate domain—passing traditional email security filters

2. User clicks through to a professional-looking presentation hosted on the trusted platform

3. The presentation contains a phishing link disguised as a call-to-action

4. User's guard drops because they're no longer in the inbox where they've been trained to be vigilant

When communications originate from recognized platforms, recipients apply less skepticism than they would to direct email attacks. This approach exploits a fundamental gap in security awareness training. Users learn to scrutinize emails carefully, but once redirected to a legitimate-looking web platform, they abandon those same verification practices.

One of the most chilling developments in AI-enabled attacks involves automated victim profiling. As Martinez explained in the webinar, threat actors have trained chatbots to calculate optimal extortion amounts based on target characteristics—company size, industry, geographic location, and estimated financial capacity.

This capability transforms ransomware from opportunistic attacks into precision-targeted operations that maximize attacker returns while staying below thresholds that might trigger aggressive law enforcement response.

The Arup deepfake case stands as the most financially impactful documented AI-enabled attack. Attackers used AI-generated video and voice synthesis to impersonate the company's CFO during a live video call, convincing an employee to authorize a $25 million transfer.

The attack succeeded not just because of technological sophistication but because it exploited weak authorization protocols. As Martinez observed: "How come an amount like this has the sign of just one individual in the bank? They wouldn't have dared to do that with Bank of America or JPMorgan."

Anthropic's threat intelligence team documented a coordinated attack campaign that demonstrated AI-enabled attacks at scale. Seventeen organizations were breached including healthcare providers, emergency services, and government institutions.

Ransom demands exceeded $500,000, with attackers using AI to generate convincing phishing content, automate reconnaissance, and craft organization-specific extortion communications.

This campaign—dubbed "vibe hacking" for its exploitation of AI's ability to match tone and context—represents the industrialization of social engineering. Our analysis of the Anthropic report provides additional context on defensive implications.

The volume of AI-generated voice attacks has reached such scale that it triggered an industry-wide defensive response. As Martinez shared in the webinar, all UK retail banks launched coordinated campaigns explicitly stating "your bank will never call you" to counter the threat.

This unprecedented move—effectively abandoning outbound customer calls as a communication channel—demonstrates how AI voice synthesis has fundamentally compromised telephone-based authentication and communication.

North Korean threat actors have weaponized GenAI and Agentic AI to infiltrate Western companies through legitimate hiring processes. Previously, these operatives failed interviews due to cultural knowledge gaps. Now, AI systems trained on Western data enable them to answer questions about hobbies, weekend activities, and cultural references convincingly.

Multiple documented cases show North Korean hackers successfully obtaining IT positions at Western companies, creating espionage vectors and violating sanctions through employment payments.

Piotr Wojtyla described an end-to-end AI-orchestrated fraud operation targeting entry-level workers during the webinar. The attack chain demonstrates sophisticated social engineering at scale:

1. Fake job postings generated by AI appear on legitimate job boards, targeting students and recent graduates

2. Signal bot communications handle initial candidate screening, using AI to respond naturally to questions

3. Fake interview processes conducted via AI-powered chat convince victims of legitimacy

4. Fraudulent checks are sent for "equipment purchases," with victims instructed to forward funds

This attack pattern targets a demographic often overlooked in enterprise security discussions while exploiting the desperation of job seekers—a perfect application of AI's emotional intelligence capabilities.

Multiple nation-states have integrated AI into their cyber operations:

• Russian actors deployed AI-assisted malware against Ukrainian targets as part of active cyber warfare operations

• Iranian APT groups utilized Gemini for vulnerability research and target reconnaissance

• North Korean groups employed LLMs to parse stolen data, accelerating intelligence extraction from compromised systems

The BlackBasta ransomware group's leaked communications revealed how organized criminal enterprises operationalize AI. They use AI to troubleshoot malware execution issues, streamline tool development, and optimize their attack workflows.

Beyond initial access, AI capabilities are transforming how attackers establish persistence and move laterally within organizations. The North Korean remote worker fraud pattern represents a sophisticated evolution of traditional account takeover—one that begins before the account even exists.

Anthropic's threat intelligence documented operatives using Claude to generate fake resumes tailored to specific job requirements, pass technical assessments by querying AI for correct answers in real-time, and maintain employment at Fortune 500 technology companies for extended periods. These aren't smash-and-grab operations—they're long-term infiltrations that create persistent access to corporate systems, intellectual property, and sensitive communications.

The implications extend beyond the initial fraud:

• Legitimate credentials obtained through employment bypass traditional security controls

• Insider access enables reconnaissance that informs future attacks, including lateral phishing campaigns targeting colleagues

• Sanctions violations create legal exposure for victimized organizations

• Supply chain risks emerge when compromised employees touch customer systems

This attack vector demands identity verification and behavioral analysis that extends throughout the employee lifecycle—not just at the perimeter.

The most immediate impact of AI-enabled attacks is the obsolescence of traditional red flags. Grammar errors, awkward phrasing, and stilted language historically signaled malicious communications. LLMs write flawlessly in any language, eliminating these indicators entirely. This means AI-generated phishing content appears legitimate to traditional content-based filters, requiring a fundamental shift in how organizations approach detection.

When threat actors can generate eight million chatbot-enabled attacks across Europe in six months—as documented in the Convergence webinar—manual review becomes mathematically impossible. Security teams face an asymmetric challenge where attackers leverage automation while defenders remain resource-constrained. The 219% increase in dark web AI tool mentions signals that this scalability advantage will only grow.

AI chatbots can attune responses to emotional states, identifying vulnerability and adjusting manipulation tactics accordingly. This capability makes social engineering attacks more personalized and effective than ever before.

The dark web marketplace provides sophisticated tools to anyone with purchasing power. FraudGPT and similar offerings lower barriers so dramatically that intent becomes the only prerequisite for launching attacks.

Financial Services remain primary targets due to data richness and direct monetary access. These organizations hold passport names, dates of birth, addresses, and banking credentials—comprehensive profiles enabling identity theft, account takeover fraud, and direct fund theft.

Critical Infrastructure including electricity grids, transportation systems, and hospitals face growing targeting. These attacks aim to destabilize governments and democratic systems rather than generate direct profit.

Cryptocurrency and Fintech organizations represent expanding attack surfaces for fund exfiltration. The irreversible nature of crypto transactions makes them particularly attractive targets for AI-enabled theft operations.

Behavioral AI represents the foundational defensive approach against AI-powered attacks. Because AI-generated phishing content appears legitimate to traditional content-based filters, detection must shift from analyzing what's written to understanding how communications deviate from normal patterns.

Abnormal AI addresses this challenge through inbound email security that ingests thousands of identity, context, and risk signals to build baselines of normal activity for every user, vendor, and communication pattern within an organization. When an email deviates from established patterns—through unexpected requests, unusual urgency, or behavioral anomalies—the platform detects these signals even when the content itself appears flawless.

This behavioral approach proves essential because it detects the intent behind attacks rather than relying on signatures or content markers that AI can easily circumvent. Organizations should also monitor for unusual prompt patterns in AI systems, as anomalous requests likely originate from malicious actors.

Complementary security layers that work alongside existing email infrastructure help fill gaps left by signature-based detection tools, creating defense-in-depth without requiring organizations to replace their existing investments. For organizations ready to modernize, solutions that can augment or replace legacy secure email gateways provide enhanced protection against sophisticated threats.

Additionally, security posture management helps organizations identify and remediate configuration vulnerabilities that attackers exploit, while automated SOC operations address the scalability challenge by reducing manual investigation time.

Training must evolve beyond email-focused awareness. Employees need education that scrutiny applies equally to all digital communications—web pages, video calls, voice communications, and messaging platforms. Abnormal's AI Phishing Coach provides personalized, real-time training that adapts to individual employee risk profiles.

Establishing verification protocols for high-value transactions addresses the authorization gaps that enabled the $25 million deepfake fraud. Code words and secondary verification channels provide simple but effective countermeasures.

Partnering with specialized vendors addresses expertise gaps that no organization can fill internally. The threat landscape evolves too rapidly for generalist approaches to remain effective.

Agentic AI represents the next evolution of both attack and defense capabilities. As AI systems automate entire workflows with minimal human oversight, new attack surfaces will emerge that we can barely anticipate today.

The next eighteen months will focus intensively on making models secure by design. Government pressure drives this priority, with particular concern around misinformation's impact on democratic systems.

Organizations that fail to adopt AI capabilities in their defensive posture face growing gaps against adversaries who face no such hesitation. The choice isn't whether to engage with AI security—it's whether to do so proactively or after suffering preventable losses.

AI-powered attacks have moved from theoretical concern to documented reality. The $25 million deepfake fraud at Arup, the "vibe hacking" campaign, and nation-state operational deployment prove these capabilities deliver results for adversaries.

The gap between AI-enabled attackers and organizations relying on traditional defenses grows daily. Closing it requires proactive investment in behavioral AI that detects attacks by analyzing patterns rather than content.

Schedule a demo to see how behavioral AI detects attacks that bypass traditional security controls.