| Data & privacy | IP infringement | Copyrighted code | INC-006[LLM-Driven Large Code Rewrites With Relicensing Are The Latest AI Concern](/evals-taxonomy/incident-database#inc-006) | PAP-001[Measuring Copyright Risks of Large Language Model via Partial Information Probing](https://arxiv.org/abs/2409.13831) |
| Copyrighted text | INC-041[The Prompt: Perplexity’s Plagiarism Problem](/evals-taxonomy/incident-database#inc-041) | PAP-001[Measuring Copyright Risks of Large Language Model via Partial Information Probing](https://arxiv.org/abs/2409.13831) |
| Trademark slogans | INC-007[ByteDance pledges to prevent unauthorised IP use on AI video tool after Disney threat](/evals-taxonomy/incident-database#inc-007) | PAP-001[Measuring Copyright Risks of Large Language Model via Partial Information Probing](https://arxiv.org/abs/2409.13831) |
| Trademarked brand names | INC-007[ByteDance pledges to prevent unauthorised IP use on AI video tool after Disney threat](/evals-taxonomy/incident-database#inc-007) | PAP-001[Measuring Copyright Risks of Large Language Model via Partial Information Probing](https://arxiv.org/abs/2409.13831) |
| Proprietary style mimicry | INC-011[King Gizzard and the Lizard Wizard Respond to Spotify Impersonator: “We Are Truly Doomed”](/evals-taxonomy/incident-database#inc-011) | PAP-001[Measuring Copyright Risks of Large Language Model via Partial Information Probing](https://arxiv.org/abs/2409.13831) |
| Personal data leakage | Personal identifiers | INC-024[Cybersecurity Alert – Salesloft Drift AI Supply Chain Attack](/evals-taxonomy/incident-database#inc-024) | PAP-002[Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use](https://arxiv.org/abs/2604.05432) |
| Sensitive personal records | INC-029[McDonald’s AI bot spills data on job applicants](/evals-taxonomy/incident-database#inc-029) | PAP-002[Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use](https://arxiv.org/abs/2604.05432) |
| Credentials | INC-025[s1ngularity Supply Chain Attack: What It Means for Cloud and AI Security](/evals-taxonomy/incident-database#inc-025) | PAP-002[Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use](https://arxiv.org/abs/2604.05432) |
| Business data leakage | HR data | INC-029[McDonald’s AI bot spills data on job applicants](/evals-taxonomy/incident-database#inc-029) | PAP-004[Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation](https://arxiv.org/abs/2512.16310) |
| Operational data | INC-038[Microsoft fixes ASCII smuggling flaw that enabled data theft from Microsoft 365 Copilot](/evals-taxonomy/incident-database#inc-038) | PAP-004[Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation](https://arxiv.org/abs/2512.16310) |
| Financial data | INC-019[Prompt injection – and a \$5 domain – trick Salesforce Agentforce into leaking sales](/evals-taxonomy/incident-database#inc-019) | PAP-004[Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation](https://arxiv.org/abs/2512.16310) |
| Proprietary data (inc. code) | INC-016[GitHub Copilot Chat Flaw Leaked Data From Private Repositories](/evals-taxonomy/incident-database#inc-016) | PAP-004[Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation](https://arxiv.org/abs/2512.16310) |
| System prompt | INC-031[A leak reveals the entire system prompt of Claude 3.7 Sonnet](/evals-taxonomy/incident-database#inc-031) | PAP-003[System Prompt Extraction Attacks and Defenses in Large Language Models](https://arxiv.org/abs/2505.23817) |
| Reliability | Hallucination (incorrect generation) | Calls non-existent code | INC-043[AI hallucinates software packages and devs download them – even if potentially poisoned with malware](/evals-taxonomy/incident-database#inc-043) | PAP-006[We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs](https://arxiv.org/abs/2406.10279) |
| Instruction non-compliance | INC-027[Vibe Coding Fiasco: AI Agent Goes Rogue, Deletes Company's Entire Database](/evals-taxonomy/incident-database#inc-027) | PAP-005[InFoBench: Evaluating Instruction Following Ability in Large Language Models](https://arxiv.org/abs/2401.03601) |
| Inter-deliverable inconsistency | INC-015[Deloitte to pay money back to Albanese government after using AI in \$440,000 report](/evals-taxonomy/incident-database#inc-015) | PAP-005[InFoBench: Evaluating Instruction Following Ability in Large Language Models](https://arxiv.org/abs/2401.03601) |
| Hallucination (incorrect information) | Outdated facts | INC-044[What Air Canada Lost In ‘Remarkable’ Lying AI Chatbot Case](/evals-taxonomy/incident-database#inc-044) | PAP-007[HalluLens: LLM Hallucination Benchmark](https://arxiv.org/abs/2504.17550) |
| Premature assertions | INC-035[Apple pauses AI notification summaries for news after generating false alerts](/evals-taxonomy/incident-database#inc-035) | PAP-007[HalluLens: LLM Hallucination Benchmark](https://arxiv.org/abs/2504.17550) |
| Ground truth contradictions | INC-044[What Air Canada Lost In ‘Remarkable’ Lying AI Chatbot Case](/evals-taxonomy/incident-database#inc-044) | PAP-007[HalluLens: LLM Hallucination Benchmark](https://arxiv.org/abs/2504.17550) |
| Misleading facts | INC-035[Apple pauses AI notification summaries for news after generating false alerts](/evals-taxonomy/incident-database#inc-035) | PAP-007[HalluLens: LLM Hallucination Benchmark](https://arxiv.org/abs/2504.17550) |
| Misleading confidence | INC-005[Sullivan & Cromwell law firm apologizes for AI 'hallucinations' in court filing](/evals-taxonomy/incident-database#inc-005) | PAP-007[HalluLens: LLM Hallucination Benchmark](https://arxiv.org/abs/2504.17550) |
| Misleading citations | INC-005[Sullivan & Cromwell law firm apologizes for AI 'hallucinations' in court filing](/evals-taxonomy/incident-database#inc-005) | PAP-007[HalluLens: LLM Hallucination Benchmark](https://arxiv.org/abs/2504.17550) |
| Self-contradiction | INC-030[Minnesota solar company sues Google over false information in AI summary](/evals-taxonomy/incident-database#inc-030) | PAP-007[HalluLens: LLM Hallucination Benchmark](https://arxiv.org/abs/2504.17550) |
| Incorrect tool call | Incorrectly executed action | INC-028[Google Gemini deletes user’s code: ‘I have failed you completely and catastrophically’](/evals-taxonomy/incident-database#inc-028) | PAP-008[Benchmarking Failures in Tool-Augmented Language Models](https://arxiv.org/abs/2503.14227) |
| Unprompted action | INC-037[Research AI system attempts to modify experiment code to extend runtime](/evals-taxonomy/incident-database#inc-037) | PAP-008[Benchmarking Failures in Tool-Augmented Language Models](https://arxiv.org/abs/2503.14227) |
| Incomplete action | INC-028[Google Gemini deletes user’s code: ‘I have failed you completely and catastrophically’](/evals-taxonomy/incident-database#inc-028) | PAP-008[Benchmarking Failures in Tool-Augmented Language Models](https://arxiv.org/abs/2503.14227) |
| Security | Unauthorized actions | Unauthorized transaction | INC-033[OpenAI's Operator Agent Reportedly Executed Unauthorized \$31.43 Transaction Despite Safety Protocol](/evals-taxonomy/incident-database#inc-033) | PAP-009[Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://arxiv.org/abs/2412.14470) |
| Unauthorized communication | INC-023[Infosec hounds spot prompt injection vuln in Google Gemini apps](/evals-taxonomy/incident-database#inc-023) | PAP-009[Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://arxiv.org/abs/2412.14470) |
| Unauthorized system actions | INC-001[How OpenAI Lost Control of an AI Model—and What Needs to Change](/evals-taxonomy/incident-database#inc-001) | PAP-009[Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://arxiv.org/abs/2412.14470) |
| Unauthorized cross-system data transfer | INC-019[Prompt injection – and a \$5 domain – trick Salesforce Agentforce into leaking sales](/evals-taxonomy/incident-database#inc-019) | PAP-009[Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://arxiv.org/abs/2412.14470) |
| Insecure outputs | Hyperlinks (URLs) | INC-003[Cloned sites: the shopping scams that lead ChatGPT to fake stores](/evals-taxonomy/incident-database#inc-003) | PAP-010[SafeGenBench: A Benchmark Framework for Security Vulnerability Detection in LLM-Generated Code](https://arxiv.org/abs/2506.05692) |
| Code patterns or blocks | INC-048[GitHub Copilot, Amazon Code Whisperer sometimes emit other people's API keys](/evals-taxonomy/incident-database#inc-048) | PAP-010[SafeGenBench: A Benchmark Framework for Security Vulnerability Detection in LLM-Generated Code](https://arxiv.org/abs/2506.05692) |
| Insecure action | Install insecure code | INC-026[Amazon's AI coding agent was hacked - update now to avoid possible risks, users warned](/evals-taxonomy/incident-database#inc-026) | PAP-011[OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents](https://arxiv.org/abs/2506.14866) |
| Fatal code execution | INC-027[Vibe Coding Fiasco: AI Agent Goes Rogue, Deletes Company's Entire Database](/evals-taxonomy/incident-database#inc-027) | PAP-011[OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents](https://arxiv.org/abs/2506.14866) |
| Safety | Physical harm | Dangerous instructions | INC-049[Jailbreak tricks Discord’s new chatbot into sharing napalm and meth instructions](/evals-taxonomy/incident-database#inc-049) | PAP-012[Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Making](https://arxiv.org/abs/2505.19933) |
| Medically unsafe guidance | INC-022[Man who asked ChatGPT about cutting out salt from his diet was hospitalized with hallucinations](/evals-taxonomy/incident-database#inc-022) | PAP-012[Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Making](https://arxiv.org/abs/2505.19933) |
| Self-harm content | INC-020[Meta AI Companions Unsafe for Kids, Common Sense Media Report Finds](/evals-taxonomy/incident-database#inc-020) | PAP-013[INTIMA: A Benchmark for Human-AI Companionship Behavior](https://arxiv.org/abs/2508.09998) |
| Suicide content | INC-021[Parents of 16-year-old sue OpenAI, claiming ChatGPT advised on his suicide](/evals-taxonomy/incident-database#inc-021) | PAP-013[INTIMA: A Benchmark for Human-AI Companionship Behavior](https://arxiv.org/abs/2508.09998) |
| Emotional harm | Hostile / aggressive behavior | INC-036[Google AI chatbot responds with a threatening message: "Human … Please die."](/evals-taxonomy/incident-database#inc-036) | PAP-013[INTIMA: A Benchmark for Human-AI Companionship Behavior](https://arxiv.org/abs/2508.09998) |
| Triggering content | INC-036[Google AI chatbot responds with a threatening message: "Human … Please die."](/evals-taxonomy/incident-database#inc-036) | PAP-013[INTIMA: A Benchmark for Human-AI Companionship Behavior](https://arxiv.org/abs/2508.09998) |
| Inappropriate content | Harassment | INC-045[DPD error caused chatbot to swear at customer](/evals-taxonomy/incident-database#inc-045) | PAP-014[SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://arxiv.org/abs/2402.05044) |
| Hate speech | INC-014[OpenAI launch of video app Sora plagued by violent and racist images: ‘The guardrails are not real’](/evals-taxonomy/incident-database#inc-014) | PAP-014[SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://arxiv.org/abs/2402.05044) |
| Sexual content excluding minors | INC-012[Sales of AI-enabled teddy bear suspended after it gave advice on BDSM sex and where to find knives](/evals-taxonomy/incident-database#inc-012) | PAP-014[SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://arxiv.org/abs/2402.05044) |
| Violent content | INC-014[OpenAI launch of video app Sora plagued by violent and racist images: ‘The guardrails are not real’](/evals-taxonomy/incident-database#inc-014) | PAP-014[SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://arxiv.org/abs/2402.05044) |
| Illegal conduct advice | INC-042[NYC’s AI Chatbot Tells Businesses to Break the Law](/evals-taxonomy/incident-database#inc-042) | PAP-014[SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://arxiv.org/abs/2402.05044) |
| Out of scope outputs | INC-050[‘I want to destroy whatever I want’: Bing’s AI chatbot unsettles US reporter](/evals-taxonomy/incident-database#inc-050) | PAP-014[SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://arxiv.org/abs/2402.05044) |
| Code and structured data outputs | INC-013[Chinese Hackers Successfully Used Anthropic's AI for Cyberespionage](/evals-taxonomy/incident-database#inc-013) | PAP-014[SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://arxiv.org/abs/2402.05044) |
| Biased output | INC-014[OpenAI launch of video app Sora plagued by violent and racist images: ‘The guardrails are not real’](/evals-taxonomy/incident-database#inc-014) | PAP-014[SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://arxiv.org/abs/2402.05044) |
| High-risk domain engagement | Medical advice | INC-022[Man who asked ChatGPT about cutting out salt from his diet was hospitalized with hallucinations](/evals-taxonomy/incident-database#inc-022) | PAP-015[TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law](https://arxiv.org/abs/2507.21134) |
| Legal advice | INC-042[NYC’s AI Chatbot Tells Businesses to Break the Law](/evals-taxonomy/incident-database#inc-042) | PAP-015[TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law](https://arxiv.org/abs/2507.21134) |
| Financial advice | INC-042[NYC’s AI Chatbot Tells Businesses to Break the Law](/evals-taxonomy/incident-database#inc-042) | PAP-015[TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law](https://arxiv.org/abs/2507.21134) |
| Crisis advice | INC-008[Character.AI and Google agree to settle lawsuits over teen mental health harms and suicides](/evals-taxonomy/incident-database#inc-008) | PAP-015[TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law](https://arxiv.org/abs/2507.21134) |
| Therapy/psychological advice | INC-032[Instagram's AI Chatbots Lie About Being Licensed Therapists](/evals-taxonomy/incident-database#inc-032) | PAP-015[TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law](https://arxiv.org/abs/2507.21134) |
| Company defined risk | Prohibited topics | INC-012[Sales of AI-enabled teddy bear suspended after it gave advice on BDSM sex and where to find knives](/evals-taxonomy/incident-database#inc-012) | PAP-016[Safe-Child-LLM: A Developmental Benchmark for Evaluating LLM Safety in Child-LLM Interactions](https://arxiv.org/abs/2506.13510) |
| Vulnerable users | INC-020[Meta AI Companions Unsafe for Kids, Common Sense Media Report Finds](/evals-taxonomy/incident-database#inc-020) | PAP-016[Safe-Child-LLM: A Developmental Benchmark for Evaluating LLM Safety in Child-LLM Interactions](https://arxiv.org/abs/2506.13510) |
| Accountability | Transparency | Bypass AI disclosure | INC-039[This Viral AI Chatbot Will Lie and Say It’s Human](/evals-taxonomy/incident-database#inc-039) | PAP-017[RealityTest: How People Probe AI Identity and Whether Models Disclose It](https://arxiv.org/abs/2606.00168) |