What to Do When ChatGPT Makes Up Legal Cases
Discovered that ChatGPT invented a legal case? Don't panic. Follow this step-by-step guide to identify, verify, and correct AI hallucinated citations.
Don't Panic — Stay Calm and Assess
Discovering a fabricated citation can be alarming, especially if you've already shared the document. Take a deep breath and approach this systematically.
Immediate actions:
- Do not delete the AI output — you may need it for reference
- Do not share the document further until verification is complete
- Document what you found — note which citations appear suspicious
Extract and Isolate All Citations
Separate all citations from the document content. This lets you verify each one without the distraction of surrounding text.
How to extract citations:
- Copy all text that looks like a citation (parenthetical references, footnotes, etc.)
- Paste into a separate document or directly into CiteClear
- Remove any non-citation text
- Keep a list of which citations came from which parts of your document
Run First-Pass Triage with CiteClear
Use CiteClear to quickly identify which citations are most likely to be problematic. This helps you prioritize your manual verification efforts.
What CiteClear will flag:
- High-risk: Impossible reporter numbers, non-existent courts, future dates
- Medium-risk: Very high numbers, generic party names, missing components
- Low-risk: Well-formatted citations that may still need verification
Manually Verify Flagged Citations
For each citation flagged as suspicious or that you couldn't verify through CiteClear, conduct manual verification.
Verification sources:
Correct or Remove Fabricated Citations
For each problematic citation, decide whether to correct it with a real source or remove it entirely.
For each unverified citation:
- If it supports an important point: Find a real citation that supports the same claim
- If it's non-essential: Remove the citation and the associated claim
- If the claim is valid but the citation is wrong: Replace with the correct citation
- If you can't find a replacement: Remove the claim entirely
Review and Test the Entire Document
After addressing all flagged citations, review the entire document to ensure no issues remain.
Final review checklist:
Document Your Process
Create a record of what you found and how you addressed it. This is especially important for legal filings.
Documentation to keep:
- Original AI output (if still available)
- CiteClear triage report
- List of citations that were flagged
- Verification results for each citation
- Changes made to the document
- If applicable, disclosure of AI use to court or client
Disclose AI Use (If Required)
Determine if you need to disclose that you used AI in creating the document.
When to disclose:
- Court filings: Many jurisdictions now require disclosure of AI use
- Client deliverables: Consider disclosing as a matter of transparency
- Academic work: Follow journal or institution policies
- Public documents: Disclose to maintain transparency and trust
What to disclose:
- That AI tools were used in preparing the document
- Which AI tools were used
- What steps you took to verify AI-generated content
- That all citations have been manually verified
Prevent Future Issues
Use this experience to improve your AI workflow going forward.
Prevention strategies:
- Always run CiteClear: Check all AI-generated citations before use
- Verify as you go: Check citations during drafting, not just at the end
- Use checklists: Create a citation verification checklist for your team
- Train your team: Ensure everyone knows how to identify and verify AI citations
- Implement reviews: Have a second person verify AI-generated work
Important Reminders
- You are responsible: Even if AI generated the citation, you are responsible for the accuracy of your work
- Courts are taking this seriously: Lawyers have faced sanctions for submitting AI-generated fake citations
- Transparency builds trust: Disclosing AI use and your verification process demonstrates professionalism
- When in doubt, leave it out: If you can't verify a citation, don't use it
Quick Reference: Immediate Actions
STOP
Stop sharing the document
EXTRACT
Extract all citations
CHECK
Run CiteClear triage
VERIFY
Manually verify flagged
FIX
Correct or remove fakes
DOCUMENT
Document your process
INFORM
Disclose if required
PREVENT
Improve future workflow
Frequently Asked Questions
What should I do first if I find a fake citation?
First, stop sharing the document containing the fake citation. Then, extract all citations and run them through CiteClear for immediate triage. This will help you quickly identify which other citations might also be problematic.
Should I tell my client or supervisor?
Yes, it's generally best to be transparent. Explain that you discovered a potential issue with AI-generated content and that you're taking steps to verify and correct it. This demonstrates professionalism and responsibility.
Can I get in trouble for using AI that hallucinated?
The key factor is what you did after discovering the issue. If you caught and corrected the fake citation before anyone relied on it, you're in a much better position. If you submitted it without verification, you could face consequences depending on the context (court filings are taken most seriously).
How do I know if other citations are also fake?
You don't know until you check. That's why it's essential to verify ALL AI-generated citations, not just the one you already identified as fake. Use CiteClear to triage them all, then manually verify the flagged ones.
What if I already filed something with a fake citation?
If you've already filed a document with a court or submitted it to a client, you need to take corrective action immediately. Consult with a supervisor or legal ethics counsel. You may need to file a corrected version and disclose the error to the court or client.
Can I use AI at all for legal work?
Yes, but with extreme caution and proper verification. AI can be useful for drafting, brainstorming, and research assistance, but you must always verify everything it produces. Never trust AI-generated content without independent verification.
What's the safest way to use AI for legal work?
Use AI as a starting point only. Treat all AI output as potentially inaccurate. Verify every fact, every citation, and every claim. Consider using AI for brainstorming ideas but writing the final product yourself, with proper verification.
Should I disclose AI use even if not required?
It's often a good idea. Transparency builds trust and protects you if issues are later discovered. It also demonstrates that you took proper verification steps. Many organizations are adopting policies requiring AI disclosure.
Don't let AI hallucinations catch you off guard
Use CiteClear to check all AI-generated citations before they become a problem.