Zero Data Retention and HIPAA: what AI coding tools actually do
A developer pastes a function referencing a patient’s Medicaid eligibility window into their AI coding tool. Was that data retained? Logged? Used for training? According to Anand Chandrasekaran, AI Chief/Principal Engineer at Arya Health, most engineering leaders genuinely do not know. IBM’s Cost of a Data Breach Report found that 63% of organizations lack formal AI governance policies.
Chandrasekaran reviewed how Claude Code, OpenAI Codex, and Google Vertex AI actually handle data in practice. The findings matter: healthcare averages $7.42 million per breach, the costliest of any industry according to IBM.
What Zero Data Retention actually means
True ZDR means the provider processes your prompt in real time and immediately discards it: no logging, no retained outputs, no training use. What major providers actually deliver is contractual ZDR, available only to enterprise customers who negotiate for it, with carve-outs. Even under a ZDR agreement, if a session gets flagged for a Terms of Service violation, inputs and outputs can be held for up to two years. ZDR and HIPAA compliance are related but distinct obligations. You can have one without the other, and neither is automatic.
Tool-by-tool breakdown
Claude Code: ZDR is real but tied to a direct Anthropic API commercial contract. Bundled Claude Code seats are not currently covered under Anthropic’s HIPAA-ready offering. Files uploaded via the Files API are retained until explicitly deleted regardless of any ZDR agreement. Metrics logging data is explicitly exempt from ZDR. ZDR does not apply to Claude accessed through AWS Bedrock or Google Vertex AI, which offer separate compliance paths via cloud BAAs.
OpenAI Codex: Enterprise API customers can get ZDR, and Codex supports it at the ChatGPT Enterprise tier. Standard API endpoints retain inputs and outputs for up to 30 days for abuse monitoring. ZDR is not self-serve and requires specific approval. Critically, Codex cloud tasks run in OpenAI-hosted containers and are incompatible with ZDR, even inside a ZDR-enabled organization. Consumer ChatGPT tiers (Free, Plus, Pro, Team) are not HIPAA-eligible under any circumstances. Research cited in the piece shows 38% of employees have shared confidential company data with unapproved AI tools.
Google Vertex AI: Takes a structural approach rather than a contractual one. Google’s AI/ML Privacy Commitment means the platform will not use your data to train or fine-tune models without prior permission, applied to all managed models including Claude via Model Garden. The BAA covers Vertex and HIPAA support is not a special add-on tier. One default behavior to know: published Gemini models cache customer data in-memory with a 24-hour TTL to reduce latency. This can be disabled at the project level.
Where agent workflows silently break ZDR
McKinsey research shows roughly 50% of U.S. healthcare organizations are already using generative AI, with more than 80% having deployed at least one use case to end users. At that scale, these failure modes are production problems, not edge cases.
- Code execution sandboxes retain container data for up to 30 days and are not ZDR-eligible
- The Batch API (commonly used for nightly eligibility recalculations or bulk claim processing) carries a standard 29-day retention policy and is explicitly not ZDR-eligible
- Beta features are categorically excluded from ZDR arrangements until explicitly documented otherwise
The architectural takeaway from Chandrasekaran: route all PHI-adjacent inference through the Messages API, avoid the Batch API for regulated data, and treat any beta feature as non-ZDR by default.
What to do now
- Audit every AI tool your dev team uses. If it is not explicitly covered by your BAA, treat it as not HIPAA-safe for PHI-adjacent work
- ZDR is a contractual relationship, not a product feature. It requires a signed agreement, the right org tier, and explicit opt-in
- Document your compliance posture per routing path. Bedrock and Vertex give a different posture than going direct to model providers
- Implement technical controls (deny rules, code review gates, network isolation) that hold under deadline pressure regardless of memory
- Stay current. Anthropic updated consumer data retention policies significantly in late 2025. The compliance assumptions from six months ago may already be outdated
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
