Conversation Logging and Data Retention in Local AI Stacks
Local AI deployments still need governance across inference, UI, and regulatory layers.
Running a model on your own machine does one specific thing well: it keeps the prompt from ever crossing the network to a provider's servers. A company you don't control never gets a server-side conversation log out of this. Nothing exists for a retention order to attach to. A breach at that provider can't expose a conversation that was never sent there, and a future change to that provider's policy can't retroactively apply to data that never left your building. Those are genuine, substantial wins.
But when inference moves local, data control no longer resolves into a single switch. It splits into at least three separate layers, and each one keeps its own rules: the inference engine itself, the UI sitting on top of it, and whatever external regulation applies regardless of where the model runs. Local LLMs reduce risk because your prompts and documents stay inside a perimeter you control, but that isn't the same as being compliant. A local deployment still needs governance, hardening, and traceability, and you have to build those in, not assume them. What follows works through each of those layers in turn, because treating "local" as a finish line is where most of the actual exposure hides.
What the inference engine layer actually logs, and what it does not
Inference engines vary widely in what they write to disk by default, and most people running one have never checked.
Ollama's server logs don't include prompts out of the box. You have to explicitly set OLLAMA_DEBUG_LOG_REQUESTS=1 to turn that on, and even then the request bodies are written to a temp directory, not the main server log. Ollama's cloud features are a separate concern: cloud-hosted models forward the full prompt to ollama.com, while web search sends only the search query and URL, not the full conversation. For anyone running local-only, OLLAMA_NO_CLOUD=1 shuts that pathway off, and the logs will confirm it with the line "Ollama cloud disabled: true." One more detail matters operationally: Ollama's API port, 11434, has no built-in authentication. Expose it on a shared network and any device on that network can send it prompts and read the responses back. Bind it to localhost, or sit a firewall in front of it.
LM Studio runs on a proprietary codebase, so claims about what its telemetry does or doesn't collect can't be independently checked by reading the source. Using it means trusting the stated policy rather than verifying it directly, which is a different kind of guarantee than an open codebase offers.
Jan takes the opposite approach. Conversations get written as plain JSON files in a predictable location, under a threads/ directory, at paths like ~/Library/Application Support/Jan/data/threads/ on macOS, %APPDATA%\Jan\data\threads\ on Windows, or ~/.local/share/Jan/data/threads/ on Linux. Those files are browsable, exportable, or deletable with nothing more than standard file tools. No database extraction, no special software. Jan is built local-first, and its network activity is limited to model downloads and any cloud extensions a user has explicitly installed.
All three tools work the same way underneath, but each ships with its own defaults. None of them inherently log conversations off the machine. But each one has specific conditions, server logging flags, cloud feature switches, telemetry settings, under which data does leave the device or get written somewhere a user didn't expect. Checking those defaults before deployment is the first piece of work, not an afterthought.
The UI Layer's Separate Logging Surface
A clean inference engine tells you nothing about what the interface running on top of it is doing. The UI layer keeps its own data store, entirely separate from the model backend, and in any setup with more than one user, that store creates visibility across users that administrators have to manage on purpose.
Open WebUI illustrates this clearly. It stores every user account, every conversation from every user, and all uploaded file metadata in one SQLite database. Raw uploaded files sit separately on the filesystem, and RAG embeddings live in their own ChromaDB vector database. An Open WebUI admin can view other users' conversations through the admin panel by default, but not another admin's. That's a deliberate feature built for organizational oversight, not a bug slipped in by accident, and it can be turned off with ENABLE_ADMIN_CHAT_ACCESS=false. Whether it's on or off, the people using the system deserve to know it exists. The audit log ships off by default (AUDIT_LOG_LEVEL=NONE); if you turn it on, it writes to DATA_DIR/audit.log, and if you set ENABLE_AUDIT_STDOUT=true, that same log routes to container stdout instead. As of early September 2026 the stable release was v0.11.3, published August 31, 2026, with v0.12.0 following on October 10, 2026.
Johannes Gutenberg University gives a sense of what this looks like at real scale: the university deployed Open WebUI to serve a large population of students and employees, keeping institutional control over the deployment and the data pipeline underneath it. So an entire university's worth of conversation history sits in one UI-layer data store, and the institution's retention policy governs it, whatever that policy is.
The model itself doesn't complicate this. When multiple users share the same Ollama instance, the model holds no memory between requests, so no cross-user leakage can happen inside it. But the Ollama server logs can still contain prompts from every one of those users if logging is enabled, and Open WebUI's database holds every conversation from every one of them in one place regardless. The isolation at the model layer says nothing about what's happening one layer up.
RAG deployments add a failure mode of their own. A common pattern has the inference API secured with token-based authentication, while the vector database sitting next to it stays open on the internal network with no authentication. In multi-team setups, that gap means retrieval queries can pull documents from a different team's collection entirely, a cross-tenant leak that has nothing to do with the model and everything to do with an unlocked database.
Put together, you might assume your conversation is private because it never touched a cloud server, but you may have no idea it's sitting in an institutional SQLite database, visible to an administrator, retained indefinitely until someone configures otherwise. None of this is unfixable. It just requires a deliberate deletion policy and explicit audit logging, decisions a team has to make.
Zero Data Retention Agreements
Zero data retention, ZDR, is a precise contractual term, and it's also one of the most commonly misread. It covers exactly one link in the chain: the AI provider's own inference pipeline.
With ZDR, the provider processes a prompt, returns a response, and writes neither one to persistent storage. Once that call completes, the input and the output are gone from the provider's systems, and no log of the covered prompt or response content remains, though billing metadata and anything flagged for safety review may still be kept. Training opt-out is a related but separate control, and the two don't automatically travel together. Opting out of training means the provider won't use your inputs to improve its models, but those inputs might still sit in a log, a cache, or short-term storage for abuse monitoring. ZDR removes that intermediate storage altogether, which makes it the stronger of the two protections. An organization can secure training opt-out and still have no ZDR agreement at all, meaning its data sits inside the provider's infrastructure for the full length of whatever retention window applies.
Provider terms carry their own carve-outs, worth checking line by line. As of 2026, Anthropic's frontier Covered Models, Claude Fable 5 and Mythos 5 among them, along with their successors, are excluded from the standard ZDR agreement. Claude 3.7 Sonnet with extended thinking remains ZDR-eligible, but it retains data for up to 30 days, an exception easy to miss during procurement if you don't read the frontier-model exclusion carefully. Since August 2025, Anthropic has defaulted consumer users into training opt-in; staying on that default extends retention to five years, while actively opting out holds it to the standard 30-day window.
None of this extends to anything downstream of the provider call. The internal data warehouse, the meeting assistant's own storage layer, the CRM, any intermediary tool in the chain, all sit outside what a ZDR agreement touches. A meeting assistant can sign a ZDR agreement with its LLM provider and still keep every transcript indefinitely on its own servers, because that storage decision belongs to the assistant, not the model provider.
The gap widens further with shadow AI, the well-documented pattern where employees bring their own AI tools into a workflow with no authorization, no data-protection review, and no contract behind them. An organization can have airtight ZDR terms on every sanctioned tool and still have no visibility into a screening browser extension three people on a hiring team installed on their own. That's a compliance gap no amount of careful procurement on the approved tools can close, because it sits entirely outside the perimeter those agreements were written for.
What EU AI Act Article 12 requires from local deployments starting December 2027 (Annex III systems) or August 2028 (Annex I systems)
Article 12 of the EU AI Act sets a legal floor: automatic, tamper-evident logging for high-risk AI systems. The Digital Omnibus on AI, Regulation (EU) 2026/1744, in force since July 27, 2026, pushed the original timeline out, Annex III high-risk systems now have until December 2, 2027, and AI embedded in Annex I products until August 2, 2028. Most local AI deployments running today generate none of the records this requirement will call for.
Many teams reach first for logging every prompt and response in full, but that creates a second problem. Those audit logs now hold personal data, so they run straight into GDPR's data-minimization principle. Satisfying Article 12 by violating GDPR isn't a resolution, it's a trade.
No official Q&A guidance on Article 12 logging compliance had come out as of mid-2026, so you're still waiting on official direction. In its absence, a workable approach already exists: storing a cryptographic hash of the prompt. A hash creates a record of who used the system and when, without duplicating the sensitive content itself anywhere in the audit trail. If an investigation later needs the full prompt, it can be pulled from the UI layer's own database, Open WebUI's, for instance, using the session ID tied to that hash. The hash satisfies the automatic-recording requirement, and the retrievable record, held separately and accessed only when needed, satisfies the investigative one. Logs built this way also have to be tamper-evident as a technical property, not a policy statement about intentions.
HIPAA runs the same pattern in a different domain. It requires hardware, software, and procedural mechanisms that record and examine activity on any system holding protected health information, every prompt and response logged with user identity attached, and six years of retained documentation behind it. You can satisfy that obligation the same way, with SHA-256 hashing, so you get an audit trail without a second plaintext copy of PHI sitting in a log file somewhere.
Building a logging posture that satisfies privacy, governance, and regulatory obligations across all three layers
Three layers, three different interventions. No single setting closes all three at once, but the hashing technique that resolves the Article 12 and GDPR conflict turns out to be the connective thread running through all of them.
At the inference layer, the work starts with an audit of defaults before anything goes into production: is server logging on, are cloud features active, is the API port bound to localhost or sitting open on the network. For Ollama, that means setting OLLAMA_NO_CLOUD=1 for local-only operation, binding to localhost, and disabling update checks entirely in air-gapped environments. For LM Studio, you turn off telemetry under Settings > Privacy, but the license is proprietary, so you can't confirm that setting by reading the source code yourself. For Jan, the local-first design and AGPLv3 license make the system auditable directly, so the remaining check is simple: confirm no cloud extensions are installed beyond what was explicitly intended.
At the UI layer, Open WebUI's audit log should be turned on deliberately, through the AUDIT_LOG_LEVEL setting, rather than left at its off-by-default state, and routed somewhere tamper-evident rather than left in a file an administrator could quietly edit. A deletion policy for the SQLite database needs to be defined and enforced, since indefinite retention is the default and stays that way until someone overrides it on purpose. If you run any RAG deployment, you have to treat vector database access controls as a core security requirement from the start, and authenticate retrieval queries per team or tenant so one group's documents can't surface in another's results.
At the regulatory layer, the fix is a logging wrapper that stores cryptographic hashes of prompts alongside session IDs, user identity, and classification metadata. That single design choice satisfies Article 12's automatic-recording requirement, HIPAA's audit-trail requirement, and GDPR's data-minimization principle, all three, at once, without one undermining another. Shadow AI still has to be addressed directly: a governance posture only covers the tools it knows about, so identifying unsanctioned AI use and bringing it inside the same perimeter is a precondition for calling any of this compliant. Where cloud providers sit alongside local inference, the ZDR agreements covering them need checking for what they actually include, abuse-monitoring logs, subprocessors, and the frontier-model or extended-thinking carve-outs that standard ZDR terms may not reach.
What ties all of this together is a simple idea: AI that actually supports human judgment depends on the people using it understanding what the system does with their input. A logging posture nobody can see and nobody is accountable for is an unexamined assumption that control exists, left unchallenged until a regulator, an auditor, or a breach forces the question. Teams that get this right are the ones who built that posture as a decision, layer by layer, rather than the ones who let it settle into place by default.
