Local AI Assistants for Managing Multiple Projects Without Losing Thread

For multi-project work, the defining advantage of local AI assistants is persistent, on-device context. A useful setup retains each project’s decisions, constraints, and next steps, then retrieves them when work resumes. That removes the need to reload the project by hand, provided memory stays project-scoped, editable, and separate from the user’s judgment.

Why multi-project work breaks AI assistants at the session boundary

A developer returns to Project A after a day on Project B and opens a new assistant session. Before useful work can begin, architecture documents go back into the chat, followed by decisions already made and approaches already rejected. The work starts with explaining yesterday’s work. A question that should move the project forward instead requires a reconstruction of how it reached this point.

For a stateless assistant, that reconstruction is necessary because each session starts with a blank context window. The user becomes the external memory layer. Across concurrent projects, the re-loading tax compounds as each return demands another account of the project’s state. This is distinct from the mental effort of switching tasks, although it adds to that effort. Better personal discipline cannot give a stateless tool a durable record it was never built to keep.

The difference between a context window and persistent memory, and why it matters for project work

A context window is the material a model can attend to during an interaction. Making it larger gives the model more room to work, but does not by itself preserve awareness between sessions. Memory serves a different purpose. It stores information externally, in files or a database, so future sessions can retrieve and update it.

There is also a middle tier called session-based memory, which persists through a multi-step workflow but resets on logout. That can support a task completed in one sitting. It does not solve the return to a project several days later. For that, the relevant capability is long-term, project-scoped memory, which can retrieve Project A’s decisions without importing Project B’s constraints.

Even within a session, the context window has a limit. When the available space is exceeded, early material can be dropped, so a long conversation is not a reliable archive. Persistent memory keeps work connected by retrieving the parts that are needed, without keeping every past exchange in active attention.

Why local over cloud for holding this context

Keeping the archive on-device separates ownership of project knowledge from access to a particular assistant. The complete archive need not live with the AI provider, even when a cloud model handles a request. Only the context needed for that interaction has to be transmitted. Local storage therefore offers portability as well as control over sensitive material.

That difference matters when the work includes proprietary code, API keys, architecture documents, or customer data. Third-party retention is a real organizational risk. Service dependency is another. Khoj Cloud shut down on April 15, 2026, requiring hosted users to migrate while the open-source, self-hosted project remained available. Accumulated project knowledge is less exposed to that kind of disruption when its storage is under the user’s control.

Local tools can still be hard to use or leave some data outside the device. Some tools keep the model on-device, some keep the data there, and some keep both. Local inference also constrains the usable context window to keep RAM demands manageable, while local tooling generally needs more configuration than hosted alternatives. The architectural gain is durable ownership of the context layer.

How persistent memory actually works mechanically in local assistants

Retrieval-augmented generation, or RAG, means searching your own archive and injecting the relevant parts before each reply. The tool stores conversations and project documents, searches them when a question arrives, and adds selected material to the prompt. Continuity comes from the external store and the retrieval process.

How well retrieval works determines how useful that continuity is. Stronger implementations combine semantic search, which finds related meaning, with lexical search, which matches words, and isolate results by project. A system that retains only session history offers less. It may extend a conversation without preserving the decisions needed next week.

With tiered agent memory, the assistant manages the store. Core memory holds facts in use, recall holds recent history, and archival memory holds older material that is retrieved when needed. The agent decides what to save, edit, or discard. This reduces manual curation but makes those decisions another part of the system the user may need to inspect.

Git-backed markdown takes a more explicit approach, keeping knowledge in readable files tracked by version control. Users can inspect, edit, move, and share that record across tools or machines. The trade-off runs through all three mechanisms. Automatic capture reduces saving effort, while structured files make correction easier but require deliberate upkeep.

The tools that implement persistent local context and what each one actually does

Khoj brings fragmented project notes into one retrieval layer. It indexes documents across Obsidian, Notion, and scattered PDFs, connects to local or cloud LLMs, and acts as an autonomous knowledge assistant. Its open-source project is AGPL 3.0 licensed. Since the cloud shutdown, users need infrastructure for self-hosting to run it, so it offers one place to reach existing knowledge but takes setup before it can work as a consumer assistant.

Jan addresses the simpler need for a local model runner with project-scoped documents. Jan v0.7.7 added file uploads to its existing Projects feature, so conversations in a project can refer to the documents without a separate RAG pipeline. It also added native MLX support on Apple Silicon. Conversation history survives switching between local and cloud models, while zero telemetry and auditability support users who want a transparent runner. The trade-off is some raw inference speed and advanced functionality.

Letta, formerly MemGPT, addresses the burden of deciding what an assistant should retain. It treats the LLM like an operating system, with the agent managing core, recall, and archival memory over time. Curation becomes part of the assistant’s job. That suits users who want the assistant to manage memory on its own, but they have to set it up to see exactly what it saves.

Basic Memory addresses the need for a knowledge base that remains readable and portable across assistants. It stores memory in local Markdown files and makes it available through Model Context Protocol, or MCP. The record stays open to direct editing. That makes it a fit for writers and researchers who value explicit control, although accumulation requires more intentional curation than ambient capture.

Pieces for Developers addresses context scattered across a developer’s toolchain. It runs in the background, automatically capturing work context across browsers, IDEs, and collaboration tools, and costs $18.99/user/month. Its Long-Term Memory is available through an MCP server to compatible assistants including Claude and Cursor. Saving requires little deliberate effort, but knowing exactly what the growing store contains requires active auditing.

ai-memory, available as akitaonrails/ai-memory, addresses portable memory across machines and small teams. Its source of truth is a git-backed markdown wiki, and multiple users can point agents at one server to retrieve shared project knowledge while personal handoffs stay personal. It requires comfort with git-based workflows. It is also a newer project with less production maturity than the other options. Across these designs, MCP is increasingly the connective layer, making the memory store and the assistant interface separable choices.

How to structure local memory so it actually holds the thread across concurrent projects

Start by isolating each project through folders, namespaces, or separate knowledge bases. Projects must stay separate during retrieval as well as in the files. Otherwise, as more projects are added, constraints from other projects can slip into answers and make the assistant less useful.

Seed each store with the project’s current state, standing decisions, constraints, and rejected approaches. For Basic Memory or ai-memory, create a README-style file containing status, decisions, open questions, and constraints, then commit it before the first session. For Khoj, attach the relevant documents at project initialization. For Letta, write core memory entries explicitly.

Context settings need attention too. When using Ollama, set the context window explicitly and high enough for typical project sessions rather than relying on a default that can silently drop early material when exceeded. Before closing Project A, save a brief handoff for the next session stating what was decided, what comes next, and what is blocked.

Keep the store inspectable and editable over time. A five-minute review per project per week can correct stale entries and remove resolved decisions that no longer belong in active retrieval. The task is small but deliberate. Without pruning, accumulated memory can become a source of distraction.

The real limits of persistent local context and the objections worth taking seriously

Persistent storage does not resolve contradictory knowledge. If one session records that a project is on track and another records that it is delayed, both claims can remain in memory. Retrieval may return the claim that scores higher even if it is wrong. There is no consensus mechanism that settles such conflicts, and conflict resolution remains an unsolved problem in production multi-agent systems. An archive can be durable and still be wrong.

As assistants process larger and more complex project contexts, users may become more willing to delegate judgment, reducing their own engagement with the material. That is a real cognitive-delegation risk. The assistant that holds the thread can gradually become the assistant that owns it, undermining the understanding that made delegation useful in the first place.

Memory should bring up past decisions, constraints, and evidence while the human makes the decisions. Reminding someone what was decided is different from deciding for them. Richer memory is not automatically better if it displaces the user’s grasp of the project.

Inspectability creates a further choice with no effortless solution. Ambient capture, as in Pieces, avoids the gaps caused by forgetting to save, but its growing store is harder to audit. Readable formats, as in Basic Memory or ai-memory, expose the record but depend on someone maintaining it. Which failure is more tolerable for the work, invisible stale context or useful context that was never captured?

Choosing a setup based on how your projects are actually structured

The decision starts with how structured the project artifacts already are and how much configuration the user will sustain. For established notes in Obsidian or Notion, self-hosted Khoj is the most direct retrieval path because it works from the existing knowledge base. For developers who want to save context across their toolchain with little effort, Pieces gathers it in the background and lets them reach it through MCP from the compatible assistant they prefer. Capture maintenance is low, but auditing remains a separate responsibility.

For readable, portable records, Basic Memory or ai-memory puts project knowledge in files the user controls. The latter adds shared project knowledge for a small team, with the trade-off of a newer project and git-based work. Letta fits users who want the agent to manage retention, provided they accept the need to configure inspectability. These choices answer different questions about who curates the archive.

For a lower-complexity entry point, Jan’s Projects feature supports project-scoped document attachment without separate memory infrastructure. It suits users who want a local runner and the option to move demanding tasks to a cloud model. The documents still need deliberate selection and maintenance. No interface removes the need for clear project boundaries or a current account of decisions.

The goal is a setup the user will maintain. Project isolation, deliberate seeding, explicit context settings, and periodic review turn storage into useful continuity. A more sophisticated architecture is not the better choice if its upkeep will gradually be abandoned. A simple store that stays up to date works better across projects than a complex one nobody trusts enough to use.