Building a Self-Hosted AI Agent with OpenCode and Shared Memory

Overview

Software engineer Victor Parmar has developed a streamlined approach to self-hosting AI agents that prioritizes continuity, security, and ownership. By combining the OpenCode harness with a shared memory architecture synced via Dropbox, Parmar has created an environment that bridges the gap between web-based conversations and local development workflows.

The project, which took approximately 90 minutes to implement, was designed to address the fragmentation common in modern AI usage. With subscriptions scattered across multiple providers and chats distributed between web interfaces and local machines, maintaining context often requires tedious manual re-entry. This setup aims to eliminate that friction while keeping the user in full control of their data and security boundaries.

The first problem was consolidating memory across machines and harnesses. My own setup is spread thin: subscriptions with several providers, chats on the web, local sessions on different machines, and project work scattered across all of them.

The Shared Memory Architecture

A central tenet of Parmar’s solution is the concept of shared memory. While individual model invocations are stateless, building upon previous discussions requires relevant context to be available to the agent. Many existing systems manage this through proprietary APIs or closed-loop conversation histories, but this approach defines a universal, file-based memory layout.

The system relies on a simple directory structure hosted in a cloud-synced folder, such as Dropbox or Google Drive. Each AI harness is instructed via global configuration files, such as AGENTS.md or CLAUDE.md, to read from and update these notes. The layout includes:

• A root MEMORY.md containing rules, conventions, and links to detailed notes.
• Individual .md files for detailed concept notes.
• Project-specific subdirectories containing summaries tailored for smaller models.

This architecture ensures that insights are not lost when switching between tools. However, to prevent sync conflicts and maintain a lean context window, every write to memory requires manual confirmation. This discipline is crucial because every token dedicated to memory is one less token available for the actual work at hand. It also serves as a critical security measure, ensuring that sensitive information is only persisted where the user explicitly approves.

Why Build Custom Rather Than Use Existing Platforms?

While open-source platforms like OpenClaw, Agent Zero, and Hermes Agent offer robust features for persistent memory and background work, they did not align with Parmar’s specific requirements. The decision to build a custom solution was driven by two primary factors: foundation preservation and security ownership.

First, the engineer already had a functioning memory layout shared across multiple harnesses. He sought a platform that could sit atop this existing foundation rather than forcing a migration that would require reshaping the foundation itself.

Second, and perhaps more importantly, was the desire for complete control over the security model. Pre-built solutions allow for customization, but drawing strict boundaries from day one is more straightforward when assembling individual components. The goal was to define exactly what agents can access, where they can execute commands, how isolation is maintained, and which actions require human approval.

I want to own the security model: what agents can access, where they can execute commands, how they are isolated and which actions require my approval.

Technical Implementation and Security Boundaries

The resulting environment is surprisingly simple, comprising an Ubuntu VPS running OpenCode, with models accessed via OpenRouter and an OpenAI subscription. The shared memory is synced to the virtual private server using the headless Dropbox client on a fixed schedule. Access to the web UI is secured through Tailscale, ensuring private connectivity without exposing the service to the public internet.

The security boundaries are rigorously defined:

1. User Isolation: OpenCode runs under a non-sudo account on the VPS.
2. Network Restrictions: There is no egress allowed to other machines on the Tailscale network.
3. File System Limits: Writes are strictly limited to working directories, with read access restricted elsewhere.
4. Credential Minimization: The system holds no credentials beyond the necessary model provider keys, which have usage limits, and the Dropbox account, which is separate and limited to the memory folder only.

The implementation process highlighted the efficiency of modern AI coding assistants. Parmar noted that while he spent most of the initial phase insisting the AI provide instructions to be run manually, he eventually delegated the final 20% of the setup, including SSH configuration, to the agent itself.

Conclusion

This self-hosted agent environment offers a compelling alternative to proprietary platforms for engineers who value continuity and security. By decoupling memory from specific providers and leveraging open-source harnesses, users can maintain a coherent knowledge base across devices and contexts. The approach demonstrates that a robust, always-on AI workspace can be assembled quickly, provided the user retains ultimate authority over the underlying infrastructure and data flow.