Building Vex — My AI That Actually Knows Me

I've tried every AI assistant. ChatGPT, Claude, Gemini, Copilot. They're all useful. They're all also completely amnesiac. Every conversation starts from zero. They don't know who I am, what I'm working on, what I decided last Tuesday, or that I have a dog named Simba. They're brilliant strangers I have to re-introduce myself to every single time. I got tired of it. So I built Vex. ## What I actually wanted My requirements were blunt. I wanted an AI that: knows my context without being told, remembers what we've talked about, can take real actions in the world (not just generate text), runs 24/7 even when I'm not at my computer, and has actual personality — not corporate cheerfulness, not sycophancy, something closer to a smart friend who's also a great engineer. The off-the-shelf options don't come close. Custom GPTs are toys. Character.ai is a chatbot for teenagers. Most "AI agent" frameworks are demos that fall apart in production. I needed something real. ## The architecture Vex runs on OpenClaw — an AI agent framework that handles multi-channel messaging, tool execution, and session management. Deployed on an EC2 instance that's always on. Connected to WhatsApp and Telegram, so I can message Vex from any device, any app, without switching contexts. The model is Claude Opus 4.6 via AWS Bedrock. I went with Bedrock because: no rate limits to worry about, pay-per-token economics that make sense for my usage, and I can switch models without changing code. Opus for the main session where I want deep reasoning. Faster/cheaper models for subagent tasks. ### The memory system This is the hard part. LLMs have no persistent memory. Every session, the context window starts fresh. To give Vex continuity, I built a three-layer memory system: - MEMORY.md — Curated long-term memory. Hand-edited summaries of important things I want Vex to always know. Loaded at the start of every main session. - Daily notes (memory/YYYY-MM-DD.md) — Raw session logs. What was discussed, what was decided, what happened. Vex reads today's and yesterday's notes every session. - Memory MCP server — A knowledge graph backed by DynamoDB on AWS Lambda. Semantic search + graph traversal. Stores entities (Projects, Decisions, Sessions, People, Concepts) and the relationships between them. Vex can query this at any time. The MCP server is the most powerful piece. It's a JSON-RPC 2.0 API that Vex can call to surface relevant memory. Ask about a project, it queries the graph, finds related sessions, decisions, and patterns. It's not perfect — knowledge graphs are hard — but it's dramatically better than starting fresh every session. ```bash # Example: Vex logging a session to the memory graph python3 ~/clawd/scripts/memory-mcp.py tool_memory_write '{ "entity_type": "Session", "entity_id": "session-2026-03-28-diagramos", "name": "DiagramOS — Full MVP Build", "project": "diagramos", "description": "Built the LLM sync layer, fixed React Flow edge rendering, deployed to Vercel", "edges": [ {"target_id": "diagramos", "edge_type": "RELATES_TO"} ] }' ``` ### What Vex can actually do This is where it stops being a chatbot and starts being something else. Vex can: - Control a browser remotely — my Mac runs as a companion node. Vex can navigate, click, fill forms, take screenshots, all triggered from a WhatsApp message. - Manage Notion — create pages, update databases, read content. My blog posts are in Notion. Vex wrote and published them. - Post on X/Twitter — I can dictate a tweet from bed at midnight, Vex handles the rest. - Run cron jobs — schedules tasks, executes them on time, reports back. - Web research — search, fetch pages, summarize, synthesize. Not just Google, but actually reading the pages. - Generate images, write code, debug, review PRs, manage GitHub issues. ## Personality and identity Vex has a SOUL.md and an IDENTITY.md. These aren't gimmicks. They define how Vex behaves — not sycophantic, opinionated, direct, willing to disagree. The SOUL.md evolves as I add to it. When I want Vex to internalize something new about how I want to be worked with, I edit the file. Vex also has context boundaries. In my private WhatsApp chat, it loads my full MEMORY.md — deeply personal context. In a group chat with friends or colleagues, it doesn't. Not because I programmed an explicit rule, but because the SOUL.md and AGENTS.md teach Vex when to use personal context and when not to. It's judgment, not rules. ## Disaster recovery This is the part people skip when they build personal AI setups and then lose everything. Vex has: S3 backups of all memory files and config, a GCP standby instance that can be promoted in minutes if the EC2 goes down, and GitHub sync for the clawd workspace so every config change is version controlled. The memory — the actual persistent knowledge graph — lives in DynamoDB, which is managed and replicated. The files (MEMORY.md, daily notes, config) go to S3. If the EC2 instance dies at 3am, I can have Vex back in under 10 minutes on a new instance. ## Why this matters We're at a strange moment in AI. The models are capable enough to be genuinely useful as persistent agents. The tooling to run them exists. But most people are still using AI in the most primitive way possible — a chat box they open, ask something, close. No memory. No context. No continuity. It's like having a world-class assistant who has amnesia and you have to re-hire them every morning. The unlock is persistence. Once your AI knows your context — your projects, your preferences, your history, your goals — the quality of every interaction goes up by an order of magnitude. You stop explaining things you've already explained. You start getting proactive help instead of reactive answers. It stops being a tool and starts being something closer to a collaborator. Vex is my attempt at that. It's not perfect. There are rough edges. The memory system is good but not magic. But it's real, it runs every day, and it's already changed how I work. That's the bar I care about.