ProustGPT

Why Proust

This is the oldest project on this site, and the one I keep returning to. I started it in 2024, when I was first studying how language models actually work — I wanted a real problem to learn RAG on, and I happened to be reading Proust. In Search of Lost Time turned out to be a perfect stress test: 1.5 million words across seven volumes, in two languages, full of questions that a keyword search can't touch. "Where does Albertine first appear" is easy. "Compare Swann's jealousy to Marcel's" is not.

What it is now

The app has two modes, and they're for two different people. Explore Lost Time is for someone reading the book: ask a question, get an answer built from cited passages in English or French. Reflect on My Day is for someone using the book as a lens: you journal about your day, and an agent responds in Proust's introspective register, pulling in passages only when one genuinely fits.

The tempting way to build that would have been one chatbot with a personality switch. I made the modes an architectural contract instead: different endpoints, different tool budgets, different routing. Explore gets the full set of retrieval tools; Reflect gets three, used sparingly, and defaults to plain conversation. The mode changes what the system can do, not just how it talks.

How a question gets answered

Explore mode runs a two-stage pipeline: the query is embedded (Cohere embed-v4.0), matched against Pinecone across paired English and French namespaces (20 candidates), reranked down to the best five (Cohere rerank-v3.5), and answered by Llama 3.3 70B on Groq, streamed token by token over SSE. Reflect mode puts an agent in front of the same retrieval tools.

Not every question deserves the agent, though. A lightweight router decides at call time: simple lookups go to fast RAG (~2–3 seconds), comparative and follow-up questions go to the agent loop (~5–15 seconds). The router is deliberately dumb — regex patterns and conversation-depth checks, no ML. A wrong call costs a few hundred milliseconds, not a failed response, and spending complexity only where errors are expensive was one of the better product decisions in the project.

Keeping retrieval honest

Retrieval is scored against a hand-reviewed gold set: 111 questions (100 synthetic, generated to avoid verbatim leakage, plus 11 hand-written), split 77 dev / 34 held-out test. On held-out, the production config scores Recall@5 = 0.82, MRR = 0.69, p50 latency 551ms. Reranking earns its keep — Recall@1 goes from 0.47 to 0.59 over vector search alone. One variant beat the baseline in tuning and gave the whole gain back on held-out, which is exactly what the held-out split is for; it's reported in the repo along with every other review decision.

The July 2026 hardening pass

Two years in, the project had accumulated the kind of debt a solo prototype accumulates. In July 2026 I ran a structured review across the whole codebase and shipped fixes for everything it surfaced. My favorite find: the Reflect agent was accumulating source passages in shared mutable state, so under concurrent requests one user's citations could bleed into another's response — invisible in single-user testing, guaranteed to happen in production. Also in the pass:

• The agent's "streaming" was fake — it generated the full response, then chunked it out. It now streams real tokens.
• Follow-up questions were sometimes misrouted between the fast path and the agent; the router now handles conversation state correctly.
• Response caching, per-request routing logs, input caps, and rate limiting — the unglamorous production layer.
• Accessibility and mobile fixes: focus states, contrast, and a broken page-navigation control in the mobile reader.
• A new feature that had been on the list for a while: guided reading paths — curated itineraries through the novel, served from their own endpoint.
• 113 backend tests now cover the agent and retrieval logic, and the legacy evaluation script is gone in favor of the gold-set harness.

How it runs

The whole thing ships as one Docker image on Render: Vite builds the React app, FastAPI serves the static bundle and the API from the same process, Pinecone runs serverless. I considered splitting the frontend onto Vercel — the code already supports it — and decided not to: one service is simpler to operate, and simple is what a solo-maintained production app should be. Next on the list are server-side sessions for chat history, going async end to end, and replacing the regex router with an embedding-based one.

Outcome

A single-purpose chatbot became a two-mode product with agentic orchestration — live on Render, open source on GitHub. The full set of decisions, including the ones I'd revisit, is written up in Redesigning ProustGPT.