Five trade-offs from building the runtime
Agent runtimes have a short feature list and deep failure modes. These five decisions were each reversed at least once; they are written down because every one of them looks like it should have gone the other way.
1. Isolation in the database, not the application
Our first version added WHERE tenant_id = ? to every query. It worked for three months, then leaked at the 41st entry point — not from carelessness, but because the design required being right 41 times.
Now every business table carries a tenant_id, and PostgreSQL row-level security enforces the filter inside the database; a session variable decides who the connection is. Forget the predicate and you get zero rows, not someone else’s data.
The costs are honest:
- every statement pays one policy check (microseconds);
- migrations, jobs and ops CLIs must set the tenant context explicitly or they cannot see anything either;
- cross-tenant reads (platform admin) need an audited escape hatch, not an opportunistic
SELECT.
Trading 41 “remember the predicate” moments for one “it simply cannot leak” is invisible in month one and obvious in month four.
2. SSE for streaming, not WebSocket
Agent output is one-directional, long and interruptible: a reply may run for minutes, interleaved with tool-call events.
WebSocket is more flexible, but there is no second direction to justify it: sending a message is a plain POST, the reply is a server-sent event stream. SSE means:
- standard HTTP, so proxies, auth middleware, logging and timeouts already work;
- reconnection follows the browser’s
EventSourcesemantics; - no hand-rolled connection state machine or heartbeats.
The price is that you cannot push arbitrary things — “another session finished” — down the same pipe. For that, a light poll or a second channel is far cheaper than living on one bidirectional socket.
3. Sandboxes are disposable; sessions are not
A sandbox lives for a task, not for an account. When the task ends, the container is reclaimed.
That has an uncomfortable consequence: users ask where the package they installed went. It is gone — but the files are in the session snapshot. So we separate three lifetimes:
| Thing | Lifetime | Stored in |
|---|---|---|
| Processes, temp dirs | Task | Sandbox instance |
| Files, artefacts | Session | Session file store |
| Skills, prompts, configs | Tenant | Database |
Once the boundary is explicit, “why did my environment change” mostly disappears — the environment was never meant to be depended on; files were.
4. Failures go into context, not just logs
The most common agent failure is a non-zero exit reported as “execution failed”. The model cannot see what broke, so it starts guessing.
Instead, put stderr, the exit code and truncated raw output into the tool result, and say plainly: this is the first N lines, the full log is at this path. Models do not need protection; they need real data.
The same applies to our own UI: a “copy diagnostics” button beats “please try again later”.
5. Slower, but explainable
Some choices trade performance for explainability:
- No implicit retries. A retry is a second ledger entry, and users notice being charged twice.
- No silent model fallback. If the primary model dies, we fail loudly instead of switching to a cheaper one mid-conversation.
- Every tool call is recorded. A task may produce forty events, all inspectable. Storage is not free, but “what did the AI actually do” must have an answer.
The common thread
All five answer one question: when something goes wrong, can the system say what happened?
Agent systems are good at producing a plausible explanation of a failure. The runtime’s job is to make “what actually happened” exist first — real exit codes, real isolation boundaries, real billing records.
That is why multi-tenancy, auditing and quotas come before features: not because they are exciting, but because they decide whether anyone dares hand over real work.