How ClawAI is built

Eighteen independently deployable services, each owning its own database, coordinated over an event-driven backbone. Responses stream as they are generated, and every layer is defended on its own.

Last reviewed 2026-07-25

How ClawAI is built

ClawAI is eighteen independently deployable services behind a single reverse proxy. Authentication, chat, routing, connectors, memory, files, research, workspace, image generation, document export, billing, audit and logging each run as their own process with their own database. The web app reaches all of them through one API surface.

The reason is containment. A slow provider, a stuck file conversion or a failed export cannot take chat down with it, because they share no process, no connection pool and no database. Each service is deployed, scaled and rolled back on its own.

18: Independently deployable services
One per service: No shared database, ever
HTTP + events: Synchronous calls plus an asynchronous event bus
One entry point: A single reverse proxy in front of everything

The services

Not every service is interesting from the outside. These are the ones a single request tends to touch, in roughly the order it touches them.

AuthenticationPostgreSQL: Accounts, sessions, token issuing and refresh rotation, roles and permission sets.
ChatPostgreSQL: Conversations and messages, context assembly, streaming execution and multi-model orchestration.
RoutingPostgreSQL: Classifies each message, applies the active mode and any policies, and records the decision with its fallback chain.
ConnectorsPostgreSQL: Provider credentials, model catalogues, capability flags and continuous health checks.
Memory and contextPostgreSQL + pgvector: Memory records, the suggestion queue, context packs and their versions, and semantic retrieval.
FilesPostgreSQL: Uploads, security scanning, text extraction, OCR, chunking and retention.
ResearchPostgreSQL: Web search, page fetching and evidence assembly for answers with sources.
WorkspacePostgreSQL: OAuth connections to twelve third-party tools, webhooks, scheduled sync and cross-tool search.
Image generationPostgreSQL: Image requests, provider adapters and per-step progress while an image renders.
Document generationPostgreSQL: Rendering answers into PDF, DOCX, CSV, HTML, Markdown, TXT and JSON.
BillingPostgreSQL: Plans, subscriptions, versioned prices, invoices and allowance enforcement.
AuditMongoDB: The immutable record of who did what, plus the usage ledger every allowance figure is derived from.
LoggingMongoDB: Structured logs from every service and from the browser, retained on a rolling window.

Each service names the database it owns. Nothing else connects to it — a service that needs another’s data asks for it over HTTP or reacts to an event.

One database per service

Every service owns exactly one database and is the only thing that connects to it. There is no shared schema, no cross-service join, and no reporting database quietly reading everyone’s tables.

The cost is that some questions take two calls instead of one join. The benefit is that a schema change in billing cannot break chat, and a runaway query in logging cannot exhaust the connection pool your conversation depends on.

No cross-database queries: A service reads its own tables and nobody else’s. Data from elsewhere arrives over an API call or an event.
Schemas are private: A service can change its own tables without coordinating with anyone, because no external consumer depends on their shape.
Migrations run per service: Each service migrates its own database on start-up. There is no global migration everyone has to wait for.
Failures stay local: A database that is slow or unavailable degrades one capability rather than the whole product.

The life of one request

What happens between pressing send and reading the answer.

The request is authenticated
the reverse proxy passes it to chat, which verifies your access token and the permissions attached to your role.
Allowance is checked
billing confirms your plan permits the model you asked for and that your daily and monthly allowance still has room.
The message is stored and announced
chat writes the message to its own database and publishes an event. Routing is listening for it.
Routing chooses a model
the message is classified, the active mode and policies are applied, connector health is consulted, and a decision with a fallback chain is recorded.
Context is assembled
chat asks memory for relevant records and pack items, and files for relevant chunks, then merges them into the prompt within a token budget.
The model is called and streamed
chat calls the provider and forwards stages, text, reasoning and metrics to your browser over Server-Sent Events as they arrive.
The result is persisted
the answer, its routing decision, its token counts and its context receipt are written down together.
Usage and audit are recorded
billing meters the cost against your allowance and audit records the event. Memory extraction runs here too.

Steps 1 to 7 are synchronous — you wait for them. Step 8 and memory extraction happen after the answer is already on your screen, so they add no latency to what you experience.

The event bus

Work that does not need to finish before you see an answer is published as an event on a RabbitMQ topic exchange and handled by whichever services care about it. Usage metering, audit records and memory extraction all run this way.

Events are the reason chat does not need to know that audit exists. Chat states what happened; anything that needs to react subscribes. Adding a consumer requires no change to the publisher.

Topic exchange: Publishers name what happened, not who should hear it. Consumers bind to the patterns they care about.
Retries with backoff: A failed handler is retried three times with increasing delay, which absorbs the transient failures that make up most of them.
Dead-letter queue: A message that still fails after its retries goes to a dead-letter queue instead of being dropped or blocking everything behind it.
Idempotent handlers: Handlers tolerate the same event arriving twice, because at-least-once delivery guarantees that eventually one will.
Everything is auditable: Audit subscribes to every domain event, so the record of what happened does not depend on each service remembering to write one.

Streaming

Answers arrive over Server-Sent Events, not polling. The connection opens when you send a message and carries everything until the answer is finished or cancelled.

Buffering is disabled end to end — at the proxy and in every service on the path — because a stream that is buffered is just a slow response with extra steps.

The same channel carries text generation, image generation progress and long-running research, so the interface has one way to show work in flight rather than three.

Stage changes: Queued, routing, assembling context, calling the model, finishing — so a pause always has a visible reason.
Content deltas: The answer text as the model produces it, token by token.
Reasoning deltas: For models that expose their thinking, the reasoning stream is delivered separately from the answer and rendered separately too.
Metrics ticks: Tokens so far, tokens per second, time to first token, and where the time actually went — model load, prompt evaluation or generation.
Terminal events: A final usage summary, or an explicit error or cancellation. The stream never just stops.

What stores what

Four kinds of store, each used for what it is good at.

PostgreSQL: The system of record for accounts, conversations, routing decisions, memory, files, connections and billing — one database per service.
pgvector: Vector similarity search inside PostgreSQL, used to retrieve relevant memories and context pack items without running a separate vector database.
MongoDB: Audit events, the usage ledger and structured logs: high-volume append-only data with a rolling retention window.
Redis: Caching, rate-limit counters and short-lived coordination state. Nothing that matters is stored only here.

Security mechanisms

Concrete controls that exist in the product. No certification claims — see the note at the end.

Access and refresh tokens: Short-lived access tokens with refresh rotation. Reuse of a rotated token invalidates the session.
Password hashing: Argon2 with per-user salts. Passwords are never stored or logged in any recoverable form.
Role-based access control: Roles carry explicit permission sets, enforced by guards on every endpoint of every service — not only in the interface.
Encrypted credentials: Provider and connector secrets are encrypted at rest with AES-256-GCM and are never returned by an API.
Schema validation: Every request body is validated against a Zod schema with explicit length and size bounds before it reaches any logic.
Rate limiting: Per-account throttling on every service, so one client cannot exhaust capacity for everyone else.
Security headers: A strict content security policy with per-request nonces, HSTS, and the standard hardening headers on every response.
TLS everywhere: TLS from the browser to the edge and again on every internal hop, with certificate verification between services.
Log redaction: Tokens, passwords, API keys and authorization headers are stripped before anything is written to a log.

ClawAI holds no third-party security certifications today — not SOC 2, not ISO 27001, not HIPAA. We describe the mechanisms above rather than implying assurances we have not obtained. Organisations with formal requirements should raise them with us so they can be scoped as part of an engagement.

Observability

Every service emits the same signals in the same shape, and they all land somewhere queryable. An operator answering “what happened to this request” should not have to open eighteen log files.

Structured logs: JSON logs from every service and from the browser, shipped over the event bus and retained on a rolling window.
Request correlation: A request ID is generated in the browser and carried through every service hop, so one identifier reconstructs the whole path.
Audit events: A separate, immutable record of security- and business-relevant actions, kept apart from operational logs.
Usage ledger: Every metered call is written as a ledger entry. Allowance figures are derived from the ledger, not from a counter that could drift.
Health aggregation: A dedicated service polls every other service and reports one consolidated view of what is up.

When something fails

Model providers have outages, rate limits and slow days. The platform is built to absorb them rather than pass them on to you as a spinner that never resolves.

Fallback chains: Every routing decision carries an ordered list of alternatives. If the chosen model fails, the next is tried automatically and the substitution is recorded.
Health-based avoidance: Connectors are health-checked continuously, and a provider that is failing is skipped by the router until it recovers.
Safe retries: Event handlers tolerate duplicate delivery, so a retry corrects a failure instead of charging your allowance twice.
Errors are visible: When every option fails, an explicit error is written into the conversation and pushed down the stream. There is no silent failure that leaves the interface waiting.
Blast radius: Separate processes and separate databases mean a failure degrades one capability. Image generation being down does not stop you chatting.

The same architecture, inside your network

Everything on this page describes the hosted service, but the architecture is not tied to it. The same eighteen services, the same event bus and the same data model can be deployed inside an organisation’s own network.

In that configuration the external model providers are replaced by open-weight models served on your own GPUs, so no prompt, document or conversation leaves your infrastructure. It is a scoped engagement rather than a plan you can buy — we size it with your team.

Talk to us about a private deployment

See it running

The fastest way to judge an architecture is to use what it produces. The free plan takes a minute to set up.

Create your account Talk to us about a private deployment