Skip to content

Storage layer

The agent uses three persistent storage services, each picked for a specific shape of data.

Turso (libsql)

Stores: the relational core — documents, document_fields, chunks, reconciliations, agent_checkpoints, tax_rules, enablebanking_sessions, dashboard tokens. ~12 migrations, versioned in agent/src/accounting_agent/store/schema.py.

Why Turso:

  • libsql = SQLite-compatible wire protocol → standard sqlite3 client works for dev (just point at a local file)
  • Hosted (no self-managed Postgres)
  • Embedded vector type via vector32(...) for chunk embeddings
  • Cheap at our scale (one operator, few thousand transactions/year)

If Turso is down: the bot still answers Discord but can't write. The EB poller can't insert. The booker can't read pending rows. Discord interactions surface clean errors; nothing silently fails. Recovery on Turso reconnect is automatic — no state needs replay.

Backups: Turso handles them. We rely on it.

Cloudflare R2

Stores: raw bytes of every ingested document (PDFs, images) under content-addressed keys. Also the OCR markdown when the ingest pipeline produces it as a sidecar.

Why R2:

  • S3-compatible (the agent's store/blob.py uses boto3)
  • Zero egress fees inside Cloudflare's network
  • Versioning enabled so accidental deletes are recoverable
  • Cheap storage (~$0.015/GB/month)

Key shape: <S3_KEY_PREFIX><kind>/<YYYY-MM>/<sha><ext> — e.g. prod/invoice/2026-06/abc...def.pdf. The prefix lets one bucket serve multiple envs (test/ vs prod/).

If R2 is down: ingest fails on the upload step; the bot surfaces a Discord error to the operator. Booker fails on the PDF-attach step (can't fetch raw bytes); the row stays at booked for the next tick.

Backups: R2 versioning + Cloudflare's own replication. Quarterly restore drills planned (not implemented yet).

FalkorDB

Stores: the tax knowledge graph. Triples extracted from uploaded tax-rule PDFs. Predicates are dynamic (HAS_THRESHOLD, APPLIES_TO, REQUIRES_VAT_REGISTRATION, ...). Connected to tax_rules rows in Turso via source_document_id references.

Why a graph DB:

  • Tax rules have free-form relationships SQL can't express ("small_business_scheme excludes import_vat" / "OSS applies if customer_country ≠ supplier_country AND customer_type = consumer")
  • The reasoner queries by predicate + entity, not by row id
  • FalkorDB runs on the shared prod-falkordb container — no extra ops cost

If FalkorDB is down: the vat_decision reasoner falls back to the SQL tax_rules table only. Confidence may be lower but the booker still functions. We did this fallback explicitly so the graph isn't a hard dependency.

Backups: the graph is derived from the source tax PDFs in R2. If FalkorDB is wiped, re-run the rule-extraction pipeline on every kind=tax-rule document and the graph rebuilds. /resync_graph does this on demand.

What's NOT a store

Worth calling out explicitly:

  • No Postgres in the agent's container. We use the shared prod-postgres only if a future product needs it (currently not). Turso is the operator's view of the world.
  • No Redis. The agent has no caching layer beyond in-process dicts (e.g. the booker's purchase-articles cache, scoped to a single run).
  • No Memgraph, no Neo4j. FalkorDB does the graph job at a fraction of the operational cost. ADR-039 covers the choice.

What state lives in memory

Almost none, by design.

  • The bot keeps the Discord gateway connection open (recoverable in seconds on restart).
  • The booker keeps a per-run purchase_articles_cache to avoid hammering the e-Fin list endpoint (rebuilt on next run).
  • The EB poller's last_polled_at lives in the enablebanking_sessions table, NOT memory.

Restart any container at any time — no state is lost that wasn't already in one of the three stores.