Resident state
RushDB keeps a few structures in memory across requests because rebuilding
them on every request is too slow: owner sets of indexed filters,
relationship graphs, the label catalog, the property dictionary, planner
statistics, the rows of vector spaces and the tenant directory. Each one is a
layer of one framework (rushdb-storage/src/resident/), and storage is
always the reference: a layer can make an answer faster, never different.
The rule
Every layer follows one version rule.
- Each layer entry reflects a set of storage areas, its deps: one declared index, one property, one label, the tenant's label catalog, its property definitions, its adjacency, one embedding space, its security rows, or the tenant directory.
- Every commit raises the version of each area it writes to its commit index before the commit becomes visible. This happens in the one place every write passes through: local commit groups, commits a follower applies from the leader, WAL replay at startup and a restore into a running store. The mapping from a written key to the areas it changes is one exhaustive match over the key families, so a new key family cannot be added without deciding what it changes.
- An entry records the version
Vof its deps when it was built (or computed by a reader and kept), at a snapshot that includesV. - A reader at snapshot
ruses the entry only if the deps' version is stillVandr ≥ V. Anything else is a miss, and the reader reads storage.
So an entry survives commits that do not touch it (other tenants, other properties, other relationship types' tenants), and any write to one of its deps makes it a miss at once, also for readers pinned to older snapshots.
Versions live in a fixed table of 2^20 slots (8 MiB), indexed by a hash of the area. Two areas that share a slot can only make an entry look newer than it is, which costs a rebuild, never a wrong answer. The table does not grow with the number of tenants or properties.
Entries are immutable
An entry is made from one pinned snapshot and shared read-only. Nothing patches it in place: no deltas, no partial refresh. When its deps change it is dropped (on the next lookup, or by the layer's LRU) and a new one is built from a later snapshot.
Building
A layer is filled in one of two ways:
- Offered: a request that read storage anyway hands its result to the layer (owner sets, catalogs, planner statistics, the tenant directory).
- Rent-or-buy: a request that read storage reports how many rows it read toward a build. Once the rows read since the deps last changed reach the cost of one build, the entry is built in the background at a fresh view, on a helper thread of the shared query thread budget; the paying request keeps its storage answer (relationship graphs, vector rows). A build that would not fit its budget stops early and is not retried until a dep changes.
A build that raced a write to one of its deps is never kept.
Budgets
Each layer has its own byte budget and LRU. A query is never refused because of a layer: an entry that does not fit is simply not kept.
Layer (layer label) | Holds | Deps | Budget | Default |
|---|---|---|---|---|
owner-sets | Owners of an indexed filter leaf | the leaf's indexes | owner-set-cache-bytes (child of the logical cache) | 64 MiB |
graph | Both directions of one relationship type | the tenant's adjacency | graph-cache-bytes | 128 MiB |
label-catalog | Labels and per-label counts | label catalog, security rows | share of logical-cache-bytes | 128 MiB |
property-dictionary | Property name → type | property definitions, security rows | share of logical-cache-bytes | 128 MiB |
describe | One unfiltered describe.properties answer (also the property part of describe.schema), before policy masking | property definitions; per described label its members, its label-scoped property rows and the properties it holds; the label catalog when no label is named | share of logical-cache-bytes | 128 MiB |
planner-stats | One value distribution | the property's values | planner-stats-bytes (child of the logical cache) | 32 MiB |
vector-arena | One space's ready bindings and their vectors as f32, with each vector's norm | the space | share of vector-cache-bytes | 256 MiB |
tenant-directory | Scope → active tenant ordinal | the tenant directory | share of logical-cache-bytes | 128 MiB |
column | One stored property of one label: each member's value | the label's members, the property's values, the tenant's definitions | column-cache-bytes | 256 MiB |
Resident typed columns
A column holds, for every member of one label at one snapshot, the value of one stored scalar property as a typed cell (absent, null, boolean, integer or datetime milliseconds, float, or string by dictionary code): about 17 bytes per member plus the distinct strings. Values are copies of the stored values, decoded the way point reads decode them, so an aggregate folds the same values in the same order either way. There is no column for a property stored as an array anywhere, for system fields, or for a label marked suspect by self-heal.
Grouped aggregates over one label, with no traversal, use columns when every property they read has one:
- Rows. Each member's cells come from the columns instead of point reads; groups are found by their key cells' codes, and a walk takes its candidates from the column's members instead of the membership rows.
- The filter. A filter on the label's stored properties is evaluated once per distinct combination of its properties' cells, by the same evaluator every read uses, and the matching members become the candidates. Above 4,096 distinct combinations (a filter over many distinct values) the planner answers it as before.
Columns are built in the background once the reads they would replace pay
for one build (about one read per member): point reads of the rows, and
the planner's candidates for a filter. Until then, and whenever a column is
not resident, the aggregate reads storage. A pass that read columns and
then exceeds query-memory-max is run again the way storage runs it, so
columns never refuse an aggregate that storage answers.
Off is the reference
resident-caches: off (or RUSHDB_RESIDENT_CACHES=off) keeps no entry in
any layer: storage answers every request. A budget of 0 does the same for
one layer. Results are the same bytes either way; only latency changes. The
test suite runs twice, with layers on and off, and differential tests
compare the two at the same snapshot.
resident-verify-rate (default 0) rebuilds a sampled share of hits from
storage in the background and compares them. A difference increments
rushdb_resident_verify_mismatch_total{layer}, drops the entry and logs the
layer at warn; the answer already returned is not changed. Only one
verification runs at a time.
Metrics
Every layer reports the same counters, labelled layer:
rushdb_resident_bytes, rushdb_resident_entries,
rushdb_resident_hits_total, rushdb_resident_misses_total,
rushdb_resident_builds_total, rushdb_resident_build_refused_total,
rushdb_resident_build_seconds, rushdb_resident_invalidations_total,
rushdb_resident_evictions_total, rushdb_resident_verify_total and
rushdb_resident_verify_mismatch_total. See the
metrics reference.
What is not resident state
These keep memory too, but they are not a cache of a storage answer, so they are outside the framework, each with its own contract:
- ANN generations (vector internals): an
approximate index by contract, patched in place as bindings arrive. With
vector-cache-bytes: 0searches answer exactly instead, which is a different (exact) answer, so "off = same bytes" cannot hold for them. - Text indexes (text internals): persisted derived
indexes with their own watermark and
eventual/immediateconsistency. Their files are memory-mapped (page cache), and the writer heap is not charged to a RushDB budget. - Derived rows in storage (value dictionary, counters, presence, index entries): they are storage, kept exact in every commit (derived state).
- The redb page cache (
storage-cache): below the reference. - Request-scoped state (reconstruction caches, per-request family and label tables, cycle graphs): lives inside one pinned view and ends with the request.
- Observability snapshots (per-tenant record counts, storage
statistics): refreshed on a timer and read only by
/metrics.