Skip to main content

Compliance and Retention Patterns

Data that enters your system eventually needs to leave it — or have its sensitive fields erased. Compliance frameworks like GDPR, CCPA, and HIPAA impose three distinct obligations:

  1. Expiration — records older than a retention window must be deleted
  2. Archival — records must be moved to cold storage before deletion to satisfy audit requirements
  3. Redaction — PII fields must be blanked on request (right to erasure) without deleting the record or breaking the graph

Each obligation maps to a concrete RushDB pattern. This tutorial covers all three.

Graph shape​

LabelWhat it represents
USERRecord that may hold PII subject to erasure
ORDERBusiness record with a retention window
RETENTION_POLICYShared node encoding retention rules (window, action)
EVENTImmutable audit log entry

Step 1: Tag records with retention metadata at creation​

Stamp every record with expiresAt and retentionPolicy at write time. This avoids a later migration and enables index-driven expiry queries.

import os
from datetime import datetime, timezone, timedelta
from rushdb import RushDB

db = RushDB(os.environ["RUSHDB_API_KEY"], base_url="https://api.rushdb.com/api/v1")

RETENTION_DAYS = 365 * 7

def expiry_date(days: int) -> str:
    return (datetime.now(timezone.utc) + timedelta(days=days)).isoformat()

order = db.records.create("ORDER", {
    "customerId": "cust-42",
    "amount": 499.99,
    "currency": "USD",
    "status": "completed",
    "createdAt": datetime.now(timezone.utc).isoformat(),
    "expiresAt": expiry_date(RETENTION_DAYS),
    "retentionPolicy": "financial-7yr",
    "archived": False,
    "redacted": False
})

Step 2: Find and archive records approaching expiry​

Run this query on a schedule (daily cron or queue worker) to find records whose expiresAt is within the next 30 days and mark them as archived before deletion.

def days_from_now(days: int) -> str:
    return (datetime.now(timezone.utc) + timedelta(days=days)).isoformat()


def archive_expiring_records(label: str) -> int:
    expiring = db.records.find({
        "labels": [label],
        "where": {
            "expiresAt": {"$lte": days_from_now(30)},
            "archived": False
        },
        "limit": 100
    })

    for record in expiring.data:
        db.records.update(record.id, {
            "archived": True,
            "archivedAt": datetime.now(timezone.utc).isoformat()
        })
        print(f"Archived {record.id} (expires {record.data.get('expiresAt')})")

    return len(expiring.data)


archive_expiring_records("ORDER")

Step 3: Delete expired records​

After the archival window has passed (e.g., 30 days after archivedAt), hard-delete the records. Query for archived records whose archivedAt is past the grace period.

def delete_expired_records(label: str, grace_days: int = 30) -> int:
    cutoff = (datetime.now(timezone.utc) - timedelta(days=grace_days)).isoformat()

    expired = db.records.find({
        "labels": [label],
        "where": {
            "archived": True,
            "archivedAt": {"$lte": cutoff}
        },
        "limit": 100
    })

    if not expired.data:
        print("Nothing to delete.")
        return 0

    ids = [r.id for r in expired.data]
    db.records.delete({"ids": ids})
    print(f"Deleted {len(ids)} expired {label} records")
    return len(ids)


delete_expired_records("ORDER")

Step 4: Redact PII fields (right to erasure)​

GDPR Article 17 and CCPA require erasing personal data on request. The requirement is to erase the data, not necessarily the record — the record may need to stay for accounting or relationship integrity. Replace PII fields with null and stamp redactedAt.

PII_FIELDS_BY_LABEL = {
    "USER": ["email", "phone", "dateOfBirth", "ipAddress", "fullName"],
    "ORDER": ["shippingAddress", "billingAddress"]
}


def redact_user(user_id: str):
    tx = db.transactions.begin()
    try:
        user_nulls = {f: None for f in PII_FIELDS_BY_LABEL["USER"]}
        db.records.update(user_id, {
            **user_nulls,
            "redacted": True,
            "redactedAt": datetime.now(timezone.utc).isoformat()
        }, transaction=tx)

        orders = db.records.find({
            "labels": ["ORDER"],
            "where": {
                "USER": {
                    "$relation": {"type": "SUBMITTED", "direction": "in"},
                    "__id": user_id
                }
            }
        })

        for order in orders.data:
            order_nulls = {f: None for f in PII_FIELDS_BY_LABEL["ORDER"]}
            db.records.update(order.id, order_nulls, transaction=tx)

        db.transactions.commit(tx)
        print(f"Redacted user {user_id} and {len(orders.data)} associated orders")
    except Exception:
        db.transactions.rollback(tx)
        raise


redact_user("user-42")
Relationships survive redaction

Redaction nulls field values but does not delete the record or its edges. Downstream records that traverse SUBMITTED relationships still reach the USER record — they just find nulled PII fields. This preserves graph integrity while complying with erasure obligations.

Step 5: Compliance reporting​

Use select aggregations to produce a retention health report: how many records are compliant (not yet expired), archived pending deletion, or already redacted.

report = db.records.find({
    "labels": ["ORDER"],
    "select": {
        "total":    {"$count": "*"},
        "archived": {"$sum": {"$cond": [{"$eq": ["$record.archived", True]}, 1, 0]}},
        "redacted": {"$sum": {"$cond": [{"$eq": ["$record.redacted", True]}, 1, 0]}},
        "policy":   "$record.retentionPolicy"
    },
    "groupBy": ["policy"]
})

for row in report.data:
    print(
        f"Policy: {row.data.get('policy')} | "
        f"Total: {row.data.get('total')} | "
        f"Archived: {row.data.get('archived')} | "
        f"Redacted: {row.data.get('redacted')}"
    )

Design rules​

  1. Stamp expiresAt at write time — never try to derive expiry from createdAt later; policies change, and a stored date is unambiguous
  2. Archive before deleting — a two-phase expiry (archive → grace period → delete) satisfies audit requirements and gives you a rollback window if a record is flagged in error
  3. Redact, do not delete, when relationships matter — deleting a USER record breaks every edge pointing to it; nulling PII fields preserves graph integrity
  4. Run expiry jobs idempotently — use archived: false and date filters so re-runs are safe
  5. Keep a separate immutable EVENT record for each redaction action — the redaction itself is an auditable fact; see Audit Trails
  6. Never expose redacted: false records in user-facing APIs without authorization checks — filter by redacted: false in all end-user queries

Next steps​