Audit log retention and the system stream (CLI)

Every tenant has its own Audit Log, which its administrators read in the application. The deployment also keeps a system stream: platform-operator actions taken with the admin key, and sign-in failures that could not be attributed to a tenant (unknown email addresses, rejected API keys). The system stream, and the retention of every stream, are the operator's, administered from the CLI with ADMIN_API_KEY.

This guide assumes the CLI is installed and pointed at your deployment; if not, start with System Administration (CLI).

Read the system stream

da admin audit query                        # last 30 days, newest first
da admin audit query --type auth. --outcome failed --from 2026-09-01T00:00:00Z
da admin audit query --tenant <tenant_id>   # one tenant's stream instead
da admin audit query --json                 # JSON Lines, one record per line
da admin audit tail                         # follow new records

Filters: --type (an event type or a category prefix such as iam.), --actor (email), --actor-id, --target-type, --target-id, --project, --outcome, --source, --ip, --from, --to. Every query is itself recorded in the stream it read.

Export

da admin audit export --format json -o system-audit.jsonl
da admin audit export --tenant <tenant_id> --format csv -o tenant-audit.csv

JSON Lines is the stored record, verbatim; CSV flattens the same fields. The export is recorded.

Retention

da admin audit retention                              # show
da admin audit retention --days 730                   # default for every tenant stream
da admin audit retention --tenant <tenant_id> --days 2555   # one tenant's override
da admin audit retention --tenant <tenant_id> --clear       # back to the default
da admin audit retention --stream system --days 400   # the system stream
da admin audit retention --unlimited                  # keep the system stream forever

Tenant streams are kept between 90 days and 7 years (2555 days); the default is 400 days. A tenant administrator sees the value and cannot change it. The system stream may also be kept forever (0), in which case purge is the only way to trim it.

Lowering retention never removes existing records early. Each stream carries a policy history, and a day's records age out under the retention that applied when they were written. A change to retention is itself recorded.

The daily sweep removes whole expired days; nothing removes an individual record.

Purge the system stream by hand

da admin audit purge --older-than 365

Removes system-stream days older than the cut-off. Tenant streams are never touched by this command. The purge is written to the stream, with the cut-off and the number of days removed, before anything is deleted.

Deleting a tenant

A tenant's audit log is deleted with the tenant, so da admin tenant delete first saves the whole stream as JSON Lines (audit-<tenant_id>-<date>.jsonl in the current directory, or --audit-export PATH) and prints the path. Keep that file: it is the customer's record of who did what in their tenant, and the only copy once the tenant is gone. --skip-audit-export is accepted on SaaS and refused by self-hosted deployments, where DELETE /admin/tenants/{id} itself requires audit_log_exported=true.

The deletion is written to the system stream as config.tenant.deleted, with the number of audit days and records that went with the tenant.

Clear a sign-in block

da admin audit unblock --email alice@customer.com
da admin audit unblock --ip 203.0.113.7

Sign-in is throttled per account (delay after 5 failures in 15 minutes, block for 15 minutes after 20) and per source address (delay after 20, block after 100). Blocks expire on their own; clearing one early is recorded as auth.login.unblocked.

Forward the system stream to your SIEM

A tenant forwards its own stream from the Security page. The system stream (every da admin and admin-key action, and the platform-admin actions mirrored into tenant streams with identity withheld) ships the same way, configured by the operator:

da admin config set security_log.forwarding_url https://siem.example.com/hooks/docai-system
da admin config set security_log.forwarding_secret <hmac-key>
da admin audit forward test
da admin audit forward status

The secret is stored encrypted and never shown again; the batches carry X-DocAI-Signature: sha256=<hmac-sha256 of the body> when it is set. The wire format is the tenant one (security_log.batch, schema_version 1) with "stream": "system" and "tenant_id": null, so one receiver parses both. Delivery starts from the moment the URL is set (history is the export's job), runs every forwarding round, backs off on a failing receiver and resumes from where it stopped; security_log.forwarding_enabled false pauses it without losing the URL. Setting any of the three is recorded in the system stream as config.security_log_forwarding.updated, with the secret shown as ***, and that record is the first thing the receiver gets.

Health

da admin audit health

Shows records written and write failures in this process, the last retention sweep, and the system-stream forwarding state (where it goes, whether batches are signed, records delivered, the last error). Audit writes never block a user's request: a failure is logged, counted here, and surfaced at ERROR in the application log. A non-zero failure count means the storage account is refusing writes and needs attention; once it reaches 5 the operator address is emailed, at most once an hour.

Volume

Two record types are aggregated so that a busy tenant cannot flood its own stream: data.document.viewed is one record per person, transaction and hour, and data.api.read is one record per API key per hour with counts (per replica, when several serve the same key). Everything else, including every download and export, is one record per action.

Deployment settings

Variable	Purpose
SECURITY_LOG_RETENTION_DAYS	Initial default retention for tenant streams (400). The CLI setting above overrides it.
SECURITY_LOG_SYSTEM_RETENTION_DAYS	Initial retention for the system stream (400; 0 keeps forever).
SECURITY_LOG_NOTIFICATIONS_ENABLED	Email the operator address about audit retention changes and audit write failures (on). Needs SMTP. Each tenant's own notifications are the tenant's setting on its Security page and are not governed by this. Also a Security setting in da admin config.
SECURITY_LOG_OPERATOR_EMAIL	Where audit retention changes and audit write failures (5 or more in a process) are sent. Empty sends the operator nothing.
SECURITY_LOG_FORWARDING_URL, SECURITY_LOG_FORWARDING_SECRET, SECURITY_LOG_FORWARDING_ENABLED	Initial values for system-stream forwarding (see above); the da admin config settings override them.
TRUSTED_PROXY_CIDRS	Networks whose X-Forwarded-For header is believed when recording the client address. Empty: the private ranges, which covers Azure Container Apps ingress, a Docker bridge and a reverse proxy on the host. Set it to the exact ingress range where you can, and to none if clients reach the application directly.
AUDIT_STORAGE_CONTAINER (Azure) / AUDIT_STORAGE_BUCKET (S3)	Optional second container or bucket, in the same account, that holds the audit log alone. Off by default.
SECURITY_LOG_FORWARD_INTERVAL	Seconds between forwarding rounds (60).

Clocks

Timestamps are UTC from the host clock. Azure Container Apps and managed Kubernetes keep time for you; a Docker host on your own hardware must run NTP (timedatectl status should report System clock synchronized: yes), or the records cannot be correlated with anything else.

Immutable storage

The audit log lives under a dedicated prefix (tenants/<tenant>/security/ and system/security/) and nothing else is written there, so a storage-level immutability policy can be applied to it without freezing the rest of the tenant's data. On Azure such policies are container-scoped, so the locked configuration is a separate container:

1. Create a second container in the same storage account, for example audit, and apply a time-based immutability policy to it (Azure: Access policy on the container; AWS: enable Object Lock on the bucket at creation).
2. Set AUDIT_STORAGE_CONTAINER=audit (Azure) or AUDIT_STORAGE_BUCKET=<bucket> (S3) and restart the application. New records are written there; the retention sweep still removes whole expired days once the policy allows it.

Existing records are not moved automatically. If a customer requires the history in the locked container, copy the security/ prefixes across once with the platform's own tooling before switching.

Client address behind an appliance

If your deployment sits behind your own appliance or firewall (a forced-tunnel network virtual appliance, a corporate proxy), confirm after go-live that a record's channel.ip is the user's address and not the appliance's. If every record shows the appliance, the appliance is not passing X-Forwarded-For and its address range needs to be in TRUSTED_PROXY_CIDRS, or the appliance needs to be configured to append the header.