Operate
SOC 2 readiness
What a CLI server records for audits on its own, operator roles, the security incident register, how operators export the evidence, and the readiness kit of policies, controls and an evidence calendar.
This page is for people who run their own CLI server for others. When a customer asks for a SOC 2 report, CLI already records much of what an auditor wants (every change, operator actions, restore drills, incidents) and can export it for any period.
How to use it: operators open Operations and export the evidence for the dates an auditor asks about. The readiness kit in the repository helps you write the rest.
It's a head start, not a certificate: only an auditing firm can issue a SOC 2 report.
If you run a CLI server for other people, sooner or later a customer asks for a SOC 2 report. CLI can't give you one, but it records much of what an auditor asks for, lets operators export it for any period, and comes with a kit to get ready. This page is for operators: the people listed in CLI_OPERATORS, and others given an operator role (see Operator roles below and /docs/self-hosting). The operator routes on it need the admin role.
What CLI records on its own
Once it's configured, these are recorded without anyone doing anything:
- Changes. Every successful change in a project goes into its audit log: who, what and when (see /docs/teams-and-security). The flight recorder adds the reason, the state before and an undo (see /docs/flight-recorder).
- Operator actions. Every change an operator makes through an operator-only route: who, the method and route, and how it went (the status code). Reads aren't recorded.
- Sign-ins. Every sign-in attempt by wallet, email code, email link or GitHub, whether it worked, the principal when it did, and the IP address.
- Second-factor changes. Every authenticator app or passkey added or removed: who, what, and the IP address.
- Access. Who is an operator and with which role, who is an admin or owner, and whether each has a second factor; which projects require one (see /docs/access-reviews).
- Access reviews finished in projects and for operators.
- Backups of CLI's own database, and the restore drills you record.
- Incidents: outages of customers' services and how long they lasted (see /docs/incidents).
- Security incidents on the platform itself, from the register below.
- Deletions of whole projects and accounts (see /docs/deleting).
- Abuse flags and what was done about them.
- CLI's own schema changes, as migrations are applied.
The evidence export
Operators download everything above for a period in one bundle.
In the dashboard: on the Operations page, under Compliance evidence, pick the dates and click Download evidence. You get one JSON file. The end date includes the whole day.
From your terminal:
npm run compliance:export -- --out evidence/2026-q4 --from 2026-10-01 --to 2027-01-01It uses your cli login session (sign in as an operator), or CLI_API_URL and CLI_TOKEN. It writes evidence.json and one CSV file per section to the folder, for spreadsheets and compliance platforms. In the CSVs, times are ISO dates.
With the API:
GET /v1/admin/compliance/evidence?from=2026-10-01&to=2027-01-01from and to are dates (2026-01-31) or milliseconds. Without them, the export covers the last 90 days up to now. from must be before to. Anyone who isn't an operator gets FORBIDDEN, and so does an operator without the admin role (with details.role and details.required).
What's in it
| Section | What it holds |
|---|---|
operators | Each operator: their role, fromEnv (true for those in CLI_OPERATORS), and whether they have a second factor |
admins | Each person who is owner or admin anywhere: how many projects, how many as owner, and second-factor status |
twoFactorRequired | Projects that require a second factor, and since when |
apiKeys | Every key active in the period: project, name, role, scopes, created, age in days, expiry, last used, revoked, and flags such as "no expiry" or "unused 90+ days" |
accessReviews | Reviews finished in the period: scope, project, opened, finished, by whom, how many kept and removed |
auditLog | How many audit entries of each action were recorded |
backups | Backups of CLI's own database: id, size, when (each is encrypted, see Encrypted backups) |
restoreDrills | Restore drills: when, by whom, whether they worked, notes |
operatorActions | Operator changes: when, who, method, route, status |
signIns | Sign-in attempts: when, principal, method, whether they worked, IP |
mfaChanges | Second factors added or removed: when, principal, the change (for example "passkey added"), IP |
securityIncidents | Security incidents open at any time in the period: title, severity, status, when detected, contained and resolved, whether customers were told, the root cause, and whether there's a postmortem |
deletions | Whole projects deleted in the period ("project (in trash)" or "project (purged)") and who deleted them, and accounts deleted (by a hash of the sign-in) |
incidents | Outages overlapping the period, with their length in minutes |
encryption | How project keys, backups and uploads, the public URL, worker requests and agent credentials are protected |
abuse | Abuse flags raised in the period |
changes | CLI's own schema migrations applied in the period |
checks | The launch checks from npm run preflight |
The JSON also has generatedAt and the period it covers.
Restore drills
A backup nobody has restored is a hope, not a control. When you restore a backup somewhere safe and check it (deploy/RESTORE.md walks through it), record it.
In the dashboard: on the Operations page, under Record a restore drill, write what you restored, where, and how you checked it, tick It worked if it did, and click Record drill.
With the API:
POST /v1/admin/compliance/restore-drills
{"ok": true, "notes": "Restored last night's backup into a scratch database; row counts match."}Notes can be up to 2,000 characters. Drills appear in the export's restoreDrills section, and they're kept regardless of the retention setting below.
Audit retention
The audit log, the flight recorder, operator actions, sign-ins and second-factor changes are kept for 90 days by default. A SOC 2 Type II period is usually 6 to 12 months, and auditors sample across it, so keep them longer:
CLI_AUDIT_RETENTION_DAYS=400The smallest value allowed is 30. npm run preflight warns when it's under 400.
Operator 2FA
Set CLI_REQUIRE_OPERATOR_2FA=true so no operator-only route works for an operator without a second factor. npm run preflight warns while it's off. See Access reviews and 2FA rules.
Session idle timeout
Sessions last 24 hours. Auditors usually expect sessions to end sooner when nobody is using them, so set an idle timeout:
CLI_SESSION_IDLE_MINUTES=120A session unused for that long ends, and its next request answers UNAUTHORIZED. npm run preflight warns while it's 0 (off). People can also see their sessions and sign out everywhere else from their account page (see /docs/teams-and-security).
Operator roles
Operators don't all need to do everything. Each has one role:
| Role | Can |
|---|---|
support | The abuse queue, resolving flags, suspending and unsuspending projects, and reading the cluster status and the list of CLI's own backups |
finance | Payout reviews (approve, reject, resolve stuck ones), the CLI token view and assigning token purchases, and the traction numbers |
admin | Everything: also taking a backup, the benchmark, compliance evidence and restore drills, security incidents, operator access reviews, and the operator list itself |
Everyone in CLI_OPERATORS is an admin. They're managed only in the server's environment, so you can always get back in. Admins add other people with a role in the app, and those roles live in CLI's database. An operator route outside your role answers FORBIDDEN with details.role (yours) and details.required.
In the dashboard: the Operators panel on the Operations page lists every operator, their role, and who added them and when (or "CLI_OPERATORS"). Admins add someone by sign-in (email:[email protected], github:<id> or a 0x wallet) and role, change a role by adding the same person again, and remove anyone not in CLI_OPERATORS.
With the API:
GET /v1/admin/operators
PUT /v1/admin/operators
{"principal": "email:[email protected]", "role": "support"}
DELETE /v1/admin/operators/email%3Asam%40company.com
GET /v1/admin/meThe first three need the admin role. PUT adds someone or changes their role. Nobody can change or remove their own role (FORBIDDEN), and people in CLI_OPERATORS can't be changed here (CONFLICT: change the environment instead). GET /v1/admin/me works for any operator and answers with your principal and role.
Each change through these routes is in the evidence export's operatorActions (who, method, route and status), and the list itself records who added each person and when. Operator access reviews cover everyone with a role, and removing one in a review takes the role away at once (see /docs/access-reviews).
Security incident register
CLI's incidents (see /docs/incidents) are customers' services going down. The security incident register is for the platform itself: a leaked key, a suspicious operator sign-in, a vulnerability in CLI, a provider's breach. It needs the admin role.
In the dashboard: on the Operations page, Security incidents lists them. To open one, write what happened in a line, pick the severity (low, medium, high or critical) and say how it was found and what's affected. Each incident has a timeline you add to as you work, Mark contained, Resolve, a Customers told box, and a root cause and postmortem to write and save.
With the API:
POST /v1/admin/security-incidents
{"title": "Payout KMS permission exposed in a CI log", "severity": "high", "summary": "Found in a public CI run; key not known to be used."}
GET /v1/admin/security-incidents?open=true
GET /v1/admin/security-incidents/:id
POST /v1/admin/security-incidents/:id/notes
{"text": "Removed kms:Sign from the CI role."}
PATCH /v1/admin/security-incidents/:id
{"status": "resolved", "rootCause": "…", "postmortem": "…", "customersNotified": true}How it works:
- Opening records who opened it, and
detectedAt(now, unless you send it in milliseconds). The first timeline entry repeats the severity and summary. - Status goes
open→contained→resolved. Moving it pastopenrecords when it was contained;resolvedrecords when it was resolved. - You can't resolve it without a root cause. Send
rootCausewith the change, or save it first. Otherwise CLI answersVALIDATION. - Customers told records when you ticked it. CLI doesn't send anything itself; tell customers your usual way.
- Every change to status, severity, customer notification, root cause and postmortem adds a timeline entry with who and when.
?open=truelists those not yet resolved; without it you get them all.
Encrypted backups
Daily and manual backups of customers' databases, and the daily backups of CLI's own database, are encrypted before they leave the machine: each gets a fresh AES-256-GCM key, and that key is wrapped by the server's key provider (AWS KMS or Vault in production, otherwise CLI_SECRET_KEY). The wrapped key travels in the file's header.
So restoring one needs the file and access to the key provider. To restore CLI's own database by hand, decrypt the downloaded file with the server's environment:
npm run backup:decrypt -- ./cli.sqlite.enc ./cli.sqliteIt doesn't need the database. deploy/RESTORE.md has the whole procedure. Customers' backups taken before Drop 22, keyed straight from CLI_SECRET_KEY, still restore as before. If the KMS key or CLI_SECRET_KEY is lost, the backups can't be read, so keep the key's recovery plan with the restore procedure.
Deep health for uptime monitors
/health only says the API answers. For an outside uptime monitor, point it at the deep check instead:
GET /health/deepIt needs no sign-in, and answers 200 when the control plane is doing its background work and 503 when something needs a person:
{
"ok": true,
"checks": { "database": true, "leader": true, "platformBackup": true },
"backupAgeHours": 9,
"at": 1767225600000
}| Check | Means |
|---|---|
database | The database answers |
leader | One replica holds the leader lease, so metering, health checks, backups and other background work are running |
platformBackup | The newest backup of CLI's own database is at most 48 hours old |
The status is 503 when the database or leader check fails. platformBackup is reported but doesn't change the status, so alert on it separately if your monitor can read the body. See Monitoring and logs.
Checks on CLI's own code
The repository has two GitHub workflows that serve as change-management and vulnerability evidence:
ciruns on every pull request and every push to main: the type check, the server tests on SQLite and on Postgres (PGlite), a check that generated files (the skill file and packages) are current, the dashboard and site tests and builds, the contract tests,npm auditfor high and critical advisories in what ships, and, on pull requests, a dependency review.codeqlruns GitHub's CodeQL static analysis with the extended security queries on pull requests, pushes to main and weekly.
Make branch protection require ci to pass before merging; the workflow can't turn that on for you.
The readiness kit
The compliance/ folder in the repository holds:
| File | What it is |
|---|---|
README.md | How to go from readiness to a Type I and then a Type II report, what CLI covers and what you must do yourself |
controls.md | A control matrix: each 2017 Trust Services Criterion for Security, Availability and Confidentiality, mapped to how CLI implements it, where the evidence comes from, and who owns it (platform, company or shared) |
evidence-calendar.md | What to collect daily, weekly, monthly, quarterly, yearly and per event, and which command, endpoint or page produces it |
policies/ | 13 policy templates written for how CLI works: access control, change management, incident response, encryption and key management, vendor management, AI agent governance and more |
Replace the placeholders ({{COMPANY}}, {{SECURITY_OWNER}} and the rest), delete anything you don't actually do, and have the policies approved. An auditor tests what a policy says, so a policy that promises more than you do produces findings.
What your company still has to do
CLI covers the platform's side. These are yours, and nothing in CLI does them:
- Governance: a named security owner, an org chart, and a yearly review of security by the board or an advisor.
- People: background checks, signed policies, onboarding and offboarding (including removing leavers' operator roles and taking them out of
CLI_OPERATORS), and security training every year. - Devices: disk encryption, screen lock and updates on every laptop that can reach production.
- Risk and vendors: a yearly risk assessment, and reviews of each subprocessor's own reports and contracts.
- Infrastructure outside CLI: your cloud accounts, GitHub organisation, DNS and payment providers, and penetration tests.
- Running incident response: CLI gives you the register, but deciding what's an incident, containing it, telling customers and regulators, and writing the postmortem are people's work.
- An auditor and the observation period: pick a CPA firm, run the controls for the period, and keep the evidence.
Before launch, check all of this on staging first: see Staging and launch checks.