LumiBaseDocs

Agent Harness Layer

LumiBase is moving beyond a human-only headless CMS toward a control plane where AI agents can work with people on business data, schema, workflows, and reusable artifacts.

The Agent Harness Layer wraps every agent action in explicit governance: a goal, a scoped run, a tool contract, permission and risk checks, audit records, evaluation gates, and human approval when the action is risky.

Product definition

The harness is the operating layer that keeps agents from running as unbounded prompt executors. An agent receives a goal, builds context under tenant and permission boundaries, calls registered tools, emits reviewable artifacts, and records enough audit data to retry or investigate the run later.

In short: LumiBase is an AI-native backend operating system where agents understand business state through schema and content, operate under governance, and return outputs as versioned artifacts instead of opaque chat text.

User-facing surfaces

Studio

The Studio Agent Harness page exposes:

  • Runs — recent goal/run execution status, run metadata, stop reasons, and budget usage.
  • Tools — enabled tool registry entries, required capabilities, owner, risk policy, and rate-limit metadata.
  • Approvals — generalized approval queue for plans, tool calls, artifacts, and schema diffs.
  • Artifacts — reviewable outputs such as schema diffs, page specs, component specs, seed data, API specs, prompts, and migrations.
  • Memory — scoped context entries with provenance, confidence, expiry, and redaction before use.
  • Generate App — an MVP template that produces app artifacts from existing schema/content instead of mutating production state directly.

API and SDK

The CMS exposes the first-class agent control surface under /api/v1/agent/*:

  • goals and runs for lifecycle tracking and retry.
  • tools for registry discovery.
  • approvals for generalized HITL decisions.
  • artifacts and evaluations for review/publish gates.
  • memory for scoped context assembly.
  • generate-app for generation templates that create artifacts.

The JavaScript SDK mirrors these APIs with typed helpers for list/create/read/decide/evaluate/publish operations.

Existing /api/v1/ai/chat and /api/v1/ai/approvals remain backward-compatible. Dangerous Copilot actions still create legacy ai_approvals rows, and the harness writes linked generalized approval records where applicable.

System collections

The harness adds additive system tables. All records are tenant-scoped with siteId and are designed for auditability rather than hidden side effects.

CollectionPurposeGovernance notes
agent_goalsBusiness goals created by users, workflows, or agent templatesOwner, priority, deadline, status
agent_runsA single execution attempt for a goalAgent name, model/provider metadata, budget, status, timing, metrics
agent_plansPlanned steps before tool executionCan require approval before execute
agent_toolsRegistry of callable skills/tools/extensionsCapability requirements, input/output schema, risk policy, enabled state
agent_permissionsAgent/role/policy capability grantsPrompt text cannot grant new capabilities
agent_tool_callsAudit log for every tool invocationInput/output/error, latency, status, denial reason, masked secrets
agent_approvalsGeneralized approvals for plans, tool calls, artifacts, and schema diffsDecision, approver, policy, expiry
agent_artifactsVersioned outputs created by agentsReviewable and publishable, hash tracked
agent_evaluationsEvaluation results for artifactsJSON schema validation, permission diff lint, API spec validation, prompt safety
agent_memoryLong-lived context outside conversation historyScope, provenance, confidence, expiry, optional embedding

Execution contract

Every harnessed action should resolve to an execution envelope:

json
{
  "goalId": "goal_...",
  "runId": "run_...",
  "agentName": "lumibase-copilot",
  "context": {
    "siteId": "site_...",
    "collections": ["products", "orders"],
    "policySnapshotHash": "sha256:..."
  },
  "budget": {
    "maxToolCalls": 20,
    "maxCostUsd": 2,
    "timeoutMs": 30000,
    "maxArtifactBytes": 524288
  },
  "risk": "safe | review_required | dangerous",
  "approvalPolicy": "none | before_execute | before_commit",
  "artifacts": []
}

Rules enforced by the harness:

  • A tool must be registered and enabled before it can execute.
  • The caller must satisfy required capabilities for the tool.
  • Risk policy can deny, execute directly, or require human approval.
  • Disabled tools and capability/risk denials are logged with reasons.
  • Risky artifacts cannot be published or submitted for approval unless evaluation passes or an explicit override reason is supplied.
  • Failed runs keep their audit history. Retries create a new run linked to the original run instead of rewriting history.
  • Budget limits stop execution for max tool calls, runtime, estimated cost, or artifact size.

Run lifecycle and async execution

Run status follows queued → running → awaiting_approval | waiting_user → succeeded | failed | cancelled (waiting_user is a run parked on an askUser question — see Asking a human):

  • POST /api/v1/agent/goals with execution: 'async' and a task: { skillName, arguments } creates the goal plus a queued run, enqueues it on the agent-runs queue via the runtime QueueProvider, and returns 202 with the runId immediately. Runtimes without a queue adapter reject async execution with ASYNC_UNAVAILABLE; sync execution is unaffected.
  • The queue worker (registerAgentRunWorker, wired in the Node entrypoint) drives queued runs through the same harness codepath — capability checks, risk policy, budgets and audit apply identically. Capabilities are captured from the enqueuing session and never widened.
  • When a dangerous action creates an approval, the run parks as awaiting_approval. An approval decision resumes it and executes only the stored skill — completed tool calls are never re-run.
  • POST /api/v1/agent/runs/:id/cancel cancels queued/running/awaiting_approval runs. Cancellation takes effect at the next tool-call boundary (the harness re-checks before every tool call), wins over late approvals, and is recorded with stopReason in run metrics.
  • POST /api/v1/agent/runs/:id/retry re-executes a failed or cancelled run as a new queued run whose retryOfRunId points at the original, and enqueues it on agent-runs — the same worker claim, harness, approvals and kill switch as any async run. The task is recovered from the run's recorded tool call and the governance envelope (role, intent, autonomy cap, budget) from its goal; the retry executes under the requester's re-resolved rights, never the original requester's. Only the goal's latest attempt can be retried, and only while nothing on the goal is in flight, so one request yields at most one new execution: concurrent retries serialize on the goal row and the loser gets 409. Responses: 201 with { goalId, runId, agentName, status: 'queued', retryOfRunId }; 404 for an unknown or other-tenant run; 409 for RUN_NOT_RETRYABLE, RUN_ACTIVE, RUN_EFFECT_RECORDED (a tool call of the run is recorded executed, so its effect already happened), RETRY_SUPERSEDED, GOAL_CLOSED, GOAL_BUSY, INTENT_NOT_ACTIVE or RETRY_UNRECOVERABLE (the recorded input has masked secrets, or no tool call was recorded); 423 FROZEN; 400 ASYNC_UNAVAILABLE; 503 ENQUEUE_FAILED, with the retry row settled failed. A run quarantined as stale_unverified may already have had its effect — retrying it is the deliberate human decision to run it again, and is recorded as agent_run.retried in activity.

Multi-agent org: roles and capability narrowing (Module C)

Agent roles are data, not code. The agent_roles table holds a per-site role library (seeded with Planner, Writer, Translator, Taxonomist, SEO, FactChecker, Librarian — each with a deliberately minimal capability set; the Writer has no schema:* at all). When a run envelope carries agentRole, the harness narrows the caller's capabilities to role ∩ grant before the capability check (Step 2): a role can never exceed the token that runs it, and a token can never exceed the role it acts under. Unknown or disabled roles fail closed to the empty capability set. Roles are managed at /api/v1/agent/roles (admin).

The Planner decomposes a goal into role-scoped sub-goals (POST /api/v1/agent/goals/:id/decompose): sub-goals link via parentGoalId with origin='planner', inherit the parent's intent lineage and autonomy cap, and split the parent's remaining tool-call budget — the planner re-slices what is left, never mints new budget. POST /api/v1/agent/goals/:id/settle settles the parent from its children's terminal states: one failed sub-goal fails the parent (acceptance unmet); all completed completes it.

Trust gradient at the risk decision (L0–L4)

When a dangerous skill reaches Step 3, the harness resolves the effective autonomy level for (agentRole, capability) via the trust ledger — min(grant-or-default, intent cap, hard ceiling) — and routes accordingly:

  • ≤ L2 — classic pre-execute HITL: an approval record is created and the run parks as awaiting_approval.
  • L3 (veto window) — stageable single-item patches (updateItem with a data patch) execute into a staged revision instead of live content, paired with a kind='veto' approval whose autoCommitAt defaults to 4 hours out. Silence means consent: a delayed queue job on agent-veto-commits (plus a 5-minute safety-net sweep) promotes the staging to live at the deadline with full provenance. A human veto (POST /api/v1/agent/staged/:id/veto, admin or veto role) before the deadline discards the staging — live content was never touched — and records a veto incident that automatically demotes the agent role on that capability. Fields pinned by a human after staging win at commit time: the pinned part of the patch is dropped (auto_commit_partial), never overwritten. Pending stagings are listed at GET /api/v1/agent/staged and announced via a veto.staged activity entry with a review deep-link. Commit failures leave the staging intact and retry with exponential backoff; exhausting the attempts opens an incident.
  • L4 (autopilot) — the dangerous action executes directly within capability and budget; the kill switch still applies.
  • Irreversible skills (deleteCollection, deleteField) are hard-capped at L2 by the resolver and can never stage or run on autopilot.

Agent-as-reviewer (Module C)

With the contentOs.agentReview flag on, agents can decide routine approvals so humans only see exceptions (POST /api/v1/agent/approvals/:id/agent-decide). Hard rules: the reviewer needs review:<domain> for the approval's domain (schema-shaped tools → review:schema, item tools → review:items); self-review is forbidden — an approval belonging to goal-tree G can never be decided by a run inside G (ancestry paths are compared up to the root); veto-window approvals are human-only by definition. Only a confident approve (≥ the per-site agentReviewMinConfidence, default 0.8) finalizes — recorded with approverType='agent' and the reviewing run in approverRunId. A rejection or low-confidence verdict never finalizes: it escalates with a review.escalated activity entry and deep-link while the approval stays pending for a human (Req 11.4).

Trust ledger: promotion is human-gated, demotion is automatic

Autonomy levels are earned, and the asymmetry is deliberate: trust rises slowly through people, falls instantly through incidents.

  • Promotion (trust-ledger-service.ts) — a periodic sweep (POST /api/v1/agent/autonomy/promotions/check, or the trust-promote-check flow operation) evaluates every (agentRole, capability) grant below L4 against evidence: an unbroken streak of succeeded runs, enough decided approvals at the required approve-rate, and zero open incidents. Eligible candidates get a kind='promotion' approval proposing exactly one level up (capped at L4) with the evidence attached. The proposal is inert until a human decides it (POST /api/v1/agent/autonomy/promotions/:id/decide, admin-gated) — there is no auto-commit path for promotions, ever. Pending proposals are deduplicated and listed at GET /api/v1/agent/autonomy/promotions; current grants and open incidents at GET /api/v1/agent/autonomy.
  • Demotion — event-driven from incident insert: −1 level immediately (severity high drops straight to L1), no human required. A veto records an incident, so a vetoed staging demotes the role on that capability as a side effect.

Constitution: versioned publish-gate evaluators (Module D)

A constitution is a versioned set of evaluators (constitutions table, at most one active per site) managed at /api/v1/agent/constitution (versions list, draft, :id/dry-run, :id/activate — admin or constitution:write). Two evaluator types: a deterministic rule DSL (required, equals, max_length, min_length, regex, contains, not_contains over content fields) and llm_judge prompts that fail loudly with LLM_NOT_CONFIGURED when no provider exists. Identity is sha256 of the canonicalized evaluator list.

  • Pinning (Property 12) — the active hash is pinned to a run once (first-write-wins into run metrics); veto stagings carry it into revision provenance and artifact evaluations record it, so results stay reproducible even when a new version activates mid-run.
  • Publish gate — publishArtifact evaluates the active constitution against the artifact content: a blocking evaluator failure blocks publish; overriding requires an explicit reason and is recorded (constitutionOverridden) in the artifact metadata.
  • Dry-run + diff audit — drafts can be evaluated against real content samples before activation; activating archives the previous version and writes a constitution.activated activity entry with the added/removed evaluator ids and both hashes.

Deciding an approval: claim, execute, quarantine

Approving is not a status change — it executes the stored action. That makes the decision path a three-step one, claim → execute → finalize, and every step exists because of a way the naive version goes wrong:

  • Claim. The decision first moves the row to deciding, a conditional update guarded on pending. Read-then-act would let two concurrent approvals both pass the read and both run the action.
  • Execute. The claim is held for exactly one skill execution. Every path that does not complete releases it — a failing skill, a kill switch, a cancellation, a thrown error.
  • Finalize. Only a completed execution records approved.

A failure that the process survives does not go back to pending. If the skill touched a service before it failed, the side effect may already exist, and re-offering it as ordinary work invites a second one. Those land in failed instead — quarantined, out of the inbox, and re-runnable only through an explicit human step.

What no in-process handler can cover is the process dying mid-execution: a crash, an OOM kill, a Worker eviction, a deploy rolling the pod. The row then stays deciding forever, and because the inbox filters on pending it also stops being visible — stuck and invisible. A periodic sweep (sweepStaleApprovalClaims, 15-minute window) is the safety net. It also lands those rows in failed, not pending: a crash says "no decision was recorded", never "nothing happened", so a crashed execution is at least as ambiguous as one that failed in-process, and gets the same gate.

Elapsed time is not evidence the abandoned work stopped. A JavaScript timeout rejects a promise without cancelling the handler behind it, and a crashed process can leave a request in flight at an external provider. The window bounds how long the system waits, never what it concludes.

Recovering a quarantined approval

POST /api/v1/agent/approvals/:id/reopen with { reason } moves a failed approval back to pending. It requires the same approvals:decide capability as deciding, because reopening is what makes the action executable again. The transition and its approval.reopened audit record commit together: the record of who authorized a possible second execution is part of the authorization, not a log line that may or may not follow it.

In the Studio, the Approvals tab calls out how many approvals were interrupted, shows the reason recorded for each, and offers Reopen only on those rows — a pending approval is ordinary work and gets no recovery affordance. The reason is required: it is what a human states they verified, and it is stored with the transition.

Only failed reopens. A decided approval must not be revived, and a deciding row belongs to a live execution or to the sweeper.

Kill switch (the human "stop" right)

Four escalating scopes, all behind POST /api/v1/agent/kill-switch with { scope: run | intent | role | site, targetId?, reason? }:

  • cancel run — delegates to the run state machine (boundary cancellation from the async-runs work).
  • pause intent — flips the intent's status; the reconciler stops generating goals for it.
  • freeze role / freeze site — recorded as agent_freezes rows (liftedAt IS NULL = active; the table doubles as the audit trail with actor, scope, reason and timestamps). The harness consults the freeze state before creating any goal/run and at every tool-call boundary: an in-flight handler finishes, the next call is denied and the run is cancelled with stopReason frozen. A site freeze dominates every role; a role freeze blocks exactly that role. Approved-and-resumed executions are denied the same way — the kill switch wins over approvals.

Freezing/lifting a role or site requires the agents:freeze capability (admin included). While a site is frozen, POST /api/v1/agent/goals returns 423 FROZEN and the reconciler creates no goals — read endpoints keep working. GET /api/v1/agent/kill-switch lists active freezes plus history; POST /api/v1/agent/kill-switch/lift restores execution.

Asking a human: askUser (#710)

An agent that cannot continue without a decision it is not allowed to make calls the askUser tool instead of guessing. The run pauses, a person answers in the Mission Control inbox, and the run resumes with the answer as the tool's result. It is the same lifecycle as a parked approval, not a parallel one.

askUser and getUserAnswer are offered to governed agents — agent runs, the agent-runs queue worker and MCP — and not to the Copilot chat (POST /api/v1/ai/chat). The chat executes one tool call per turn and has no run to pause and resume, so its tool list (chatToolSkills() in llm-provider.ts, also used for the context budget) leaves both out.

Input (canonical contract askUser in @lumibase/contracts, limits in ASK_USER_LIMITS): question (1–1000 chars), optional options (2–10 choices, 200 chars each, trimmed and unique after trimming — they are compared with a trimmed answer), allowFreeText (default true; false requires options and makes the answer one of them), context (≤ 2000 chars) and timeoutMinutes (1–10080, default 1440 = one day). Required capability: items:read — asking has no side effect, so it is not on the autonomy gradient and is allowed at every level, L0 included.

Lifecycle. A question is an agent_approvals row with kind = 'question' (next to approval, veto, promotion); no new table, no migration.

StepRunQuestionaskUser tool callAudit (activity)
askqueued/running → waiting_userpending, expires_at setwaiting_useragent_question.asked
answerwaiting_user → succeeded (stopReason: answered)answered; answer in decision_reason, decided_by, decided_atexecuted, answer as outputagent_question.answered
timeout→ failed (stopReason: question_timeout)expiredfailedagent_question.expired
cancel / kill switch→ cancelled (stopReason of the cancel, frozen for a freeze)cancelleddeniedagent_question.cancelled

Asking, answering and expiring are each one short transaction with guarded conditional updates. Asking moves the run, records the question and marks the askUser tool call waiting_user in the same transaction, before any notification, so an answer that lands right after the commit is never overwritten. A waiting run leaves waiting_user only through its question (answer, timeout, cancel): a generic close or fail — say, a worker that throws after the park committed — and parking it on an approval are refused, so the question can never be left pending under a run that already ended. A waiting run holds no queue worker (the agent-runs worker returns as soon as the harness has parked it) and no database lock. It is in flight for the dispatcher (waiting_user is in the same in-flight set as awaiting_approval), so a goal does not get a second run while it waits, and POST /agent/runs/:id/retry refuses it.

Resume. A harness run is one governed tool call, so resuming means the parked askUser call completes with the answer as its result and the run settles succeeded. What the agent does with the answer is its next tool call, authorized on its own. The agent reads the answer with the getUserAnswer tool ({ questionId }), which only the principal or agent role that asked may read (or an admin grant; anything else is NOT_FOUND). The answer reaches the model as untrusted data: wrapped with untrustedBlock('ask-user-answer', …) plus UNTRUSTED_CONTENT_RULE, in both the tool call output and the getUserAnswer result. A person typed it, but an answer box is an input like any other.

Who may answer (POST /api/v1/agent/questions/:id/answer, behind the control-plane guard like the rest of /api/v1/agent). Only a signed-in user answers. Any API key — site-scoped or organization-scoped, whatever capabilities it carries — is refused with 403 before anything else is checked: a key is how an agent acts, and an agent must not answer its own question (an organization key resolves as a control-plane admin, so it would otherwise pass as an approver with no human in the audit trail). Among users:

  • a holder of approvals:decide, or a control-plane admin — the operators already trusted to unblock a parked run, and the only path for reconciler runs, which have no human requester;
  • the run's requester, when that requester is a user who still resolves on the site.

Today only site administrators reach this route: the control-plane guard in front of /api/v1/agent admits nobody else, and an administrator already counts as an approver. The requester rule is therefore not reachable yet; it takes effect once backlog B207 moves the answer route out of the control plane.

The answer is 1–2000 characters, trimmed; with allowFreeText: false it must be one of the options (400 ANSWER_NOT_AN_OPTION). The claim of the pending row and the resume of the run are one transaction, so a second answer gets 409 ALREADY_ANSWERED and never a second resume. Before resuming, the requester's rights are re-resolved (#472): a requester who lost access gets the run failed (stopReason: requester_revoked) and 409 REQUESTER_REVOKED. Other refusals: 404 (unknown or other-tenant id), 409 EXPIRED (the answer route settles an overdue question itself), 409 QUESTION_CLOSED (cancelled), 409 RUN_NOT_WAITING, 423 FROZEN.

Timeout. The agent-question-sweep (Docker cron every 5 minutes, and the 5-minute Cloudflare cron trigger) expires every pending question past expires_at and fails its run with question_timeout, which frees the goal for the dispatcher (a reconciler goal is then blocked with RUN_FAILED, never re-asked automatically). The failed run can be retried like any other; the retry asks the question again.

Kill switch. A running agent meets a freeze at its next tool-call boundary; a waiting run has none, so freezing a role or the site, and pausing an intent, cancel the waiting runs in that scope (stopReason: frozen) and close their questions. The run scope (cancel run) closes the question through the same cancel. An answer that races a freeze is refused with 423 FROZEN and the run is cancelled. A frozen agent cannot ask in the first place.

Approvals are not questions. GET /api/v1/agent/approvals excludes kind = 'question' rows; POST /approvals/:id/decide and /reopen answer 409 NOT_AN_APPROVAL for a question id; the agent reviewer refuses them with 403 HUMAN_ONLY.

MCP. A tools/call of askUser returns an ordinary tool result (never a JSON-RPC error, isError: false) with status: "waiting_user", questionId and runId — the same shape rule as pending_approval with approvalId.

Autonomy metrics. lumibase_agent_questions_total{status} (asked | answered | expired | cancelled) and lumibase_agent_question_wait_seconds{status} count questions as human interventions; answered runs are counted in lumibase_agent_runs_total with stop_reason="answered", timeouts with question_timeout. The trust ledger shows questions: { asked, answered, expired } in a promotion's evidence; it is evidence for the human deciding, not a gate — an unanswered question already breaks the run streak because its run ends failed.

Multi-tenancy. Isolated per tenant: the question row (agent_approvals, site_id, already under site_isolation RLS), its run and tool call, the audit rows, the list/read/answer routes (every query filters site_id; an id from another site is 404) and notifications (fan-out filtered by siteId). Shared by the deployment: only the sweep, which scans across sites on the system connection and settles each row inside its own site scope. Verified by agent-ask-user.db.integration.test.ts (two sites on real PostgreSQL).

Notifications. Asking emits an agent notification of kind question with a deep link to /mission-control/inbox?entry=question:<id>; its body carries the question masked the same way as the stored tool input. The agent-runs queue worker passes a Web Push sink for approval and question only, so a background run's question reaches a person without the Studio tab open; queued runs still send no run succeeded/failed (or other lifecycle) pushes. The send is best effort: on Workers it is kept alive with the queue consumer's waitUntil, on Docker the worker awaits it, and a failed send never fails the run.

Rollout flags and Content OS metrics

Four per-site flags (settings key contentOs, all default off) gate the autonomous surfaces: reconciler (drift scans generate goals), vetoWindow (L3 stages instead of falling back to classic pre-execute HITL), agentReview (agents may decide approvals), mcp (the /api/v1/mcp endpoint answers). With every flag off the system behaves exactly like the pre-Content-OS Copilot + harness baseline.

Rollout metrics (Prometheus): lumibase_agent_autonomous_operations_total{level} (L3 stagings + L4 autopilot — autonomous operation rate), lumibase_agent_veto_stagings_total / lumibase_agent_vetoes_total (veto rate), lumibase_agent_coalesced_writes_total / lumibase_agent_write_flushes_total (coalescing ratio), lumibase_agent_backpressure_activations_total, lumibase_intent_open_drifts{intent} and lumibase_intent_breaker_trips_total (intent health).

Load-aware autonomy (Load Guard)

A system that generates load must also sense load. Three guards bound agent-originated work:

  • Write coalescing — every skill handler runs inside a coalescing window: item writes defer their materialized-view refresh and flush exactly once per collection at the tool-call boundary (N writes to one collection cost one invalidation), on success and failure alike.
  • Write rate budget — when a run envelope carries budget.maxWritesPerMinute (reconciler goals attach it from their intent), write-capable tool calls consume a sliding-window quota scoped to ${siteId}:${intentId}. An exhausted budget defers the tool call with write_budget_exceeded and a retry hint — the run is not failed.
  • Backpressure — the Node entrypoint feeds event-loop pressure samples into the guard every 5s. Overload pauses reconciler-origin runs only (human-triggered work is never auto-paused) with a load_guard incident recorded once per activation per site; a hold-down of continuous calm auto-resumes. Activations are counted in lumibase_agent_backpressure_activations_total, budget deferrals in lumibase_agent_write_budget_denials_total.
  • Maintenance windows — intents may declare { tz, windows: [{ dow, start, end }] }; outside the window the reconciliation cycle is a no-op and open drifts queue until the window opens. Overnight windows span midnight; an invalid timezone fails open.

Core Skills Registry

Skills are defined in two synchronized locations:

  • Public registry (packages/ai-skills/src/skills.ts) — LLM tool definitions exposed via getAISkillsAsTools()
  • Harness handlers (apps/cms/src/services/ai-harness.ts → buildCoreSkills()) — actual execution logic

A skill is classified as DANGEROUS (requires HITL approval) when:

  1. It sets the explicit dangerous flag (governed namespaces access:*, intents:*, flows:*, and content-version writes createVersion/updateVersion/promoteVersion), OR
  2. Its requiredCapabilities includes any schema:* except schema:read, OR
  3. Its name starts with delete

The dangerous flag is honoured by both AISecureHarness.evaluateRisk and ToolRegistryService.coreTool, so item CRUD keeps its existing (non-dangerous) classification while new governed write/delete skills are always gated.

Item CRUD is additionally classified by impact (#578): createItem with status: 'published', an updateItem that changes the item's status, and an updateItem on a published item go through the same approval and autonomy gradient as a dangerous skill. Draft writes are unchanged. See governed tool contract §3a.

SkillServiceRequired CapabilityRiskHandler
listCollectionsschemaschema:readSAFEReal → SchemaService
createCollectionschemaschema:createDANGEROUSReal → SchemaService
deleteCollectionschemaschema:deleteDANGEROUSReal → SchemaService
createFieldschemaschema:updateDANGEROUSReal → SchemaService
deleteFieldschemaschema:deleteDANGEROUSReal → SchemaService
listItemsitemsitems:readSAFEReal → ItemService
createItemitemsitems:writeSAFEReal → ItemService
updateItemitemsitems:updateSAFEReal → ItemService.patch()
deleteItemitemsitems:writeDANGEROUSReal → ItemService.softDelete()
listVersions / compareVersionitemsitems:readSAFEReal → ContentVersionService
createVersion / updateVersion / deleteVersionitemsitems:writeDANGEROUSReal → ContentVersionService (draft branches)
promoteVersionitemsitems:writeDANGEROUSReal → ContentVersionService.promote() → main via ItemService.patch (writes a revision; revision-protected, so not hard-capped like schema drops)
aiSuggestFieldaischema:readSAFEReal → LLM + existing-field context (offline registry: keyword patterns)
aiContentAssistaiitems:readSAFEReal → LLM + RAG item samples via ItemService
generateAppSpecaischema:read, items:readSAFEReal → LLM + live schema introspection; sections must declare source bindings
generateApiDocsaischema:readSAFEReal → deterministic OpenAPI 3.1 from live schema (no LLM needed)
generateSeedDataaiitems:writeSAFEReal → LLM rows matching real field definitions
listRelationsschemaschema:readSAFEReal → SchemaService
createRelationschemaschema:createDANGEROUSReal → SchemaService
deleteRelationschemaschema:deleteDANGEROUS · irreversibleReal → SchemaService
listRoles / listPoliciesaccessaccess:readSAFEReal → AccessService
createRole / createPolicyaccessaccess:createDANGEROUSReal → AccessService
deleteRole / deletePolicyaccessaccess:deleteDANGEROUS · irreversibleReal → AccessService
listIntentsintentsintents:readSAFEReal → IntentService
createIntent / deleteIntentintentsintents:writeDANGEROUSReal → IntentService
listFlowsflowsflows:readSAFEReal → DB (tenant-scoped)
createFlow / deleteFlow / runFlowflowsflows:write / flows:runDANGEROUSReal → DB + runFlow
listApiKeysaccessapi-keys:readSAFEReal → AccessService
createApiKey / rotateApiKey / revokeApiKeyaccessapi-keys:create / :write / :deleteDANGEROUS (revoke irreversible)Real → AccessService (token returned once)
listUsersaccessusers:readSAFEReal → AccessService
inviteUser / updateUser / removeUseraccessusers:write / :deleteDANGEROUS (remove irreversible)Real → AccessService
listTeamsaccessteams:readSAFEReal → AccessService
createTeam / deleteTeam / addTeamMember / removeTeamMemberaccessteams:write / :deleteDANGEROUSReal → AccessService
listSettings / listTranslations / listWebhooksaccessconfig:readSAFEReal → ConfigService
upsertSetting / deleteSetting / create*/update*/delete* translation & webhookaccessconfig:write / :deleteDANGEROUSReal → ConfigService
listExtensionsaccessextensions:readSAFEReal → ExtensionsService
installExtension / updateExtension / uninstallExtensionaccessextensions:write / :deleteDANGEROUSReal → ExtensionsService
askUseragentitems:readSAFE · parks the runIntercepted by the harness → AgentQuestionService.ask (run waiting_user)
getUserAnsweragentitems:readSAFEIntercepted by the harness → question outcome, answer as untrusted data

Irreversible skills (hard-capped at autonomy L2): deleteCollection, deleteField, deleteRole, deletePolicy, deleteRelation, revokeApiKey, removeUser.

Standalone-only (not on the governed endpoint): NDJSON export_backup/restore_backup, marketplace install_marketplace_extension/publish_extension, and binary media — these stay on the @lumibase/mcp-server REST passthrough where their bespoke crypto/SSRF/NDJSON logic already lives.

Skills can be overridden per-site via the agent_tools database table without redeploying.

Generation skills resolve the LLM through createConfiguredLLMProvider. When LLM_PROVIDER (plus credentials) is not configured they fail with an explicit LLM_NOT_CONFIGURED error — there is no silent stub fallback in production routes. Provider failures surface as LLM_PROVIDER_ERROR, malformed model output as LLM_INVALID_JSON. Successful LLM-backed runs record { llm: { provider, model, estimatedTokens } } in agent_runs.metrics.

App generation MVP

When a user asks LumiBase to generate an app, the agent reads the existing schema, content, policies, and scoped memory, then produces artifacts:

text
Goal: generate storefront
Context: products, orders, customers, storefront policy summary
Outputs:
  - page_spec artifact
  - component_spec artifact
  - api_spec artifact
  - seed_data artifact
Evaluations:
  - JSON schema validation
  - API spec validation
  - permission diff lint
  - prompt safety check
Approval:
  - required before publishing risky artifacts

The initial e-commerce template targets products, orders, customers, and storefront pages. It is intentionally artifact-first: generated outputs are reviewed, evaluated, approved, and then published.

The current MVP returns four artifacts from /api/v1/agent/generate-app: page_spec, component_spec, seed_data, and api_spec. Publishing is idempotent, rollback is available for published artifacts, and failed schema/migration evaluations block publish unless an override reason is supplied.

Runtime compatibility

The harness service layer runs inside the CMS request/runtime boundary and uses existing database and route abstractions. It does not require Cloudflare-only APIs for the current MVP.

  • Cloudflare Workers: CMS routes and Drizzle-backed harness services run in the Worker runtime.
  • Docker / Node.js: the same API routes and services run in self-hosted mode.
  • Queues: repeated run failure uses the runtime QueueProvider to enqueue agent-dead-letter; if no queue adapter is available, the failed run remains fully audited in agent_runs and agent_tool_calls.
  • Observability: Prometheus metrics cover run status, stop reason, tool latency, approval latency, evaluation status, estimated token/cost usage, and dead-letter enqueue rate. Docker mode auto-loads the LumiBase Agent Harness Grafana dashboard.
  • Long-running generation/evaluation jobs that exceed request runtime limits should be moved behind queue/workflow execution in a later phase. Until then, the MVP keeps evaluation runners short and synchronous.

If future evaluation runners depend on runtime-specific APIs, they must be feature-flagged and documented in runtime-abstraction.md.

Security model

  • All agent tables are scoped by siteId.
  • Item revisions carry provenance: skill-driven writes are stamped authorType='agent' with the executing createdByRunId, while Studio/API writes by people record authorType='human'. The harness sets this on the ItemService before any skill handler runs.
  • Law Zero (override-is-law): human edits on collections governed by an active content intent pin the touched fields (items.pinnedFields). Agent writes to pinned fields are denied at the ItemService boundary with PINNED_BY_HUMAN; pins are listed/released via GET/DELETE /api/v1/items/:collection/:id/pins[/:field] and every pin/release is audited in the activity log.
  • Tool inputs and memory context are redacted/masked before audit or prompt assembly.
  • Prompt text cannot grant permissions; only policy snapshots and capability grants can.
  • MCP control-plane backstop (defense-in-depth): the /api/v1/mcp endpoint is intentionally not in CONTROL_PLANE_PATHS, so — like /api/v1/agent/* — it relies on the harness's in-code capability + HITL checks, which are byte-for-byte identical to the Agent API (Property 14). On top of that, a tools/call targeting a control-plane skill (any DANGEROUS / schema-mutating / delete* skill, per isControlPlaneSkill — e.g. deleteCollection, triggerDeployment, createCdcSubscription, replayCdcSubscription, deleteCdcSubscription) is rejected with 403 CONTROL_PLANE_FORBIDDEN before the harness runs unless the caller is an admin principal, and the denial is audited (mcp_control_plane_skill_denied). Discovery (tools/list, initialize, ping) and safe read skills stay open to non-admins. This mirrors withControlPlaneAccessGuard()'s intent — control-plane operations stay behind an admin even if the in-code check is later weakened.
  • Approval decisions are recorded with actor, decision, reason, and timestamps.
  • Tool calls preserve input/output/error metadata for audit while avoiding plaintext secrets.

Operational metrics

The harness records the raw fields needed for:

  • Run count and success/failure rate.
  • Approval latency.
  • Tool latency.
  • Evaluation failure rate.
  • Budget stop reason.
  • Estimated cost.
  • Artifact size/hash tracking.

These fields can be aggregated by the observability layer described in observability.md.

Push notifications

The harness accepts an optional notify sink (AISecureHarnessConfig.notify, type AgentNotifier) — this is an out-of-band operator signal only; it does not change any skill, handler, capability, or risk classification. When a request-context caller wires it (buildAgentNotifier(c)), the harness and the services it drives push a notification at the moment of the event:

  • approval — a HITL approval was created (a reviewer is needed).
  • question — an agent asked a human with askUser (an answer is needed).
  • veto — a write was staged into an L3 veto window (auto-commit deadline).
  • incident — an agent_incidents row was recorded (severity ≥ warning).
  • run — an agent run succeeded or failed.
  • goal — a parent goal settled to completed/failed.

Delivery is best-effort over two transports — the per-site SiteRoom realtime WebSocket (in-app) and Web Push (VAPID, even with the tab closed) — and never blocks or fails execution. The agent-runs queue worker passes a Web Push-only sink (it has no realtime namespace); other background callers (reconciler/cron) omit the sink and rely on the Mission-Control inbox poll, so no event is lost; only its latency changes. See push-notifications.md.

Relationship to AI Copilot

The existing AI Copilot remains the human-facing chat entry point. The Agent Harness Layer is the governance and execution substrate underneath it.

Use ai-copilot.md for legacy Copilot chat and HITL behavior. Use this document for first-class agent lifecycle, tools, approvals, artifacts, evaluations, memory, and app generation.

Last modified: 15/11/2026