Deep dive · Operations & reliability

Observe. Respond. Recover. Learn.

Monitoring, alerting, and resilience are part of the existing loops. Criticality determines the depth: agree and verify requirements in Build, interpret relevant operational indicators in Value, and learn from incidents in Learning.

Plan for operations Proportionate to criticality

A change is not complete simply because it works. Critical components must also be observable. LoopsWork therefore includes monitoring, alerting, and resilience in the specification.

Observe

How can we tell that the component is working?

Define relevant metrics, logs, and traces. Datadog, Grafana, Sentry, or similar monitoring and observability tools are possible choices. LoopsWork remains vendor-independent.

Alert

When must someone respond?

Only relevant deviations trigger an alert: clear, with as few false alarms as possible and a defined response. Where possible, the alert links directly to the affected component or error. Ownership and escalation are clear.

Recover

What happens when something fails?

Before implementation, agree on system behavior during an outage: for example, retry, fallback, graceful degradation, rollback, circuit breaker, or manual intervention. Not every component needs every mechanism; criticality determines the effort.

Learn

What can we learn from incidents?

Incidents, false alarms, and near misses feed back into the Learning Loop . Learning changes specs, tests, alert thresholds, monitoring, architecture, agent context, or working agreements and shapes the next change.

  1. Operations · feedbackIdentify an incident, false alarm, or near miss.
  2. Learning Loop · L1–L6Investigate causes and embed improvements.
  3. ↻ Next Build Loop · B3Use the new knowledge to agree on the spec, tests, and operational behavior.Go to the next spec

The more critical a change, the more precisely its observability and failure behavior must be described. Simple changes remain lightweight. The existing loops, roles, and rituals are sufficient.

Open the optional spec section · Acceptance Slot

Build also means ensuring failures can be detected and the response is understood.

Show evidence of operational quality

For critical components, completion criteria include agreed observability, useful alerts, and tested failure behavior. Is the evidence available? Scale the scope to risk and keep it lightweight for low-risk changes.

Agree on operational requirements in B3

Operational indicators that matter

Alongside business metrics, technical operating indicators show whether a change actually works: error rates, latency, availability, SLO violations, abandonment rates, or technical failures directly affecting users.

Only metrics with a clear influence on users, business impact, or system reliability belong in the Value Loop. Select relevant signals for the Scorecard. The Value Loop is not a technical monitoring dashboard. Address acute incidents immediately.

Learn from incidents

Incidents are feedback. LoopsWork uses them to improve specs, tests, monitoring, and working conditions over time, alongside fixing the immediate issue. False alarms and near misses also provide useful signals.

Choose relevant learning questions
  • Why was the error not detected earlier?
  • Was the alert useful?
  • Were there too many or too few alerts?
  • Was failure behavior described adequately?
  • Could a test or clearer spec have prevented the error?
  • Does the agent context need to change?
  • Does the architecture need to become more resilient?

↻ Learning → next Build: Proven learning changes the next spec in B3, test cases, and operational rules. Versions and named owners make adoption traceable.

AI / MAKE OPERATIONS VERIFIABLE

AI builds more than features.

AI also helps make operations verifiable. For critical changes, an agent task includes more than implementation code when required by the confirmed spec.

Agents support

  • Generate monitoring configuration
  • Add logs and define metrics
  • Configure alerts
  • Create failure-case tests
  • Implement fallbacks and recovery behavior
  • Document runbooks or technical guidance

Humans are accountable for

Criticality, acceptable risk, alert thresholds, SLOs, escalation, and approval remain human decisions. Agents work within confirmed constraints and permitted tools. A proposal is not an approval.

Existing accountabilities remain in place. Extend the task only as far as criticality requires.

Record operational requirements in the spec · Observe, Alert, Recover, Learn