JSON contracts between pipeline stages: treating remediation as data

An early version of the remediation pipeline passed positional, whitespace-delimited text between stages. It worked until one evidence field contained a space. Nothing crashed. The parser produced a plausible-looking plan for the wrong object.

That is the worst class of infrastructure bug: silent, confident, and downstream of the thing that actually broke.

The fix was an interface, not a parser

Every stage boundary now exchanges typed JSON records, validated on read. Detection writes detected-issues.json. Planning writes action-plan.json. Execution writes execution-result.json.

{
  "phase": 1,
  "actions": [
    {
      "type": "patch_api_version",
      "target": { "kind": "Ingress", "namespace": "payments" },
      "justified_by": ["deprecated_api_version"],
      "order": 2
    }
  ]
}

What that buys you

• Malformed input fails loudly at the boundary instead of silently producing wrong work.
• Each action references the findings that justify it, so a plan can be audited without reading code.
• Stages version independently — a new detector field does not require a new executor.
• Fleet-wide reporting becomes a query over result records rather than a manual sweep of logs.

Plans as reviewable artifacts

Once a plan is data rather than a script, review becomes possible. Execution supports DRY_RUN=true mode, which replays a phase and prints exactly what it would do. The plan is read before the apply run is authorised, and every applied action logs its inputs, decision and result.

The general rule I now apply to any infrastructure automation: if a stage cannot serialise its output, it cannot be reviewed, and if it cannot be reviewed it should not have write access.

More notes

• Detection first: remediating Kubernetes upgrades without mutating clusters
• Exit code zero is not a success signal: closing the remediation loop