An eight-stage development pipeline. Work starts by claiming a bead, then is specified, planned, built, proven, reviewed by agents and then by a developer, and handed to QA on an isolated environment. Each stage reads the files the previous stage produced. It does not depend on the chat transcript that produced those files.
Background
I was given a mission to figure out how to build software in an agentic-native workflow that is reproducible: next-generation software development. The assignment was a new registration platform. It replaces an old third-party tool that comes with a yearly six-figure cost and many five-figure bug fixes. The replacement is a Django 5.2, Wagtail, and Vue application that handles applications, payments, catalog, and admin. People still own the result: they approve the spec and the plan, they adjudicate review findings, they walk the feature themselves, and they hand a working environment to QA. Agents do most of the implementation work in between.
This is a bet on a different way to build custom software going forward. Large language models draft specs, write text, write tests, implement stories, and can guide a developer through the feature. Those are the agents' strength, and two runs of the same prompt can reasonably differ. The jobs that must not differ are not given to a model. Which story is next, whether a story is done, what risk tier a change is, whether the required criteria are proven, and whether lint and tests passed are decided by Python scripts. Quality is not a later inspection that someone might forget. It is a gate the loop cannot skip. Judgment is reserved for the human.
The spine · stage, artifact, gate
01 · Specify
We chose spec-driven development so the thing being built is
defined before an agent starts coding. It does not just come
from a Jira ticket and a developer in the prompt. This allows
later stages to compare against that spec instead of against a
chat in agent memory. The tool for that
is Spec Kit.
/speckit-specify takes a claimed bead and turns it
into a spec package: user stories, acceptance scenarios, a UAT
checklist, and related notes. Those acceptance scenarios are what
the rest of the pipeline treats as the contract. A person approves
the spec before planning starts. Rejecting it stops the workflow.
Jira is for people. Beads are for agents. The bead is long-lasting context outside those sessions: claimable so two agents cannot take the same story, and enrichable so the spec (and later the plan and tasks) is written back onto the bead instead, which exists between agent sessions. The path is Jira ticket to bead to spec.
Spec Kit is thorough. A single feature directory can hold a
thousand-plus-line spec.md plus plan, research, data
model, contracts, checklists, and a quickstart. That volume is
useful for the machinery and unusable as a review surface. The
dashboard we built parses the package and shows the human
reviewer the facts that matter: the user stories, their
acceptance scenarios, and the current stage. The raw markdown
is still there if someone needs it. The default view follows a
principle of progressive disclosure.
02 · Plan
The plan gives us orientation and logistics: what is being built, for whom, what is out of scope, and how the existing system will change.
The dashboard also flags mutative operations called out in the
spec: schema or data changes, and whether they are additive,
data-mutating, or destructive. Implementation phases stay behind
Source. The raw plan.md and
data-model.md are one click away if someone needs
them.
The piece built for this stage is the system-change graph. It is derived from the data model and contracts, not drawn by hand in the plan. The graph is the existing system. Yellow nodes are what this plan adds. Red nodes can change stored data or fan out widely. Clicking a node opens an inspector (kind, risk, and what it touches). A second diagram, the runtime workflow, is the planner's mermaid of the happy path and the important failures. Together they answer the logistics question: what will mutate, and in what order, before a builder is spawned.
03 · Build
When the build is kicked off, it starts with an orchestrator.
The orchestrator does not decide what to do next, and it does
not write application code. After every handoff it runs a
Python program, build.step, which classifies the
current story, records progress on a receipt, and prints one of
three instructions: spawn a named specialist with a written brief,
commit a story the verifier already passed, or stop because the run
has reached a terminal state. The orchestrator is only allowed to
do what the script printed.
Specialists are separate subagents with narrow jobs. A test-writer
writes the failing acceptance tests for a story (pytest for
backend scenarios, Playwright for scenarios tagged
[UI]) and is required to leave them red. A builder
then reads the spec and those tests and implements the feature. The
builder is not allowed to edit the acceptance tests to make them
pass. If a test is wrong, the builder reports blocked rather than
"fixing" the proof. A separate verifier re-runs the gates and
writes a pass or fail. The builder never marks its own story done.
That split exists because same-context test-driven development (one agent writing the proof and then the code) tends to green-bar its own misunderstanding. People can hold both jobs in their head. This loop is not allowed to. Progress is written to a receipt the dashboard reads, not to the chat. If the session crashes, the next run continues from that receipt.
A build ends when there is nothing left to implement. Every user story from the spec has been through the verifier: the acceptance tests that were written first now pass, and the builder did not get to mark that itself. That is the pipeline saying the stories in the spec and plan were actually built. The run can also stop short: something in the environment is blocked, retries ran out, or two attempts in a row changed nothing. Those are hand-backs, not a finished feature.
Passing the stories is still not the same as being ready for review. Before functional review unlocks, the change also has to pass the ordinary quality gate that any pull request would: lint, the test suite, Django's system check, and a check that the migration files still match the models. If that gate is red, the dashboard keeps review locked. The next stage asks a different question: not "did these stories pass their tests," but "does every required criterion in the spec have proof."
04 · Prove
Functional review is how we prove the feature works the way the
spec said it would, on the running application. A person starts
/speckit-review. Pytest then runs two lanes. The
first is ordinary review-marked tests: services, views, and
models, no browser required. The second is Playwright against a
live Django server. Those tests log in as the actor the spec
named, go to the real URL, and do the thing a student or an
admin would do. If the spec says a student can upload a PDF in
the application portal, an agent writes a Playwright check that
opens that portal, attaches the file, submits, and asserts the
page that comes back. A view test that the form exists is not
that proof.
When a required scenario still has no such check, the loop spawns a specialist to write one. Browser-tagged scenarios go to a UI test-writer (Playwright). The others go to a backend test-writer. If a check is red, a builder is spawned to fix the product, not to weaken the test. After each wave, pytest runs again on this feature. The agents are not asked to narrate whether it worked. The suite either did the flow or it did not.
Every review test carries the criterion it proves, the layer (browser or not), the actor, the URL, and how to re-run it. After the assertions pass, Playwright takes a screenshot of the proven page. Those rows, screenshots, and failing traces land in a review packet. The dashboard shows the human the verdict, the criteria that still need a person, and the ones already proven, with the raw packet one click away. That is both a spec proof and a regression of the flows this feature touches: the agent actually clicked through the app, and the packet is the record of how.
05 · Regression
After functional review converges, make review-regression
runs an unscoped crawl: both the deterministic test lane and the
browser lane, against the whole review suite, not only the feature
that just landed. The result is written to
review-regression.json, including the git SHA and the
timestamp of the summary it was compared against.
Sibling features can fail in ways that are informational for the gate (for example a flaky browser check in an unrelated spec). Those still run. The crawl is not allowed to skip or deselect sibling files in order to go green by omission. Code review and the QA handoff both require this artifact. They do not re-run the crawl themselves and they do not treat the feature-scoped packet as a substitute for it.
06 · Code review
Code review runs on the bead branch after regression, before anyone opens a pull request into main or develop. It is a hybrid: an agent does an exhaustive, citation-backed read of the diff, and a person decides what matters. The agent is not asked to "find bugs." It is asked to fill two documents. The verification lane is a behavior-to-proof matrix (what changed, which test proves it, whether that test goes red without the change, whether a failure path exists). The correctness lane is a defect report in which every finding has a location, input, expected result, actual result, reproduction, severity, and route. Anything that cannot fill those fields goes on a separate questions list. It is not ranked as a bug.
Risk tier is computed from the changed files, not declared by the author. Paths are matched against a single JSON list. One high-risk file in an otherwise documentary pull request makes the whole pull request high-risk. Money-handling code is also matched by content (SDK calls and settings keys), because filename globs cannot express "anything that talks to Stripe." A developer may raise the tier. Lowering it requires a named approver and a recorded reason.
Before the agent lanes run, a deterministic gate executes a fixed set of commands (lint, the test suite, Django's system check, a migrations-match-models check, and a migration linter) and records real subprocess output against the SHA it ran. Those commands are not optional decorations. Several of them catch classes of error that pytest structurally cannot see, because the test suite builds its schema from the models and never executes the migration files. This stage has been run on live pull requests, including the recent production-hardening work.
07 · Developer review
After code review, the next step is not a pull request into
main or develop. A developer reviews the feature locally. They
already know what to check: the spec's acceptance scenarios,
plus the review packet that said which of those are proven.
/speckit-uat-prepare turns that into a brief, a
checklist, and a walkthrough. The developer starts the local
server, goes through every item, and only then is the feature
ready to leave their machine.
That walk is both a last look at the spec and a regression the person can feel. Playwright already drove the same flows in stage 04. Here a person clicks them. If something is wrong, it comes back to the bead branch.
08 · QA
This pipeline ends when QA can use the feature, not when the
business signs it off. Official UAT sits after that handoff.
To get there, the same bead key that named the git branch
also names a data branch and a live box. The git branch is
bead/<KEY>. The database is Dolt on a Fly
box: a branch pr-<id> is peeled from
main and migrated so its schema matches this
feature's code. Those two branches stay paired. The app is
not pointed at shared staging data, and the data is not
pointed at someone else's code.
/speckit-uat-prepare also stands up that
environment. spawn-preview.sh creates a
scale-to-zero Fly app, preview-<id>.fly.dev,
pins it to the Dolt branch, and gives it its own media
bucket. The developer hits the URL, confirms the box woke
and the flows still work there (not only on localhost), and
hands the environment to QA. QA is a separate team doing an
independent review. They do not walk the developer's
checklist. That would be leading the witness. When they
are satisfied, the work can go to
official business UAT. Merge into main or develop is a
later decision. It is not this stage.
Why this exists
The pipeline and the dashboard exist for one reason. Agents generate a lot of finished-looking text: specs, plans, research, tasks, packets. A person cannot read all of that at the pace tickets move, and polish is no longer proof of correctness. That is the argument in Distilling the Signal: when production gets cheap, verification gets heavier, and attention does not scale with volume. So we put a real software development lifecycle, with deterministic gates, around agents that are probabilistic. Programs decide what happens next and whether it counted. The dashboard is the distillation layer. It does not summarize the ticket. It shows a verdict, points at what still needs a person, and leaves the rest collapsed.
That is a balance, not a finished product. Agents draft and implement. People approve the spec and the plan, adjudicate findings, walk the feature, and hand QA an environment. We're seeing early success. Features move, and the same gates run every time instead of depending on the developer, the agent, or the context window.
The basics of agentic engineering are already in this harness. Loop engineering is the control plane: observe, decide, spawn, stop. The session does not invent the next step, and a loop that lets the model declare victory is not a loop we run. Prompt engineering is how specialists are briefed. A test-writer gets a written job and is required to leave the tests red. A builder gets the spec and those tests and cannot edit the proof. Context engineering is where memory lives when the chat is gone: on the bead, in the spec and plan files, in the receipt the dashboard reads. The dashboard is progressive disclosure over that context, not a second copy of the markdown.
The plan is to expand on this. Right now this is a one-developer, one-agent, one-feature pipeline. Once that is dialed in, we scale it: multiple agents, multiple specs. Any developer can pick up the work as if they had already been onboarded for six months. We also want the loop to become more autonomous. That sounds counterintuitive, like writing the implementation job away. The job is already becoming something else: architect, reviewer, and a conduit between the technology and the business.