AI Engineer World's Fair 2026

Every Solo Agent Builder Eventually Reinvents a Worse Version of CI/CD

Loading slides…

Read the talk

Every Solo Agent Builder Eventually Reinvents a Worse Version of CI/CD

A polished artifact can still be a failed run. A recurring content pipeline shows how blocking contracts and audit trails keep agent failures from shipping downstream.

From a talk by Sumaiya Shrabony

From prompts to seven handoffs

You start building an agent to write, research or automate a workflow. Then a prompt change breaks a downstream step, a scheduled run disappears, or an output reaches its destination when it should have stopped. Maintaining the workflow begins to resemble maintaining a delivery pipeline. Sumaiya Shrabony encountered this while running a nineteen-skill Claude Code system spanning writing, research, vault sync, analytics and other tasks.

Her open-source Agentic Content System runs every other Saturday and produces twelve content pieces per run. It reads a knowledge vault, creates a research brief, builds a content plan, produces the pieces, then applies verification, reviewer gates and deduplication before saving Markdown files. Those numbers describe the configured workflow, not measured throughput or reliability.

The workflow has seven handoffs: scheduler → command → research → content plan → production skill → verifier → reviewer → output folder. Each arrow is a boundary where one component can pass misleading output to the next. For a solo builder, there may be nobody else to notice that the workflow’s apparent success conceals a failure.

Workflow from Scheduler through Command, Research, Plan, Production Skill, Verifier and Reviewer to Output, with three failure points marked in red.Open full source frame
Seven handoffs connect the scheduler to the final output.
0:000:17
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:00 · section reference included

Five controls rediscovered through failures

These operational obligations tend to appear one failure at a time. First, a prompt or skill changes and breaks an expected output. Later, the scheduler or a downstream consumer fails. Each repair recreates a familiar software-delivery control.

Failure that prompts a repairControl rediscovered
A prompt change breaks the expected output shapeRegression testing
A scheduled run fails silently and goes unnoticedCI monitoring and alerts
A schema change breaks downstream skillsContract testing at the boundary
A finished-looking artifact should not enter the ready folderA staging checkpoint
A bad output cannot be traced to its originAudit trails

The progression moves from checking the result to controlling its passage through the system, then preserving enough evidence to diagnose what happened. Logging becomes necessary when the final artifact cannot tell you which prompt, skill or handoff introduced the problem.

The CI/CD comparison concerns operational guarantees. It does not require treating an agent run as a software build. Shrabony’s point is that builders need those guarantees but end up assembling them independently, often without recognizing what they are rebuilding. Her criticism concerns the guarantees their systems actually enforce; domain-specific acceptance rules do not appear simply because a workflow uses agents.

1:522:14
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:52 · section reference included

A polished artifact can still fail acceptance

An obviously bad output invites correction. A polished output can evade it. The piece reads well but uses the wrong voice, makes an unverified claim, repeats an old angle or omits required sections—and still receives a ready-to-publish label. Accepting it is like shipping compiled code without running the tests. Apparent completeness is not evidence of acceptance.

The demonstration starts with a small, privacy-safe version of the content pipeline, separate from the full production repository. Its sequence is simple: generate a content artifact, run gates, then allow or block the output. On the happy path, the Markdown contains a caption, pinned comment, visual brief, verification log, vault assets, production notes and a ready status.

Terminal preview showing generated content, green PASS messages for production, review and saving, and a final READY status.Open full source frame
The happy-path demo saves the generated artifact as READY output.

The terminal result looks successful, and the content looks professional. Seen alone, that run would make the system appear finished. The useful test comes next: can an artifact still look ready when the path that produced it has failed?

3:393:53
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:39 · section reference included

Voice drift passes the shape check

The first failure opens with “Unlock the power of AI adoption,” followed by a game-changer promising to transform teams in a fast-paced enterprise landscape. The problem is recognizable generic marketing language: a structurally complete piece that does not match the intended voice.

Naive mode saves it anyway. All the required sections exist, the artifact carries a ready status, and it looks complete. Guarded mode instead blocks it at the voice contract, stopping the pipeline before the artifact enters the publish-ready folder.

The gate does not rewrite the piece or improve its voice. It prevents an unacceptable piece from being promoted. For a content system, voice is a useful first boundary; for another domain, the equivalent question is which domain rule can be violated even when the output has the right shape.

5:135:26
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:13 · section reference included

A precise claim without evidence

The second artifact asserts that teams with a clear semantic ownership model reduce AI rollout rework by 37%; this is an unsupported demonstration claim, not an established result. Its precision makes the next question unavoidable: where did the number come from? The verification log is empty.

The prose is usable and the number sounds plausible, so naive mode saves it. Guarded mode blocks it under a verification contract: claim-bearing content cannot ship without a verification trail. The artifact’s confidence cannot substitute for a traceable source. When a system makes assertions about data or users, missing verification becomes a credibility problem even if the surrounding writing is excellent.

6:296:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:29 · section reference included

New output can repeat an old angle

The third failure is subtler. The output is newly generated and technically coherent, but its opening hook repeats an angle already present in the knowledge vault: “AI adoption fails when the dashboard looks right but the workflow is wrong.” Checking the piece in isolation misses the defect. The comparison must include historical content.

A stream of near duplicates can make the system’s output feel repetitive even when every individual piece is sound. Guarded mode blocks this artifact at the data contract and also writes an audit record. The rejection protects the output folder; the record makes the rejection explainable.

For an unattended run that fails at two in the morning, the final artifact is insufficient. You need to know which gate failed, which contract was violated and why. An audit trail preserves that context so diagnosis does not depend on guessing from the output or reproducing the entire run. This is the fifth control solo builders often add only after a failure has already caused damage.

7:257:38
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:25 · section reference included

Define what must be true before saving

These failures do not require waiting for a new platform or framework. They require a small set of explicit controls around the artifact and its passage downstream.

ControlQuestion it answers
Pre-save output contractDoes the artifact have the required shape?
Voice or domain contractDoes it obey the system’s intended rules?
Verification contractCan its claims be traced to sources?
Deduplication checkDoes it recycle earlier material?
Audit trailCan a failure be reconstructed without rerunning the pipeline?

The checks establish whether the output may advance; the audit trail preserves the evidence needed to understand a failure. Together, they make readiness something the system must establish rather than a label the artifact merely contains.

Software delivery already distinguishes code existing from code being fit to deploy. Agent workflows need the same discipline for artifacts. Failure itself is expected; the dangerous behavior is formatting that failure convincingly and shipping it downstream because it looks complete.

Five cards labeled Output Contract, Voice Contract, Verification Contract, Dedup Check and Audit Trail, above a warning against shipping artifacts merely because they look complete.Open full source frame
Five controls check output shape, voice, sources, duplication and auditability.
8:388:52
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:38 · section reference included

Put the first gate where failure costs most

Start with the workflow you already have:

  1. Map every handoff from input to final output. Every arrow is a possible point of corruption. Identifying them does not commit you to fixing all of them immediately.
  2. Choose the most expensive failure, not the most complex handoff. A false claim published publicly, a schema break cascading into downstream skills or a duplicate eroding audience trust can each determine where the first gate belongs.
  3. Make that gate block progress. A failed check must prevent the artifact from moving forward.

A warning-only check leaves the delivery decision unchanged. To function as a gate, it must stop the artifact from advancing; that is what separates a convincing demo from an operable system. Before adding another agent, add one boundary.

Slide titled “Before you add another agent, add one boundary,” with three takeaway cards: Map your handoffs, Pick the most expensive handoff, and Make it say no.Open full source frame
Map your handoffs, pick the most expensive one, and make its gate say no.
9:5210:04
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:52 · section reference included

Resources

Updates since the talk

Transcript download
Full transcript

Loading transcript…