Open Source · Loop Engineering Factory

From architecture to shipped software, in a governed loop

Standalone coding agents generate code fast — and stop explaining. A Loop Engineering Factory keeps a coding agent (it delegates BUILD to OpenHands) and wraps it in a LangGraph state machine that makes the process visible, controllable, and self-improving — so the architecture you designed becomes verified, shipped software, on your own infrastructure, with a human still on the approval loop.

Why wrap the agent

Visibility and control a raw agent can't give you

The factory is not a rival to OpenHands. It is a governed, observable, pausable workflow around it — the difference is in the parts an agent doesn't have.

Dimension	Standalone agent	Loop Engineering Factory
Visibility	Streaming chat text; the outcome appears at the end	Every phase emits typed artifacts and audit records, streamed live to the dashboard
Control	One-shot or ad-hoc steering; no structured pause points	Deterministic human-in-the-loop interrupt() gates: review, reject with feedback, pause, or abort
State	Ephemeral — the agent's context dies with the session	AsyncSqliteSaver checkpoints every phase; a run resumes from any gate
Verification	The agent asserts it did the work	A deterministic VERIFY gate runs machine-checkable acceptance tests — never a "trust me"
Improvement	None across runs	REFLECT stores patterns in ChromaDB; the next cycle starts smarter

The pipeline

Nine phases, two kinds of gates

DISCOVER → DEFINE → PLAN → ARCH_REVIEW → BUILD → SEED_DATA → VERIFY → SHIP → REFLECT. Two human-in-the-loop pauses (DISCOVER, ARCH_REVIEW) and one deterministic gate (VERIFY) decide what ships.

DISCOVER

HIL ×2

Setup and structured interview. An ArcKit pre-scan auto-populates project fields from schema-validated artefacts when present; two human-in-the-loop pauses (project setup + interview).

DEFINE

Generates the specification and the API contract. Two LLM calls (source-driven + api-design) run in parallel via asyncio.gather(); ArcKit advisory context is appended to both.

PLAN

Implementation plan, doubt resolution, and four architecture diagrams. Diagram generation runs in parallel — one extra LLM call per use case.

ARCH_REVIEW

HIL ×1

A human reviews spec, plan, and diagrams. Approve → BUILD. Reject → PLAN, carrying the feedback forward.

BUILD

gates

Delegates to the OpenHands coding agent over the Gateway API; writes a machine-readable build_report.json manifest. Falls back to an inline LangGraph subgraph if the agent is unreachable.

SEED_DATA

Loads test fixtures. Model-driven when an ArcKit data model is present; otherwise a pass-through placeholder. No LLM on the model-driven path.

VERIFY

gate

The deterministic gate. Runs machine-checkable acceptance tests parsed from the spec; sets verify_status = pass or fail. LLM review text is advisory only.

SHIP

Deployment packaging and forward to REFLECT: observability, launch checklist, version tagging.

REFLECT

Self-improvement: records the cycle's metrics and learned patterns to ChromaDB and emits structured config/guardrail diffs for the next cycle.

BUILD phase

Delegating the build — with a floor, not a guess

BUILD hands off to an OpenHands coding agent over the Gateway API. The agent must hand back a machine-readable build_report.json manifest. A missing or malformed manifest is a hard fail — the factory never falls back to guessing from free-text. If the agent is unreachable or times out, an inline LangGraph subgraph takes over instead.

Step	Implementation	Note
Health check	GET /health	Any failure routes to the inline fallback
Create conversation	POST /api/conversations	OpenHands Gateway API (container :8000, host :43005)
Poll	GET /api/conversations/{id}	5s interval, 1h timeout; on timeout → inline fallback
Parse manifest	build_report.json	Derives status pass / partial / fail; a missing or invalid manifest is a hard fail, not a guess
Quality gates	route_phase() in edges.py	Security findings, UAT pass rate, review revisions decide the route forward

The retry budget is bounded — at most two BUILD → BUILD loops before the run halts to ERROR. It never "gives up" by skipping to SHIP.

The gate that decides

VERIFY is deterministic — it never ships on a "trust me"

VERIFY is the quality gate, and it runs machine-checkable acceptance tests parsed from the specification — not prose a model wrote about itself. LLM review is advisory; the decision is made by the tests.

Pass → SHIP

Every acceptance check passes → the run advances to SHIP and REFLECT.

Fail → BUILD (bounded)

A failed check loops back to BUILD, carrying the failing test ids. Up to two loops.

Budget exhausted → ERROR

Past the loop budget the run halts to ERROR. It does not quietly ship broken code.

Self-improvement

REFLECT makes each cycle start smarter

REFLECT records the cycle's metrics and learned patterns to ChromaDB and emits structured config and guardrail diffs for the next run. Low-risk changes auto-apply; anything touching security requires a human. The loop learns; it doesn't just repeat.

Guardrails

Quality thresholds enforced per phase

min_spec_confidence ≥ 0.9 DEFINE
max_arch_uncertainty ≤ 0.8 PLAN
max_security_findings = 0 BUILD
max_review_revisions ≤ 2 BUILD
min_uat_pass_rate ≥ 0.95 VERIFY

Security-sensitive keywords (auth, payment, credential, secret, …) force a human approval — see config/guardrails.yaml.

Trust by construction

What happens when the LLM fails

A self-improving loop is only as good as its failure handling. LEF treats LLM errors as typed, bounded, and non-fatal to the workflow.

Typed errors, no sentinels

LLM failures raise LLMError (fatal) or _LLMTimeout (retryable) — never a magic string that leaks into state.

Bounded exponential backoff

Transient 5xx, timeout, and connection errors retry with backoff (base 1.0s, cap 15s, up to 2 retries). 401/403/404 and model-not-fault do not.

Graceful degradation

When a fatal LLM error is exhausted, the call returns None — and active-path nodes coerce None to an empty result with a warning instead of crashing the workflow.

Component overview

Where each part lives

Component	Responsibility	Config key
main.py	CLI entry — headless auto-approve	workflow.auto_approve
frontend/backend/app.py	FastAPI backend — workflow API :48011	services.loop_api.*
graph/main.py	LangGraph StateGraph definition (9 phases)	workflow.hil_mode
graph/edges.py	Conditional routing via route_phase() — gates, loop limits, forward paths	—
graph/executor.py	WorkflowRunner — orchestrates execution with HIL pauses	—
graph/checkpointer.py	AsyncSqliteSaver — resume a run from any gate	—
tools/llm.py	LLM dispatch with bounded retry & context compression	services.llm.*
tools/loader.py	Skill registry discovery & hot-reload	workflow.skill_registry_path
feedback/chroma_client.py	ChromaDB pattern storage / retrieval	services.chroma.*
service/evaluator.py	LLM-as-judge scoring of phase outputs → Phoenix UI	services.otel.*

All external parameters are centralized under a three-tier priority — environment variables, then config/config.yaml, then built-in defaults. Zero hardcoded URLs or ports in production code.

Quick start

Three ways to run it

One Docker Compose stack, an OpenAI-compatible LLM endpoint, and a coding agent. Roughly 25–40 LLM calls per full cycle.

CLI (headless, auto-approve)

# Build & start the full pipeline, no human intervention
docker compose up -d --build loop
docker compose logs -f loop
curl http://localhost:48081/health

Web UI (human-in-the-loop)

# Same stack, interactive dashboard + HIL gates
docker compose up -d --build loop
# Frontend  → http://localhost:4080
# API       → http://localhost:48011
# Health    → http://localhost:48081

Point it at your LLM

# Any OpenAI-compatible endpoint (e.g. a local SGLang model)
export LLM_BASE_URL="http://host.docker.internal:8080/v1"
export LLM_MODEL="Qwen3.8-27B"
export LOG_LEVEL="info"

A build loop you can read, pause, and prove

The Loop Engineering Factory is open source — the state machine, the gates, the agent hand-off, and the self-improvement loop are all inspectable. No vendor lock-in, runs on your own infrastructure.

Loop Engineering Factory on GitHub Executable enterprise architecture →