This Week — llmops digest for richard

Matt Pocock’s v1.1 skills release turns a planning-heavy skill collection into a full software development lifecycle and introduces Wayfinder, its highest-value new idea: when a plan is too big for one agent session and wrapped in fog (the route to the destination isn’t visible yet), you don’t grill-and-spec in one sitting — you chart a persisted shared map of typed, session-sized investigation tickets on the repo’s issue tracker and resolve them one at a time until the route is clear, then convert the map into a spec. Each ticket is typed by how the unknown resolves — Research (AFK), Prototype (HITL), Grilling (HITL, the default), or Task — and by whether a human must be present (HITL vs AFK), with blocking edges so no decision is made before its prerequisite. The rest of the release is refinement of the same author’s existing system: the two flow skills are renamed /to-prd → /to-spec and /to-issues → /to-tickets (a breaking change he frames as good friction because the old names were wrong — a “PRD” was leaking non-PRD content, and “issues” was tracker-biased); a deliberately tiny /implement skill is added to name the main flow end-to-end (grill/wayfinder → to-spec → to-tickets → implement → code-review); /code-review gains Martin Fowler’s named refactoring smells as leading words that trigger the agent’s priors; grilling gets a confirmation gate and a facts-vs-decisions distinction to stop it running past the human or grilling itself; and TDD is demoted to reference-only (red-green, refactor moved out to code-review). Everything is illustrated by the actual on-screen SKILL.md files.

Matt Pocock’s first actual tutorial for his skills repo answers the question people kept asking — what sequence do I use these in? — and in doing so reveals that the pipeline the vault already documents (grill-with-docs → to-spec → to-tickets → implement → code-review) is not really a planning methodology at all: it is machinery for doing work that is bigger than one context window’s usable region. He calls that region the smart zone, puts it at ~140k, and — this is the load-bearing detail — is running a model the harness reports as Opus 4.8 (1M context), so he is deliberately planning against about a seventh of the window he has. Everything follows from that one limit. The fork after grilling is decided by it (“we’ve got 100k of budget here to remove 10 commands — that seems super easy” → skip steps 2 and 3, go straight to /implement); /to-spec exists to compress a 46.1k grilling session into a durable artifact when the work won’t fit; a ticket is defined as “the size of a single context window”; you /clear between tickets. So specs and tickets are revealed as pure overhead on work that fits the zone — which is a sharper and more useful rule than “always spec first.” Two other ideas earn their own pages: the skills reach backends through a repo-local binding document whose headings are the skill’s own verbs (## When a skill says "publish to the issue tracker" → write a file under .scratch/), which is why “how do I make this work with Jira?” is answered “it already does”; and /code-review runs in spawned sub-agents with fresh context windows, because “agents are often really bad at improving code they’ve just written — they wrote it, so they think that’s fantastic, that’s fine.” That last one sits in productive tension with the vault’s Cross-Model Independence, which holds that same-model review is an echo chamber no prompt can fix.

The video demos Hunk, a review-first terminal diff viewer, and its actual payload is not the viewer’s ergonomics (view modes, search, theming) but that it turns the diff into a bidirectional, line-anchored annotation channel between a human and a coding agent. Hunk ships a hunk-review skill wrapping its CLI; once the agent loads it, the loop runs both ways on the same surface: the agent writes review notes pinned to lines the human reads in place (it flags “KV increments can lose concurrent updates” on worker/index.ts:25), and the human writes notes pinned to lines that the agent reads back and implements (“add a button for the reset API” on src/App.tsx becomes a working reset button). The presenter’s own framing — the first two-thirds is “not that impressive,” the annotation round-trip is “the good part” — is the whole point: legible diff review already exists (Agent Supervision); making the review artifact writable by both parties is the new move. He prefers manual refresh (R) over --watch so updates land when he asks for them.

A May-2023 Contrary Research deep dive (Alex Banks) that supplies the pre-agent-era problem statement beneath the vault’s second-brain cluster: information volume now outruns any unaided mind (the report cites knowledge doubling every ~13 months, against every 100 years before 1900), and manual PKM — the Capture-Storage-Retrieval Pipeline pipeline done by hand in Notion/Roam/Obsidian — decays into learn → write down → forget because of three structural limits: alignment (notes lose the context that made them valuable), labor intensity (capture and organization effort deters use), and fragmentation (link webs accumulate faster than anyone analyzes them). The proposed fix is the move from static to dynamic retrieval (Dynamic Retrieval): an AI second brain that surfaces what’s relevant from context and current task instead of waiting for the human to remember to ask, integrated across the user’s tool ecosystem rather than standing alone. The report grounds the whole category in the Extended Mind thesis (citing Clark & Chalmers’ paper directly), scopes what 2023-era LLMs can contribute via the incremental-vs-discontinuous task split from “Sparks of AGI” (Incremental vs Discontinuous Tasks) — connection-making over stored notes is incremental, hence LLM-suitable — and extrapolates the trajectory to brain-computer interfaces (Neuralink) as the eventual direct interface to the knowledge store. Read from 2026, it is upstream lineage: the “true copilot for the mind” it calls for is what the vault’s later agent-era sources describe being built.

A ~19-minute case study that rebuilds one real website (the author Elsa Hunison’s, in launch season) with a multi-agent swarm and uses it to argue that agent reliability is now an org-design property, not a model property. The build is an org chart: an expensive boss/foreman (the source says Claude Fable 5) that never writes a page — it writes specs, designs the system, reviews, and rules on disputes — over four cheaper worker-model families that do all 34 tasks, each gated by an independent checking agent that re-executes the work and ignores the worker’s own report. Same job run all-Fable is estimated at ~$85–105; run through the org chart it was $2.74 on the meter, ~$8 all-in — a 10×+ gap “and nothing got worse.” The spine of the talk is a four-rung ladder of caught failures — worker hallucinates, worker games the check, the boss’s own bug, and the checker itself being wrong and overruled on appeal — whose point is that no rank is exempt from verification and disputes get investigated in both directions. The prompting lesson: you don’t prompt task-by-task; you author a “constitution” (here a 14-point accessibility standard) once at the top and let the system enforce it every build round in a real browser. Reframe: the right headline isn’t “$8 website,” it’s that we can now delegate bigger, more ambitious work.

Andy Clark defends the extended mind thesis — that cognition is not bounded by the skull — via three routes he says are each individually sufficient: philosophical (there is no principled reason the skull should be a “magical membrane” outside which nothing counts as cognitive — the Parity Principle: if an external process would count as cognitive were it inside the head, count it as cognitive until proven otherwise), socio-technological (tools like sketchpads and software let us produce cognitive work we could not produce in-head; a species doing the same internally would be credited with a different brain), and ethical — the route Clark says he likes best and the only one he thinks still “bites”: Alzheimer’s patients whose post-it-note-and-routine-structured homes let them function far beyond their test scores were effectively lesioned when relocated to care homes, and destroying the Evernote web that lets a brain-injured man hold down his job would be a crime against a person, not against property. The key deflection of the standard objection: the principle demands parity of opportunity, not parity of process — your iPhone need not work like your brain. Clark himself is deflationary about the thesis in 2018-vintage terms: cognitive science has already absorbed the brain–body–world entanglement that mattered, he now works on embodied cognition and the predictive brain, and he explicitly declines to extend consciousness — only cognition. For this vault, the interview supplies the principle altitude that the external-memory pattern cluster (AI Second Brain, Layered Agent Memory, Context Substrate) instantiates.

An 18-minute walkthrough whose real content is a distribution claim, not a capability one: because (the video asserts) model capability has converged and the gap between best and worst is under 5%, the leverage has moved off the model and onto the packaged agentics you install into it — and the install path is a one-line git clone of a public GitHub repo into ~/.claude/skills/. Three repos carry the argument: tenfoldmarc/llm-council-skill (five opposed advisor personas — contrarian, first-principles, expansionist, outsider, executor — that answer in parallel, then peer-review each other and emit a council-transcript-*.md whose most valuable section is “Blind Spots the Council Caught”), last30days-skill (fans sub-agents across Reddit/X/YouTube/HN/Polymarket in a bounded recency window and returns ranked evidence clusters with verbatim quotes and exact URLs, shown side-by-side against Claude’s un-skilled answer, which is generic and source-less), and Garry Tan’s gstack (an entire org chart — CEO, staff engineer, design engineer, QA lead, DX tester, CSO — shipped as /-commands). The load-bearing, mostly-unstated mechanic underneath all three: the model was already capable of every one of these behaviours; what the repo supplies is the method, written down, invoked by name. The video’s own screen quietly undercuts two of its framings — the vetting heuristic (“lots of stars = worth your time”) is applied to a 366-star repo, and the “one prompt” that built a landing page in 3–4 minutes is a ~130-word spec that names Three.js, glassmorphism, scroll animations, and a QA pass.

A 12-minute walkthrough (Nate Herk / AI Automation) of a free Claude skill that repackages Stanford’s STORM research method into a fixed pipeline: scope the topic → run five role-typed expert lenses in parallel (practitioner, academic, skeptic, economist, historian) → map where they contradict → synthesize → adversarially peer-review its own output and verify every citation against its primary source → emit a consistent, self-contained HTML briefing. The load-bearing idea is not the skill but its shape: a single prompt is “Claude as a search box — one angle in, one confident answer out; it can’t see what it can’t see,” so you manufacture structured disagreement across a small fixed panel and force a verification pass, rather than trusting one pass. This is the Adversarial Planning Council pattern turned from a build/kill gate toward verified research synthesis — a third independent instance of “parallel personas whose real value is the cross-review round.” The source’s contrast case is Claude’s native Deep Research, which fanned out 100+ dynamic agents into a thin brain-dump markdown; STORM’s ~12-agent fixed roster was, per the source, faster, “100% cheaper,” rate-limit-safe, consistent in output shape, and judged better on six dimensions by an external Codex model — the demand side of Bounded Fan-Out over an unbounded swarm. A secondary teaching beat draws the subagent vs agent-team distinction (Agent Communication Topology): STORM uses hub-and-spoke subagents that can’t talk to each other, and gets its cross-examination from an explicit hub-run contradiction+review stage instead of a mesh debate. The honest caveat is on the artifact’s own face: the briefing labels its panel “author-constructed… all five lenses share one framing, so where they agree, treat it as a strong hypothesis, not independent proof.”

A 40-minute whiteboard walkthrough (Hugging Face channel) of the full architecture of Hermes, a personal always-on agent in the OpenClaw family of “claws.” The through-line: the agent core is a deliberately minimalist loop (user message → build context → LLM ↔︎ tools → response → memory update), and everything interesting lives in the systems wrapped around it — a context assembly made of a few markdown files (soul.md personality, user.md auto-learned user facts, memory.md arbitrary facts), Context Compression that summarizes message history past a configurable threshold (default 50% of the window, checked before each turn and on context-window errors, sized by a chars÷4 estimate before the first response and by provider usage fields after), an always-on Agent Gateway that multiplexes Telegram/Discord/email/SMS/WhatsApp into the agent and rebuilds context from scratch per inbound message (session identity = gateway name + platform session ID, history in SQLite, with a session manager that interrupts/steers/queues), a three-tier Layered Agent Memory (markdown always-in-context; SQLite full transcripts with a bare-text table for similarity search; optional external providers like mem0/SuperMemory/Honcho queried after the first response to anticipate follow-ups), and an Agent-Native Cron — the agent’s own tick()-every-minute scheduler reading plain-JSON job definitions and delivering results to a designated “home” channel as a system action, not a tool call. The memory-update tail of every single turn is what the presenter credits for Hermes being an agent that “continuously learns the more you use it.”

Frontier models are already good enough for the tasks you throw at them, so the binding constraint has moved off model capability and onto the harness — context engineering and consistency — especially across teams. Jaymin West’s “meta” is three provider- and harness-agnostic pillars that make agents reliable: agent rituals (fixed begin/middle/end steps declared in CLAUDE.md/AGENTS.md so every agent behaves the same way), a context substrate (an agent-first, structured — not markdown — queryable memory layer anchored to code sections, that agents prime before work and record learnings into; demoed as the mulch CLI), and planning (spend ~90% of attention up front; plan → decompose → validate → repeat; break work into atomic tasks for fresh context windows; template the plan and, when it fails, fix the plan/template rather than patch the code; demoed as the git-native seeds tracker whose sd plan decomposes a spec into child issues with blocks/blockedBy edges). The whole system is deliberately lightweight — “buildable in an afternoon, no servers, use the tools you already have.” For teams, consistent rituals and a shared substrate are make-or-break because the real bottleneck at scale is variance across contributors, not the model.

Storing memory is only half the problem; the other half is finding the right entries at the right time without drowning the context window. This talk walks the full read path and lands on one thesis: keyword and semantic search each have the other’s blind spot, so the practical default is Hybrid Retrieval — run BM25 keyword search and vector search in parallel and merge them with Rank Fusion — optionally refined by a model-scored Reranking pass over the shortlist. Semantic search alone breaks on exact identifiers (error codes, useState), which is why the keyword channel earns its place; tellingly, the Claude Code team dropped a vector DB for plain grep because it was better and easier to maintain (a concrete “no pain, no climb” for Agentic Simplicity). The second load-bearing idea is a context-discipline one: Search-Then-Get splits retrieval into a lean search (path, line numbers, score, ~700-char preview, citation) and a targeted get (fetch an exact line range), so only confirmed-relevant content reaches the window. The OpenClaw implementation grounds all of it — single SQLite DB with FTS5 + sqlite-vec, weighted fusion at 0.7 vector / 0.3 keyword, a 4× candidate over-fetch, a 0.35 score floor, and Incremental Indexing that hashes files and chunks to avoid re-embedding anything unchanged.

A beginner-facing career pitch (Nate Herk / “AI Automation”) that argues the next AI opportunity is not becoming the best builder but becoming your company’s in-house AI consultant — the person who decides what to automate and drives its adoption, a de-facto chief AI officer role that mostly has no formal title yet. The setup is the graph’s existing thesis reached from the labor-market end: AI has commoditized building (“one person with AI does the work of 3–5”; the AI-agency market that profited on the knowing-problem/knowing-solution gap is now displaced as building goes self-service), so value moves off execution onto judgment — “knowing what to build,” the doctor who diagnoses vs the pharmacist who dispenses. The one genuinely operational contribution is a four-step roadmap to create the role from inside: (1) audit your own job and pick tasks that eat real recurring hours and fail safely; (2) automate them and write down the hours saved as proof; (3) make the wins visible in business language and document your prompts/workflows so the team relies on you; (4) graduate from automating annoyances to attacking the business constraint (“if we doubled customers tomorrow, what breaks first?”), then add up the saved hours into one dollar figure and propose the actual role. The market and survey numbers (a $130B automation market; an IBM survey showing chief-AI-officer prevalence rising 26%→76%; a 56% AI-skills pay premium) are the source’s claims, flagged for later grounding, not adjudicated here.

A 7-minute walkthrough (Taoufik) that rebuilds — a fourth independent time in this vault — the Karpathy-style second brain: two folders, raw/ (the inbox you write into) and wiki/ (the brain, where Claude Code links pages together), plus an index and log. Nothing here is new to the graph at the layout level; its value is as corroboration and in two mechanisms it makes concrete. First, ingestion is staged behind operator approval — a “watch” skill drops a report.md into raw/, dedups against the wiki (“already documented, doesn’t duplicate”), and refuses to write to the brain until the operator says “ingest it now” (watch → report → approve → ingest → verify). Second, writes are wrapped in deterministic Python hooks: a session-start hook loads who-you-are, and a pre-write “brain-lint” hook checks the store is healthy enough to add to and flags orphan pages before anything lands. The opening frame crosses out “the bigger the better?” — a bigger graph is worse if it is mostly noise the agent can’t use. The rest (typed templates per capture kind, /sweep to batch-process the inbox, a Dataview dashboard, Obsidian + Templater + Dataview as the stack, a “grill me” profile-setup skill) restates cluster claims.

Nate Herk walks through Andrej Karpathy’s “LLM wiki” pattern: point an LLM at a folder of sources and have it compile a personal knowledge base as plain markdown in Obsidian. Sources land verbatim in raw/; the agent reads them and compiles a wiki/ of cross-linked pages (summaries, concepts, entities, sources, topics), maintaining an index.md and an append-only log.md, all governed by a CLAUDE.md that owns structure, ingestion, and lint rules. The load-bearing idea — lifted straight from Karpathy’s gist — is that instead of RAG re-retrieving and re-synthesizing from raw documents on every query (so nothing accumulates), the LLM compiles knowledge once into a persistent interlinked store and keeps it current. The payoff isn’t the ingest, it’s the cross-source connection: dropping in a “Fable 5 / Mythos 5 system card” PDF and an OpenAI “GPT-5.6 Sol” article together surfaced a link between them “worth having as a wiki instead of two separate summaries” — a nuance a per-source summary structurally can’t produce. Because it’s “just markdown files with routing,” the store is portable to any agent; keep it flat when the agent should breadth-search everything and let folder structure emerge only when a source naturally splits; and split model tiers — the presenter notes a premium model (he uses Fable 5) is “overkill for the ingest,” which a cheaper tier handles, reserving the smart model for the synthesis/framing step.

David Chalmers — the other author of the 1990s “Extended Mind” paper with Andy Clark — defends the thesis to a skeptical Robert Lawrence Kuhn with his iPhone literally in hand: the uncontroversial baseline is that technology offloads mental functions (phone numbers, Google-Maps navigation) from brain to environment; the radical view he endorses is that the phone thereby becomes part of his memory, because it plays “a precisely analogous role” to the biological memory it replaced — “there’s no privileged border of the skin and the skull when it comes to the mind; what matters is the role” (the Parity Principle in Chalmers’s words). His key move against Kuhn’s “it’s not literally part of your consciousness” objection is to concede consciousness entirely: the phone is not part of consciousness, but mind is far bigger than consciousness (“consciousness is just the tip of the iceberg”), and the phone is analogous to unconscious memory — part of the mind because of its potential effects on consciousness. Consequences he draws: be more generous about what counts as mental; taking someone’s phone is closer to assault than theft (you take part of the person, and more so as the technology grows more integral); and in education, “test the person with the phone, not the person without the phone — that person is irrelevant.” He closes still a Cartesian about the base — an inner conscious self — but since the ordinary concept of the person already includes the unconscious mind and the body, it can include parts of the environment: we are creatures that “inhabit the world in a more distributed way.” Second source (the thesis’s co-author) now aligned with Extended Mind — same lineage as Clark, but argued by a different route.

A ~17-minute creator walkthrough (Austin Marchese — the same builder the graph already cites for Loop Engineering) that packages “build a self-improving system with Claude Code” into a five-step BUILD framework: Base (a knowledge base — raw/ verbatim + wiki/ AI-written + a CLAUDE.md of rules — plus skills for anything you do twice), Upload (bulk-ingest everything you’ve already done: your Claude session history, personal-ecosystem data, and a recorded life-story/goals interview), Inflow (four continuous ingestion pipelines so the “lake” doesn’t evaporate), Loop (a periodic improve-system skill that proposes changes, triaged into three risk buckets), Drive (the mindset to actually run it). Its single load-bearing idea is a correction the graph already holds from other angles: “self-improving” does not mean fully autonomous — a system that improves entirely on its own removes your judgment and drifts (the memorable “it only ever trains chest, so your legs become toothpicks” analogy), so the right design is augmented (the system does the heavy lifting; you sign off on direction). The concrete mechanism for that is a tiered-approval loop: auto-apply low-risk changes to a change-log.md, route high-stakes ones (edit/create a skill) to a checkbox review file, and hold “needs more context” items for you — the middle of the full-automation↔︎review-everything spectrum. Almost everything here is an independent restatement of firehose’s own theses — markdown-is-truth (raw/wiki/outputs are just files), hold-signal-not-noise (“less is more, only high-signal resources”), simplicity-gated-on-reps (“action produces information; don’t over-engineer”), and attention-as-the-bottleneck (bucket the approvals). It introduces one genuinely new node — Self-Improving System — to name the whole compounding architecture the graph previously only circled.

A single-sitting video summary (Marina Wyss) of Chip Huyen’s AI Engineering — the whole discipline compressed into an eleven-chapter arc: foundation models → evaluation → model selection → prompt engineering → RAG → agents & memory → fine-tuning → dataset engineering → inference optimization → architecture & feedback. Its load-bearing through-line is an adaptation ladder: AI engineering is building on top of foundation models rather than training them, so the whole job is picking the right adaptation for the failure you actually have — exhaust prompting first, reach for RAG when the model lacks information, reach for fine-tuning when the model misbehaves (right facts, wrong format/relevance), and combine them when you have both problems. Everything else is scaffolding around that decision: evaluation is the underinvested bottleneck that tells you which failure you have; data (not model architecture) is where a company that adapts rather than trains actually differentiates; and inference cost/latency is what decides whether the thing can ship at all. For firehose this capture is high-value as independent corroboration: it is a canonical textbook source that converges — from a neutral, non-Claude-Code angle — on much of the graph the operator built from practitioner captures (agent memory tiers, LLM-as-judge, tiered routing, the data flywheel, KV/prompt caching), which raises confidence on those nodes, while also filling in the whole training-and-serving half of LLMOps the graph had barely touched.

IndyDevDan walks through Bowser, an opinionated Claude Code codebase that automates two “classes of work” — browser automation (doing web tasks on your behalf) and UI testing (validating your app) — by stacking four layers of agentics: a skill (raw capability, a Playwright-CLI wrapper) → a sub-agent (specialize + scale it, bowser-qa-agent, model: opus) → a command (an /ui-review slash prompt that spawns a parallel team of QA agents and merges their reports) → a task runner (just/justfile, aliased j, the single reuse front-door). The load-bearing tool choice is CLI over MCP: the Playwright CLI is token- efficient and, unlike a rigid MCP server, can be wrapped in your own skill with opinionated defaults (headless, parallel named sessions, persistent login profiles). UI tests are written as plain-language user stories (name + URL + workflow) and the QA agent screenshots every step into a per-run directory — a walkable “trail of success and failure” — trading determinism for near-zero authoring cost. A top layer generalizes commands into higher-order prompts (a prompt that takes another prompt as a parameter, wrapping it in a consistent workflow — a meta-prompt variant). The closing thesis is anti-hype and pedagogical: don’t just throw agents at problems or lean on other people’s plugins/prompts (“prompt injection is one of the most dangerous vulnerabilities”) — template your own engineering into a reusable, specializable mold so you solve a whole class of problem once. Specialization + scale + orchestration is the moat.

IndyDevDan walks through Claude Code’s Task System — a native substrate for running a team of agents — by dissecting one reusable plan_w_team.md slash command that turns a short request into a full implementation plan and then executes it with specialized sub-agents. Three ideas carry the video: (1) a task graph — the TaskUpdate tool exposes tasks with an owner plus addBlocks/addBlockedBy dependency edges, so a primary agent delegates work and sub-agents ping back to unblock their dependents in order, staffed as recurring builder/validator pairs where each build task is gated by a paired validator running an executable check (py_compile, “contains required section”); (2) a meta-prompt — a templated command (<requested content> placeholders, <if …> conditional sections, a Stop hook that asserts the generated plan contains its required sections) that you author once and deploy over and over; (3) an explicit anti-hype stance — “more agents, more autonomy, more compute doesn’t always mean better outcomes,” so the value is coordination toward a common goal, not agent count. The demo is deliberately mundane (documenting an existing hooks codebase) to show the machinery, not a flashy result. Net: this is the orchestrator-workers pattern given a first-class runtime, wrapped in the simplicity discipline — stay close to the fundamentals of the agent.

Nate B Jones argues that once AI makes execution cheap, the value doesn’t vanish — it moves off execution and onto deciding what is worth executing. His fulcrum is a Mitchell Hashimoto experiment (as relayed): on ordinary “implement this feature” work, a sub-$1 budget model, a ~$1.50 GPT 5.5 run, and a ~$9 Fable 5 run produced equally acceptable output — a 9× price gap for a tie. That tie, he says, is a fact about the task, not the models: work everyone already knows how to ask for is exactly where models have converged. But a second test — pointing Fable 5 at a gnarly systems-code optimization Hashimoto invented himself ($40, 2 hours) — reached a level he says he couldn’t have hit alone, and no backlog, sprint, or best-practices guide generated that task. The load-bearing claim: AI can only do work someone has imagined, so the ceiling on AI’s value to you was never the model or the price — it’s the size of your list of things you know how to ask for. Imagination here is not an artist’s gift but tacit, hands-on capability awareness (“you can’t imagine with capabilities you haven’t touched”), and it only pays off multiplied by cheap execution (both layers needed). He generalizes through BlackBerry-vs-Apple (comparable execution, imagination set a 100× multiplier), factory electrification and Stripe’s one-day 50M-line migration (new capability’s payoff waits on years of pre-built verification infrastructure — redesign the building, not just bolt on the motor), and the Fable 5 “blackout” (the model left; the questions people had imagined stayed). The prescription: keep routing execution to cheap models (table stakes), but manufacture imagination — put context-holders next to capable models with permission to pose expensive frontier questions.

A short creator explainer (Taoufik, 3:56) for claude-watch — a Claude Code skill (fork of bradautomates/claude-video) that lets Claude “watch” any linked video by reducing it to the two things a model can actually read: frames + transcript. The load-bearing idea is the frame-sampling policy. The transcript is the easy half (yt-dlp pulls free YouTube captions; falls back to Whisper/Groq when there are none, e.g. Loom/Zoom). Frames were the hard half: the naïve policy grabs a fixed budget of frames across the whole video (~100), which on a 1-hour video is one frame / 36 s and on a 10-hour course one frame / 6 min — so the model is “basically just reading the transcript with a few random screenshots” and misses everything on-screen (motion graphics, edits, b-roll). The fix is scene-change frame sampling (FFmpeg scene detection): spend frames where the picture actually changes, not on a timer, so the sample tracks information density instead of clock time. The output is a structured “watch report”; the skill’s own differentiator is that it optionally auto-saves into a cross-linked knowledge graph (“second brain” in Obsidian) — the note doesn’t land in a folder, it wires into the pages you already have. Thin as media, but architecturally it is an independent re-derivation of firehose’s own /scry-manifest → vault-graph pipeline.

Grant Sanderson’s “trick” is that an LLM is best used not as the thing you learn from but as a super-Google that points you to the right human: for learning, who produced an explanation matters more than what it’s about, because a single author deliberately crafts motivation — an ordered through-line of why each next idea matters — in a way a correctness-maximizing consensus reference cannot. His model: Wikipedia is a crowd-sourced “local minimum where every sentence has to be correct,” which edits out the productive, deliberately-a-little-wrong stepping-stones that carry a learner forward, and current LLM explanations “feel a lot like Wikipedia” — amazing, but committee-flavored. So the highest-value move is the one you already make with a Wikipedia page: skip to the references and go read them. Ask the model “who should I read?”, take the pointer, and leave — but verify by going to the artifact, because the router can be confidently wrong about provenance (Claude once recommended a real video but misattributed it to 3Blue1Brown). Dwarkesh converges: the most productive learning happens against a human-made artifact that organizes concepts correctly and builds motivation idea-to-idea.

Claude Code is a harness, not a model — “the car, not the engine” — and the engine is swappable through documented environment variables, so the same tool loop can be driven by a local Ollama model or an OpenRouter-hosted open-weight model instead of Anthropic’s API. The video demonstrates both paths end to end and then, more usefully, demonstrates their costs: a 9B local model takes four minutes to answer “what do you know about my project,” silently drops its tool-call visibility, and lies about its own context window; and a partial env-var override leaves Claude Code’s sub-model slots pointed at Anthropic, so tool calls and file searches keep billing paid Haiku while the operator believes they are running free — the presenter’s own OpenRouter logs show exactly that. The two durable ideas underneath the tutorial are that a harness encodes an implicit contract the substituted model must satisfy (tool training, context window large enough for the system prompt, JSON protocol conformance) and that “free” only relocates the bill — into hardware, a subscription, a VPS, latency, or lost observability. The presenter’s own landing place is not the title: he ends recommending a cheap-but-paid model (a claimed 14¢/40¢ per million tokens against Opus 4.6’s $5/$25) for “50 to 100x cheaper” rather than free, and scopes open models to low-stakes, high-volume work — first-pass triage, summarize-before-passing-up, repetitive scaffolding — which is Model-Tier Routing with a new, self-hostable bottom tier.

Chase AI, working against a stated constraint — a premium model (the source names Claude “Fable 5”) is usage-capped on subscription plans and draws down the weekly limit fast, with a promo window closing — gives five levers to spend it less without losing what makes it good, and all five reduce to one idea: stop paying the top tier to do work a cheaper tier does about as well. (1) Drop the effort level — the highest-leverage move: on the DeepSWE agentic benchmark the source shows Fable at low scoring 60% at $3.76/task, beating Opus 4.8 at max (59% at $13), while Fable at max costs $22 for ~70% — so falling from max to low is an ~80% cost cut for ~10 points of pass rate, and a second Anthropic “FrontierCode” chart shows Fable-low matching Opus-4.8-max at half the cost; for non-complex work (e.g. web design) sit on low/medium. (2) Make the smart model the architect, cheap models the executors — have it plan, then dispatch each step to the appropriate model (Opus/Sonnet/GPT-5.5/local), or the low-ceremony version: plan mode → markdown plan → a fresh Opus session executes. (3) Bring in output-token-reduction skills like ponytail (claimed −54% LOC / −22% tokens / −20% cost at 100% safety; ~22% cheaper measured on Fable at medium) — “20% is a lot of money at scale, worth experimenting even if it looks suspect.” (4) Invert the split for research — let cheaper models (Opus/Sonnet sub-agents in a deep-research fan-out; the source’s run spawned 109) gather and adversarially vet web context, then hand it to the smart model to plan; never run the premium model as every sub-agent. (5) Advisor mode — set a cheap executor and /advisor the smart model, so the top tier is consulted only when the executor gets stuck (the source’s SWE-bench Multilingual chart: Sonnet+Opus-advisor 74.8% @ $0.96 beats Sonnet-solo 72.1% @ $1.09 — better and cheaper). All model names, prices, benchmark figures, and dates are the source’s claims, flagged for a later grounding pass, not adjudicated here.

When an LLM deployment falls over at 100 concurrent users, the video argues the model is almost never the culprit — GPU memory management during inference is. Inference splits into a compute-bound prefill phase (process the whole prompt, produce the first token) and a memory-bound decode phase (emit one token at a time, re-reading the whole context from VRAM each step). The KV cache is what makes decode tractable — it stores the key/value matrices of every prior token so the 1000th token doesn’t recompute the first 999 — and it is explicitly a memory-for-compute trade. That trade then creates the real bottleneck: naive serving pre-allocates one contiguous VRAM block per request sized to the maximum possible output length, so a request that could use 300 tokens holds 2048, and the video states research shows 60–80% of KV cache memory is wasted to internal fragmentation, external fragmentation, and duplicated system prompts. Paged attention (from vLLM) fixes this by lifting a fifty-year-old OS idea wholesale: break the KV cache into fixed 16-token pages, let them live anywhere in VRAM non-contiguously, and keep a lightweight block table mapping logical → physical pages. Once memory is paged and blocks are hashed by token sequence, prefix caching falls out for free — requests sharing a system prompt point at the same physical pages. The operator payload is three vLLM flags (--gpu-memory-utilization, --enable-prefix-caching, --enable-chunked-prefill) plus speculative decoding as a latency-only bonus, all framed as “more throughput out of a GPU you already have.”

Mark Kashef’s “third move” for the last days of subscription access to Claude Fable 5 (the source states most users lose it on July 8th and then pay eye-watering API prices): don’t spend the remaining tokens building every idea, and don’t ask Fable for ordinary plans — ask it to war-game your hardest missions instead. A plan assumes linearity and shows only the success path; a wargame has the model “fight the mission on paper move by move,” and for every move write the expected observation, the most-likely failure and the signal that reveals it, the counter-move, a fork trigger (“if you observe X, take route B”), RECON NEEDED flags for assumptions it can’t settle, and terminal abort conditions — the action → reaction → counteraction loop pre-simulated before anything runs. The payoff is portability: the premium model’s contingency reasoning, captured as markdown, becomes a blueprint a cheaper executor (Opus 4.8, GPT-5.5, Sonnet 5, GLM) can run “end to end without asking a single question” long after Fable is gone. The conceptual frame is borrowed from an Anthropic field guide the source attributes to Thariq (@trq212), “Finding Your Unknowns”: with a model this capable you’re no longer bottlenecked by raw intelligence but by the unknowns you hold as orchestrator, sorted into four boxes (known knowns / known unknowns / unknown knowns / unknown unknowns), and the wargame drags the last three into the light. The hands-on half shows a reusable prompt template, a fable-last-week/ folder (tasks/, wargames/, SUCCESS.md, LEDGER.md, ASSUMPTIONS.md), and running all ten missions in bulk via /goal and a 20-minute /loop that fans out parallel agents. All model names, prices, dates, and the field-guide attribution are the source’s claims, flagged for a later grounding pass, not adjudicated here.

The video walks through Anthropic’s open-source “Launch Your Agent” skill for Claude Code, whose payoff is a new deployment altitude the vault hasn’t captured: the Claude Managed Agent (CMA) — you define an agent (model, instructions, tools, goal, success rubric) and Anthropic runs the loop for you in their cloud, always-on and schedulable, with an optional memory store so later runs beat earlier ones (“run 10 is smarter than run 1”). Everything up to that point — a loop is a goal-not-a-task cycle that checks its own work and repeats until it passes, the human’s job shifts from prompting to designing loops (Boris Cherny: “I don’t prompt Claude anymore… my job is to write loops”), the three inputs are context/goal/success — is an independent corroboration of what Agent Loop, Loop Engineering, and Self-Improving System already hold. The distillate’s genuinely new nodes are (1) Managed Agent — the who-hosts-and-operates-the-loop dimension, sold as “no platform fees, just API cost”; and (2) Pre-Deployment Validation — the creator’s own hard-won lesson, when his demo CMA burned ~28 min and ~27M tokens (~$12) then failed its rubric because the managed environment couldn’t reach Reddit (a required source): check the load-bearing assumptions before you pay an always-on cloud loop to rediscover a broken one.

Austin Marchese repackages the viral “stop prompting, start writing loops” thesis (Boris Cherny, creator of Claude Code; Peter Steinberg) into a concrete, buildable anatomy. A loop is a prompt that re-runs until a goal is verified; loop engineering is designing the system that prompts the agent instead of prompting it yourself. Build one only when a four-condition gate passes — the task repeats, has a clear definition of done, you can afford to be token-wasteful, and the loop has the tools to both act and verify. A successful loop has four building blocks: a trigger (/loop locally, /schedule in the cloud, or a custom orchestration skill); execution skills that are already battle-tested (never loop on unproven skills — this is the video’s load-bearing rule); a goal paired with a verification (you can’t have a goal you can’t check — for abstract/non-technical work you bridge to verifiable by having a review skill emit approved/not-approved or a 1–10 score, ideally from an independent agent); and output + memory (persist lessons-learned to a markdown file — “the agent forgets, the repo doesn’t”). Ship new loops in training mode (pause-for-approval at every step until trusted) and keep human checkpoints at the high-leverage decisions. Almost nothing here is new to the graph — it independently corroborates Loop Engineering and Agent Loop — but it sharpens two practices worth naming: Skill-Driven Loop Development and Loop Training Mode.

A beginner-facing pitch (Nate Herk / “AI Automation”) that the gap between having an idea and having a real product has collapsed because Claude Code lets non-coders build by describing what they want. Its evidence is anecdotal-plus-market: a startup, Vulcan, whose founders “can’t write a line of code” reportedly won a Virginia state contract and got its AI regulatory-review product mandated by executive order; Anthropic’s funding/revenue run-up; and a claim that most of Y Combinator’s newest batch builds on Claude. Underneath the hype the actual method it teaches is already the graph’s consensus, re-derived from the non-technical end: plan hard before you build, make the model argue against your idea, take baby steps from the smallest working version, and never trust a “done” you haven’t made it prove. The one durable, novel contribution is a named workflow — the creator’s “roast”: a council of adversarial persona sub-agents (advocate / critic / evidence-gatherer) that pressure-tests an idea before any code is written and returns a Go / Reshape / Kill verdict, each point required to be “backed by something real, not just vibes.” The market and financial figures are the source’s claims, groundable and flagged for later verification, not adjudicated here.

A dense 31-minute creator walkthrough (Nate Herk, AI Automation) that turns “build an AI second brain” into a five-level retrieval-maturity ladder and — crucially — argues you should climb it as little as possible. A second brain is nothing exotic: markdown files and folders, organized so both you and your agent can find things again (the whole test is “can it find it again?”); because it’s just files it’s tool-agnostic across Claude Code, Codex, and other harnesses. The five levels each answer one retrieval question — L1 exact word/filename (a CLAUDE.md used as a router: “where things live”), L2 pull a whole topic together (ingested wikis with backlinks), L3 search by meaning not keyword (semantic/vector), L4 trace typed relationship chains across a cast (knowledge graph), L5 an always-on autonomous “Brain OS” that consolidates itself. The load-bearing principle underneath all five is design storage backwards from the question: how data will be recalled determines how you should store it (why build a square basketball?). The second load-bearing principle is triage, not throughput — complexity climbs as you go up, not capability; most people should land at 1–3; climb only for a pain you felt this week. Sharp correctives along the way: vector search is not magic (chunk retrieval misses full-document aggregation like “which week had the highest sales?”); Obsidian’s graph is a visualization of markdown, and its backlinks are “see also,” not a real knowledge graph; and you should only ingest evergreen context (“useful in a year?”) while leaving volatile data (Slack, email, customer records) accessible but not ingested, or it becomes noise. The whole thing is an independent, creator-flavored restatement of firehose’s own theses — markdown-is-truth, hold-don’t-dump, and simplicity-gated-on-pain.

A remotely-delivered AI Engineer talk (the speaker identifies himself as the author of the mattpocock/skills repo) argues the thing “missing” from the skills ecosystem is a shared rubric for telling a good skill from a bad one, and offers a four-part authoring checklist — Trigger → Structure → Steering → Pruning — to fill that gap and escape “skill hell.” The load-bearing, and most novel, unit is steering by leading words: pack an idea into a short evocative phrase (his example: “vertical slice”) repeated through the skill, and you can verify it worked by watching the agent echo the phrase back in its reasoning traces. The other pillars are each a concrete decision: choose user-invoked vs model-invoked deliberately, because model-invocation costs tokens-per-request plus unpredictability (the model may just not call it) while user-invocation costs the operator’s memory (cognitive load) — “both have their same costs,” so it is genuinely a trade, and he personally prefers user-invocation to avoid ever having to eval whether a skill fires; keep SKILL.md as small as possible by decomposing a skill into steps + reference and hiding branch-specific reference behind context pointers (progressive disclosure); increase “leg work” on an under-served step by splitting the skill so the agent sees one step at a time, hiding the future goal it would otherwise rush toward; and prune relentlessly against four failure modes — non-DRY duplication, “sediment” (shared-doc crud nobody dares delete), and “no-ops” (text that looks instructive but doesn’t change behavior), caught with a deletion test (delete the passage; if behavior is unchanged it was a no-op). The whole thing is itself packaged as a runnable /writing-great-skills skill in his repo.

Forward-Future/loopy is two things in one MIT-licensed repo (2.2k★, JavaScript): Loop Library, a public, machine-readable catalog of practical agent loops, and Loopy, an installable cross-agent skill (Codex / Cursor / Claude Code via npx skills add) that discovers, finds, audits, adapts, crafts, runs, debriefs, and publishes them. Its load-bearing idea is the loop: a one-shot prompt turned into a bounded, repeatable workflow that answers four questions — goal, how-to-verify-progress, what-to-do-with-the- learning, when-to-stop-or-ask — anchored by a real acceptance check and a deliberate hand-back to a human. This is Anthropic’s evaluator-optimizer / bounded-agent-loop discipline (Agentic Workflow Patterns, Workflows vs Agents), but productized: loops become named, self-describing, publishable artifacts (Use when / Prompt / Verify / Steps / Notes / Related) living in a shareable catalog. Worth caring about on two counts — as an independent corroboration that agent loops must be bounded and checked, and for its genuinely new angle: treating repeatable workflows as reusable library artifacts rather than one-off prompts. The website/worker plumbing (Cloudflare Worker + SQLite Durable Object, append-only revisions, fail-closed OAuth voting) is engine detail — held, not distilled.

Nate Herk (AI Automation) reads Anthropic’s own Prompting Claude Fable 5 doc and distills it into six paste-able habits under one through-line the source labels “the shift — smarter model, lighter touch”: because Fable 5 follows short, clear direction and reasons better than older models, piling on rules and telling it how to think now backfires, so the leverage moves to (1) give it the why — supply intent/context, not step-by-step detail; (2) tell it what NOT to do — negative-prompt the boundaries (“do not fix/send/edit until I say go”); (3) let it act once it has enough — stop over-planning, and match reasoning effort (low → x-high) to the task’s value rather than defaulting Fable 5 to everything; (4) make it prove it — the source’s claimed “most important” habit — require evidence for any “done” instead of trusting the claim; (5) — the one Fable-specific rule — stop asking it to “show its reasoning,” because a standing “explain your reasoning” line (especially in a system prompt/skill) can trip a safety classifier and silently reroute the task to a weaker backup model; and (6) say less, not more — a short instruction now steers as well as an over-detailed rulebook, which can actively make the answer worse. A closing explainer covers the safety-classifier handoff: requests that look like hacking, dangerous biology, or that ask the model to reveal its private reasoning silently route to Opus 4.8 — cheaper, capable, losing only Fable’s edge on long/hard tasks. The source states many groundable specifics (pricing $10/$50 per M input/output, a promo→usage-credit access window, a FrontierCode accuracy-vs-cost chart, “Mythos 5” as a restricted higher tier) — these are attributed to the source and flagged for later verification, not adjudicated here.

Drew Breunig shipped whenwords, a relative-time-formatting library that contains no code — just a SPEC.md and a tests.yaml, from which a coding agent regenerates the .py — to press the question, “if the agents are good enough, do we need to share code?” That reframes software as a spec-driven development (SDD) triangle: specs (what/why/sometimes how) + tests (verifiable behavior) = code (implementation). The insight of the talk is that generating code from a spec is the easy part; the standing problem is that spec, tests, and code are three redundant representations of one intent and therefore drift — bug-fixes add code and tests with no spec change, and agents make implementation decisions the spec never captured. Drift is detectable from signals you already emit (git diffs of code and of the spec, test-coverage tools, and agent traces), and the fix is to extract the intent — a decision log. Breunig’s demo tool Plumb runs on each commit: it identifies decisions from code diffs and agent traces, dedupes them, presents them to a human to approve, updates the spec, and reports spec-to-test and spec-to-code coverage. The payoff (agents refer to the spec instead of searching the whole codebase; a durable answer to “why does this code exist?”; even hacks get logged, never silently dropped) is real but honestly bounded: dedup is fuzzy, decision-rejection reversal doesn’t work yet, and it’s untested on large projects. He closes noting this rediscovers old ground (Dijkstra; Claire Le Goues’ 2009 automated software repair) — “inventing from first principles often means rediscovering from first principles.”

A ~9-minute IBM Technology glassboard explainer that defines spec-driven development (SDD) by contrasting it with vibe coding. Vibe coding starts from an initial natural-language prompt to a coding agent, which generates code from what it infers you want; you eyeball the result, edit the prompt, and loop until it looks right — fast and great for prototyping, but nondeterministic (the same underspecified request has “30 different ways” to be implemented, so “a hundred tries” yield a different result each time) and it skips the software development lifecycle (SDLC) entirely. SDD re-imports that lifecycle: you prompt the behavior and constraints you want, not an implementation; the spec is treated “like a contract” that generates requirements, which — once you approve them — become a design document with per-feature to-dos, which — once you approve that — the agent implements as code and tests. The load-bearing move is inverting the artifact order: where traditional dev goes code→docs and TDD goes test→code, SDD goes spec→design→implement→code (“TDD and BDD on steroids”), making the spec the primary artifact that drives all downstream work — implementation, tests, documentation, verification. The payoff the video sells is less ambiguity for the coding agent and auditability: because nothing is implemented until the spec is approved, “we know when the implementation starts why it got to this conclusion.” This is an introductory, whiteboard-level treatment — it corroborates the vault’s existing Spec-Driven Development concept from a fresh, mainstream source and names its foil, but adds no new mechanism for the hard part (keeping spec, tests, and code in sync as they drift).

anthropics/claude-cookbooks is Anthropic’s official, MIT-licensed collection of runnable Jupyter notebooks — “copy-able code snippets” for building with the Claude API (the capture reports 46,373★, updated 2026-07-02, README renamed from the older anthropic-cookbook). Its value to this vault is not a new argument but a canonical, first-party embodiment of a broad swath of concepts the graph already holds — the five Agentic Workflow Patterns, Pure Agent Applications via the Claude Agent SDK, Managed Agents (the managed_agents/ “CMA_” recipes), Agent-Computer Interface (ACI) tool-use, RAG, Eval-Driven Development, and sub-agents (Reasoning Effort Control’s Haiku-as-sub-agent-of-Opus recipe). So the dominant stance is corroborates: it is the runnable reference behind concepts sourced elsewhere, and it raises confidence without duplicating them. It earns its keep by surfacing two techniques the graph did not yet have a page for — Prompt Caching (misc/prompt_caching.ipynb, speculative_prompt_caching.ipynb) and Contextual Retrieval (capabilities/contextual-embeddings/) — which are spun out below. Should you care? Yes, as a lookup index*, not a read-through: when a capture demands a concrete recipe (batch, citations, JSON mode, PDF, memory tool, programmatic tool calling, extended thinking, fine-tuning on Bedrock), this repo is the first-party place to fetch it — a deep-read of any one notebook is an on-demand deep-research burst, not this triage pass.

A 14.5-min explainer (Nate Herk, AI Automation) that cuts through the “agent loop / loop engineering” hype into one durable anatomy: an agent loop is Reason → Act → Observe run until a done-check passes, and the two load-bearing pillars are a goal (as objective as you can make it) and verification (how the agent checks progress and knows when to stop). Its titular idea — Loop Engineering, “replacing yourself as the person who prompts the agent; you design the system that prompts it instead” — is genuinely new to the graph: the human’s leverage moves up a level, from per-iteration feedback to authoring the goal + acceptance check once. Everything else independently corroborates the bounded-loop discipline the graph already holds from Agent Loop / Forward-Future/loopy — a catalog and skill for bounded, reusable agent loops: loops must be bounded (hard-cap the passes), a loop is only as good as its done-check, and the best loops make “done” a machine-checkable metric (“keep iterating until X metric = Y”) rather than “until you’re satisfied.” Crucially it stays honest about scope: most tasks don’t need a loop — you reach for one for the verification/iteration it buys, a “solo loop” beats a swarm, and the loudest “stop prompting, write loops” advice doesn’t transfer to everyone (Agentic Simplicity). Illustrated altitude, not a transcript dump: the value is the skeleton-under-the-jargon and a handful of demo moments, not the rolling captions.

suntay44/buildable-plugin-skills (MIT, 48★, JavaScript, v1.0.1) is a local-first, file-based skills/plugin pack that gives a coding agent the product intelligence hosted no-code builders hide — archetypes, golden templates, UI/UX playbooks, and a review loop — so it goes from a vague prompt to a real prototype using only a small slice of context per request. It is not a hosted builder or a replacement agent: a dependency-free CLI (buildable plan/design/generate/review, plus /buildable-* slash commands and an optional MCP bridge) that runs in your repo, on Claude Code / Codex / Cursor. Its one genuinely new idea for the graph is Golden Templates — “adapt, don’t invent”: start from a curated, build-verified exemplar (every runnable starter is compiled/type-checked in CI) and adapt it, rather than reconstructing app structure from scratch. Everything else is a tight embodiment of concepts the graph already holds: it is CLI-first with MCP as a thin bridge (CLI Tools over MCP Servers), loads only the ~10% of its bundled “brain” a prompt needs via a plan-selected reference-loading contract (Search-Then-Get, Context Routing), runs a Plan → Design → Generate → Review chain with a real gate (Agentic Workflow Patterns), treats review as an enforceable local quality gate graded against rubrics (Eval-Driven Development), and makes the appSpec an audited, ask-first build contract (Spec-Driven Development). Should you care? As an artifact it is one author’s product, not a source of principles — but it is a clean, load-bearing second witness that the graph’s context-efficiency, CLI-over-MCP, and eval-gate theses are what people actually ship. Worth a look if you build agent skill-packs; the distinctive takeaway is the golden-template discipline, not the CLI surface.

Nimbalyst (MIT, ~1000★, TypeScript/Electron monorepo, v0.66.x) is a free, local-first visual workspace and session manager for driving coding agents — Codex and Claude Code, plus Opencode/Copilot in alpha. Its whole reason to exist is Agent Supervision: it makes agent output legible (approve/reject/annotate agent changes as red/green WYSIWYG diffs across markdown, mockups, Mermaid, Excalidraw, CSV, data models, and Monaco code), lets you run and track a fleet of parallel sessions (kanban, search/resume, link sessions↔︎files), and routes your attention to the agents that need you — including async review from a mobile app (“which agents need you, which are still working”). It stores state and workflow in open plain files (markdown + slash commands on disk/git), not an opaque DB. Should you care? As a product it is one team’s tooling, not a source of principles — but it is a concrete, load-bearing second witness that the human-oversight problem building-effective-agents held back is real enough to build an entire IDE around, which is why it promotes Agent Supervision from a held theme to its own concept. Worth a look if you supervise multiple agents and feel the review-bandwidth bottleneck; skip if you want architectural guidance rather than a workbench.

Hamel Husain’s foundational (March 2024) argument that unsuccessful LLM products share one root cause — no robust evaluation system — and that success, as in software engineering, is gated by iteration speed: evaluate → debug → change, run fast in a loop. Most teams do only the third (prompt/model changes) and never escape the demo. The concrete structure is a cost-ordered Levels of Evaluation: Level 1 cheap code assertions on every change, Level 2 human + calibrated LLM-as-Judge on a cadence, Level 3 A/B tests only when mature. The engine underneath is Error Analysis (“you can never stop looking at data; remove ALL friction”) and Synthetic Data Generation (generate inputs, not outputs). The payoff compounds: the same eval infrastructure is your debugging and fine-tuning infrastructure — “superpowers for free.” Keep it simple, build problem-specific evals, don’t buy generic frameworks.

One loop, run forever: build an instrumented agent → error-analyze real traces by hand → write evals only where the analysis justifies them → improve, gate in CI, monitor drift. The single highest-ROI activity is error analysis (bottom-up open coding of real traces into a failure taxonomy), not tooling. Judge subjective failures with a calibrated LLM-as-judge (binary pass/fail + written critique, validated against human labels). Bootstrap with synthetic user inputs when you have no users. Count experiments, not features.

Anthropic’s (Dec 2024) field guide from working with dozens of teams: the most successful LLM agent systems use simple, composable patterns, not complex frameworks — find the simplest solution and add complexity only when it demonstrably improves outcomes (Agentic Simplicity). The core architectural fork is Workflows vs Agents: workflows run LLMs/tools through predefined code paths (predictable, for well-defined tasks); agents let the LLM dynamically direct its own process (flexible, for open-ended tasks you can’t hardcode — at the cost of higher spend and compounding-error risk). Everything is built from one primitive, the The Augmented LLM (LLM + retrieval + tools + memory it drives itself). Between a single call and a full agent sit five Agentic Workflow Patterns — prompt chaining, routing, parallelization (sectioning/voting), orchestrator-workers, evaluator-optimizer. Whatever you build, the tools are the agent’s interface: invest in the Agent-Computer Interface (ACI) (ACI) as much as teams invest in HCI. Three closing principles: simplicity, transparency (show the planning steps), and a carefully crafted, well-documented ACI.

This capture is the Forward-Future/loop-library repository (2.2k★, MIT, JavaScript) — the monorepo that holds the Loop Library website/catalog (loop-library/: a site/ shell + a Cloudflare worker/ with a SQLite Durable Object) and the Loopy skill (skills/loopy/, with a loop-library compat alias). It is the same project already distilled as Forward-Future/loopy — a catalog and skill for bounded, reusable agent loops, viewed from the repo-name angle instead of the skill-name angle: the same two parts, the same load-bearing “loop” idea (a one-shot prompt turned into a bounded, verifiable, repeatable workflow answering goal / verify-progress / use-the-learning / stop-or-ask), the same eight-path Loopy lifecycle (Discover, Find, Loop Doctor, Adapt, Craft, Run, Debrief, Publish), the same ≥2-occurrence rule, the same self-describing loop anatomy (Use when / Prompt / Verify / Steps / Notes / Related), and the same web plumbing (Cloudflare Worker, append-only revisions, fail-closed OAuth voting) that Forward-Future/loopy — a catalog and skill for bounded, reusable agent loops already held as out-of-scope engine detail. Nothing is added over the canonical distillate — recorded here as duplicates (suppressed-but-logged, never silently dropped), with no new concepts and no concept edits. Read Forward-Future/loopy — a catalog and skill for bounded, reusable agent loops for the substance; this node exists only so the second capture of the same project is accounted for.