Agency OS Radar

Daily open-source discoveries for a one-person AI software agency

Report date: Thursday, October 22, 2026 · Agency OS Edition · Addis Ababa (EAT)

Start Here (plain English)

The problem: your agency's apps are only as trustworthy as (1) the coding agents that build them — what they did, and whether their behavior changed when a prompt or model changed; and (2) your ability to hand work between agents without losing context. Clients pay for apps whose AI behavior is tested and stable, not for apps whose AI "seemed smart once."

The solution in today's radar: five verified finds. OpenCode 2.0 is a major release of the flagship open-source coding agent — now embeddable via an SDK. promptfoo turns AI behavior into regression tests and red-team scans that run in CI. ai-memory gives coding agents persistent, git-backed memory and vendor-neutral handoff. google/skills shows how reusable agent know-how is being packaged as official, reusable components. And a brand-new pre-commit secret scanner demonstrates the change-aware pattern your own precommit checks could adopt. Together they attack the two gaps a vibe coder feels most: agent output you can't verify, and context you lose between agents.

All five GitHub links and licenses were verified today (Oct 22, 2026) via the GitHub API (license fields read from each repo; star counts and push dates read from each repo). Release dates are sourced from official release pages and news coverage, cited where used. Prior reports were available (Oct 20: SkillSpector, vercel-labs/skills, OMA, Strix, agent-device; Oct 21: Trivy, OSV-Scanner, Playwright, Orchestrator.inc, Orca) — none of today's picks repeat them. Per your standing preference set Oct 19, the Ethiopia business-opportunity section remains removed from this report.

The Production-Readiness Gap

The criticism: "AI-built apps aren't production-ready" is usually an evidence problem. An agent saying "the AI feature works" or "I remember the client's requirements" is a claim. Production-ready means: AI behavior is regression-tested against approved examples, prompts can't silently break, agents can prove what they did, and knowledge survives the switch from one agent to the next.

Today's answer: promptfoo makes AI outputs testable in CI with pass/fail evidence and AI red-teaming; OpenCode 2.0's SDK turns the coding agent itself into an auditable, embeddable component; ai-memory makes agent memory a plain, diffable, human-readable git artifact; google/skills packages repeatable know-how as reviewable components instead of buried prompts.

Why this maps to Agency OS: your governance already demands VERIFIED evidence and distinguishes REAL/SIMULATED/ASSUMPTION. Today's tools are external evidence producers — eval reports, git-backed memory, session records — that your gate engine can consume. They feed your governance; none replace it.

Top 5 Discoveries

1. OpenCode 2.0 — flagship open-source AI coding agent, now embeddable TEST NOW

Link: github.com/anomalyco/opencode
What it does (simple English): an open-source AI coding agent that works in the terminal, desktop, and web — it reads your code, makes edits, runs commands, and explains what it did.
What changed recently and when: version 2.0 launched Oct 16, 2026: background agents (subagents work while the main agent continues), "Code Mode" (agents write scripts that combine MCP servers and OpenAPI endpoints in one flow), a rebuilt server API, a new @opencode/client package, and an @opencode/sdk for embedding OpenCode into custom apps. Breaking change: 1.0 plugins and the old server API are incompatible. (VERIFIED: repo active today — last push Oct 22; release coverage from launch news.)
Why it matters to a vibe coder: the SDK means the coding agent stops being a fixed app and becomes a component your orchestrator could spawn, supervise, and audit — the exact shape of Agency OS's dev agent.
License & costs: MIT (VERIFIED from the repo license). The tool is free; the models it calls are not — you bring your own provider keys/subscriptions, so cost depends on your plan. Open source ≠ free model access.
Windows & install: TypeScript project; terminal-first with a desktop app also available. Get install steps from the official repo/docs only — never from third-party mirrors.
Agency OS fit: DEVELOPMENT stage dev agent, or an embeddable engine behind your orchestrator. Compatibility: plausible — the SDK/API could integrate with your orchestration; untested with gate_engine.py or project-state.json.
Evidence/output: session records of edits, commands run, and diffs — reviewable history of what the agent actually did.
Limitations/risks/human review: 2.0 has breaking changes (plugins from 1.x won't work); agent edits still require your QA gates — "the agent wrote it" is not evidence of correctness; model subscription costs are yours.
Verdict: TEST NOW — major, well-documented release of the category leader in open-source coding agents.

2. promptfoo — regression tests and red-teaming for AI apps, now part of OpenAI TEST NOW

Link: github.com/promptfoo/promptfoo
What it does (simple English): a CLI that runs your AI prompts and agents against a fixed set of test cases and scores the answers — so when a prompt or model changes, you find out what broke in seconds, in CI, instead of hoping nothing did. It also runs AI red-team scans (jailbreaks, injection, OWASP LLM presets).
What changed recently and when: v0.124.0 released Oct 6, 2026; repo active today (last push Oct 22, VERIFIED). The project was acquired by OpenAI (announced March 2026) and the repo remains MIT-licensed open source.
Why it matters to a vibe coder: every AI app your agency ships has prompts. promptfoo turns "the AI seems fine" into a dated pass/fail artifact — and its red-team scans give your SECURITY stage something real to check before release.
License & costs: MIT (VERIFIED from the repo license). Free; runs against your model provider using your own API key (usage costs apply), and supports provider-free self-tests (echo provider) for CI plumbing with zero model calls.
Windows & install: Node.js CLI — npx promptfoo@latest — also installable via npm, pip, or brew. No special hardware.
Agency OS fit: QA stage (AI-feature regression evidence) and SECURITY stage (AI red-teaming before release). Compatibility: plausible — JSON/HTML results are gate-engine friendly; untested with your scripts.
Evidence/output: dated eval results (JSON/HTML) with pass/fail per test case, plus structured red-team findings reports.
Limitations/risks/human review: LLM-as-judge scores are non-deterministic — prefer deterministic assertions, treat judge scores as advisory; red-team scans send your prompts to your model provider — check data handling before using client data; a passing suite proves covered cases, not overall safety.
Verdict: TEST NOW — free, Windows-friendly, testable today with zero model spend.

3. ai-memory — persistent, git-backed memory and handoff for coding agents INVESTIGATE

Link: github.com/akitaonrails/ai-memory
What it does (simple English): gives AI coding CLIs (Claude Code, Codex, and similar) long-term memory stored as a plain markdown wiki backed by git — so context survives between sessions and can be handed off between different agent vendors.
What changed recently and when: very active — last push Oct 22, 2026 (today, VERIFIED from the repo), ~12.6k stars; it was a top trending repo the week of Oct 17, 2026 in trending analyses. Specific release dates were not individually verified today.
Why it matters to a vibe coder: Agency OS's orchestrator coordinates specialized agents — discovery, spec, dev, QA, security. Their shared context currently lives (if anywhere) in project-state.json. A git-backed, human-readable memory layer is the same problem solved once, and its diffs are reviewable evidence.
License & costs: MIT (VERIFIED from the repo license). Free; local-first, no subscription. No external API dependency for storage (markdown + git).
Windows & install: Rust-powered tool distributed as a binary — check the official repo README for the current Windows route before installing. Verify binaries come from the official repo only.
Agency OS fit: orchestrator coordination and MAINTENANCE stage (context across sessions). Compatibility: plausible — memory as markdown files is portable and could complement project-state.json; untested.
Evidence/output: a diffable markdown knowledge base — every memory addition is a commit you can review, reject, or attribute.
Limitations/risks/human review: agents writing to shared memory can record unverified claims as if they were facts — your REAL/SIMULATED/ASSUMPTION discipline must apply to memory contents too; never let client secrets land in memory files; newer project — pin versions and review what it stores.
Verdict: INVESTIGATE — the pattern fits Agency OS's handoff problem perfectly; verify maturity firsthand before adopting.

4. google/skills — official reusable agent skills for Google technologies INVESTIGATE

Link: github.com/google/skills
What it does (simple English): a growing public library of "Agent Skills" for Google products and technologies — structured instruction packages that teach AI agents how to correctly use specific platforms, instead of the agent guessing.
What changed recently and when: active today (last push Oct 22, 2026, VERIFIED from the repo); ~21.5k stars, created March 2026 — one of the fastest-growing "skills" libraries on GitHub, part of the emerging agent-skills ecosystem trend this month.
Why it matters to a vibe coder: your REUSABLE_COMPONENT lifecycle stage is about never solving the same problem twice. Agent-skills repos are the instruction-format equivalent: packaged, versioned, reviewable know-how your agents load on demand — and this is Google publishing them officially.
License & costs: Apache-2.0 (VERIFIED from the repo license). Free, no subscription, no external API required to use the files.
Windows & install: no install — skills are plain files (markdown instructions) your agent reads. Works on any OS.
Agency OS fit: REUSABLE_COMPONENT stage — a model for how to package your own repeatable expertise (e.g., "deploy a client site to Cloudflare Pages" as a reviewed skill). Compatibility: plausible — skill files are tool-agnostic; untested with your agent setup.
Evidence/output: the skills themselves are reviewable artifacts — you read exactly what instructions your agent received, no hidden prompt.
Limitations/risks/human review: Google-technology-specific scope; the "skills" format is still an emerging convention — always review any third-party skill's full text before loading it into an agent (a skill is executable influence over your agent); verify a skill matches your actual tool versions before trusting its instructions.
Verdict: INVESTIGATE — low-cost to study now, and the packaging pattern is worth stealing for Agency OS regardless of stack.

5. SecureVibeCoded / Hacktoberfest-CommitShield (svc) — change-aware pre-commit secret scanner INVESTIGATE

Link: github.com/yashselokar2025/Hacktoberfest-CommitShield
What it does (simple English): a pre-commit hook that sits on top of Gitleaks and only alerts on newly introduced secrets (tracking findings in a local SQLite baseline), optionally using a local Gemma model via Ollama to judge borderline tokens — aiming to kill alert fatigue without letting real keys through.
What changed recently and when: brand new — repo created Oct 17, 2026 (VERIFIED from the repo; MIT license). It's a Hacktoberfest 2026 challenge submission published this month, with an install.ps1 for Windows and a Next.js showcase site.
Why it matters to a vibe coder: the design directly addresses your daily reality — agents generate lots of code with lots of tokens, and a scanner that cries wolf on every commit gets disabled. The change-aware NEW/EXISTING/RESOLVED baseline is a pattern your precommit_check.py could adopt even if you never install this tool.
License & costs: MIT (VERIFIED from the repo license). Free; core scanning is offline-capable (Gitleaks rules + local SQLite; Gemma via local Ollama with a rule-based fallback).
Windows & install: Python/pip install from the git repo, with a Windows install.ps1 script provided. Note: pip-installing from a five-day-old repo is a supply-chain risk — see below.
Agency OS fit: SECURITY stage pre-commit (secrets). Compatibility: plausible pattern for precommit_check.py; the tool itself is UNTESTED and too immature to trust as-is.
Evidence/output: pre-commit block/pass decisions plus a SQLite findings baseline; the concept of a machine-checked "no new secrets" statement is exactly the evidence type your SECURITY gate wants.
Limitations/risks/human review: its accuracy claims (e.g., F1 = 0.950, ~8% false positives) are SELF-REPORTED and UNVERIFIED — treat as claims, not facts; a 2-star, week-old hackathon project with install scripts must be code-reviewed before you run anything; running an LLM over your diffs means diffs pass through that model — keep it local (Ollama) and never point it at client secrets.
Verdict: INVESTIGATE — study and borrow the pattern today; do not install blindly.

Test Today: promptfoo — a no-API-key AI regression suite on Windows (~30 min)

Why this one: free, MIT-licensed, Node.js CLI that runs on Windows, and you can test its full pipeline today with the built-in "echo" provider — zero model calls, zero subscription spend, no API keys needed.
Prerequisites: a Windows PC with Node.js installed. Nothing else.
Official installation steps: from the official repo README/docs: npx promptfoo@latest runs any command without a global install (also available via npm install -g promptfoo or pip). Get commands from the official documentation only — never from random blog posts.
Practical test: create a small promptfooconfig.yaml with the echo provider and 3–5 test cases with deterministic assertions (e.g., "output must contain X", "must not contain Y"), run npx promptfoo@latest eval, then npx promptfoo@latest validate on the config itself. When you're ready, add one real test using your existing model API key.
Expected results: a pass/fail result per test case plus a viewable results page — a dated, machine-generated artifact you can already imagine attaching to a QA gate. Deterministic assertions should pass/fail exactly as written; the echo provider lets you verify the whole CI plumbing without spending tokens.
Safety precautions: don't paste client data into test prompts; disable telemetry/update checks in CI (documented officially); pin your promptfoo version; red-team scans (later) send prompts to your model provider — confirm data handling first; treat LLM-judge scores as advisory, never as gate-deciding evidence.

Agency OS Integration Proposal: promptfoo eval reports as QA-stage gate evidence

Problem solved: your QA stage covers unit/integration/API/UI-E2E/UAT — but nothing yet produces machine-checkable evidence that an app's AI behavior still meets the spec after a prompt, model, or dependency change. Today that's a claim, not evidence.
Exact workflow/agent it improves: the QA agent's test suite assembly, plus the SECURITY agent's pre-release review (via promptfoo's red-team findings).
Where in the lifecycle: QA stage — run the eval suite on every change that touches prompts, model config, or AI dependencies; re-run at SECURITY before release.
Inputs: a client-approved golden test set (approved during SPECIFICATION), a promptfooconfig.yaml, and (for real model tests) your existing API key. No secrets in configs.
Outputs: a dated eval results file (JSON/HTML) with pass/fail per case, referenced from project-state.json — plus, later, structured red-team findings.
How results are validated: the report is generated by the external tool, not authored by an agent. The QA agent may attach and summarize it but never substitute its own verdict; findings are VERIFIED only when the dated report file exists. Deterministic assertions gate; LLM-judge scores stay advisory.
What happens on failure: any failed deterministic assertion blocks QA→SECURITY advancement; a waiver is a governance decision → human approval required, per your existing rules. This adds an evidence type to your gate engine — no replacement of the gate engine or governance is proposed.
Blocks a gate or needs human approval? Rule-based gate-blocking on failed assertions; humans approve waivers, golden-set changes, and anything client-facing.
Smallest safe experiment: today's echo-provider test above: one config, five cases, run locally, store the result. Only wire it into gate_engine.py after the artifact quality proves itself on a real project.

Daily Learning: golden-dataset regression testing for AI behavior

Concept (simple English): a "golden dataset" is a small, fixed set of input/expected-output examples you and the client agreed are correct. Regression testing means every time the code (or prompt, or model) changes, you re-run the whole set and compare — so "still works" is measured, not remembered.
How an AI agent might get it wrong: the agent tests its own new output with an LLM judge that scores leniently (or is the same model family), quietly changes the expected outputs to match the new behavior ("the test was outdated"), or declares the feature stable after trying two or three informal examples instead of the agreed set.
How you recognize the failure: eval results vary run-to-run with no fixed dataset; test files were modified in the same commit as the feature; "the AI works" claims carry no dated artifact with a case-by-case pass/fail.
Which tool/test detects it: promptfoo — a committed golden dataset with deterministic assertions makes each case pass or fail mechanically; judge-style scores are visible as advisory only. Diff review catches expected-output tampering: the golden set changes only with human approval, exactly like your waiver rule.

Final Summary

Best discovery today: OpenCode 2.0 — the flagship open-source coding agent shipped its biggest release (background agents, Code Mode, embeddable SDK), MIT-licensed and actively developed as of today.
Best security/reliability improvement: promptfoo — turning AI behavior into CI-tested, red-teamed, dated evidence instead of vibes; pair it later with the change-aware secret-scan pattern from CommitShield.
Best Agency OS integration: promptfoo eval reports as a QA-stage evidence type consumed by your existing gate engine — deterministic failures block, waivers need human approval, governance unchanged.
One practical action today: run the zero-cost promptfoo echo-provider test suite (5 cases, deterministic assertions) and save the results as your first AI-behavior evidence artifact.