Agency OS Radar

Start Here (plain English)

The problem: a one-person agency using AI coding tools moves fast, but every external piece you bolt on — agents, skills, plugins, test frameworks — is a supply-chain risk you didn't write and can't fully read. Speed without checks ships defects and leaked secrets to paying clients.

The solution in today's radar: this week's most valuable releases are mostly trust tools — a scanner that checks AI agent skills before you install them, a governance toolkit that decides what agents are allowed to do, and AI code review that flags bugs before a human sees them. They let a vibe coder keep the speed and add the evidence.

Window: last 24h–7 days. All findings below are marked VERIFIED (seen in official repo/release sources) or UNVERIFIED (third-party reports). No prior radar reports were available to compare against, so repeats could not be checked — disclosed per research rules. Category 5 (reusable business-app components) had no verified new discovery this window; that slot is filled by the closest cross-category pick, and the gap is disclosed.

Top 5 Discoveries

1. NVIDIA SkillSpector — security scanner for AI agent skills TEST NOW

• Link: github.com/nvidia/skillspector
• What it does: scans AI agent skills (Claude Code, Codex, MCP) for malicious patterns, prompt injection, data exfiltration, and supply-chain risks before you install them, and outputs a risk report with severity and confidence.
• What changed & when: actively developed — repo pushed 2026-10-04 (VERIFIED via third-party tracker citing the repo); README updated ~Oct 14, 2026 citing research that 26.1% of a 31,132-skill sample contained vulnerabilities (UNVERIFIED — from third-party blog citing the README; re-check the README yourself).
• Why it matters to a vibe coder: you install skills and MCP servers written by strangers into the same environment that holds your clients' code and API keys. This is antivirus for that habit.
• License & costs: Apache-2.0 (VERIFIED, multiple sources). Free, open source; runs locally; no API key needed for scanning.
• Windows: Python-based (uv/pip, virtualenv). UNTESTED on Windows by me; plausible since it is pure Python, but the Makefile-driven workflow is Unix-flavored — verify on first run.
• Agency OS fit: SECURITY stage gate evidence before any REUSABLE_COMPONENT or skill is adopted. Agency OS compatibility: plausible, untested.
• Agent/stage: Security agent · SECURITY and REUSABLE_COMPONENT stages.
• Evidence output: a per-skill risk report (score, severity, findings with file/line and confidence) — a REAL artifact you can store as gate evidence.
• Limitations/risks: pattern-based scanning has false positives/negatives; a clean report is not proof of safety. Human review still required for anything touching production clients.

2. Microsoft Agent Governance Toolkit (AGT) INVESTIGATE

• Link: github.com/microsoft/agent-governance-toolkit
• What it does: a pip-installable governance layer for autonomous agents: policy enforcement ("is this action allowed?"), zero-trust identity ("which agent did this?"), execution sandboxing, and reliability engineering. Claims coverage of all 10 OWASP Agentic Top 10 risks.
• What changed & when: new-ish Microsoft project that surfaced in agent-governance coverage this week (Oct 2026). Exact release date and activity stats UNVERIFIED — check the repo's releases page.
• Why it matters: Agency OS already has a gate engine and governance rules; AGT is the same idea from a big vendor. Studying its policy/audit design (Merkle audit logs, privilege rings) can strengthen your own evidence model — without replacing it.
• License & costs: MIT (VERIFIED). Free and open source; full Azure-based deployment would incur Azure costs — not required for evaluation.
• Windows: Python pip package — plausible on Windows; UNTESTED.
• Agency OS fit: reference design for orchestrator policy checks. Do NOT swap your gate engine for it without evidence.
• Agent/stage: Orchestrator · all gated transitions (PROPOSAL→SPEC, RELEASE approvals).
• Evidence output: policy decisions and audit logs per agent action.
• Limitations/risks: young project, Azure leanings, and adopting a second decision engine risks conflicting with your single-source-of-truth rules. Investigate, don't adopt yet.

3. OpenAI Codex CLI 0.163.0 TEST NOW

• Link: github.com/openai/codex
• What it does: OpenAI's open-source terminal coding agent that reads, edits, and runs code locally.
• What changed & when: v0.163.0 went GA on October 15, 2026 with "mid-response steering" now default — you can redirect the agent while it is still generating (VERIFIED via release coverage; check the releases page for the official notes).
• Why it matters: steering a running agent instead of letting it finish a wrong path saves time and reduces runaway edits — the exact failure mode that hurts vibe coders most.
• License & costs: Apache-2.0 (VERIFIED). The CLI is open source; the model is not free — it requires a ChatGPT plan or paid OpenAI API key. Open source ≠ free model access.
• Windows: official Windows install: powershell -ExecutionPolicy ByPass -c "irm https://chatgpt.com/codex/install.ps1 | iex" (VERIFIED from official docs). Runs locally; model calls are the external dependency.
• Agency OS fit: dev-agent harness for DEVELOPMENT stage; steering helps human-in-the-loop control the CEO_REVIEW philosophy expects. Compatibility: plausible, untested.
• Agent/stage: Dev agent · DEVELOPMENT.
• Evidence output: session transcripts, diffs, and test runs produced during steering.
• Limitations/risks: agent can execute commands — use its sandbox/restricted modes, never give unrestricted computer access, and keep it out of client credential stores. Ongoing subscription/API cost.

4. Alibaba Open Code Review (ocr) TEST NOW

• Link: github.com/alibaba/open-code-review
• What it does: hybrid code-review CLI — deterministic rule pipelines (NPE, thread-safety, XSS, SQL injection) plus an LLM agent — producing precise line-level review comments from git diffs.
• What changed & when: latest release v1.12.12 on 2026-10-05, with releases landing every 1–3 days (VERIFIED via third-party tracker citing the repo); repo was on GitHub trending this month.
• Why it matters: a solo agency has no second engineer. This is a stand-in reviewer that catches the classic dangerous bug classes before your own eyes see the diff.
• License & costs: Apache-2.0, self-hosted — your code does not leave your infrastructure; you choose any OpenAI- or Anthropic-compatible model endpoint (VERIFIED). Model costs depend on the endpoint you pick (a local endpoint = free).
• Windows: Go CLI distributed via npm — plausible on Windows; UNTESTED. Verify on first run.
• Agency OS fit: a QA/CEO_REVIEW helper that turns every commit diff into structured findings — evidence, not vibes. Compatibility: plausible, untested.
• Agent/stage: QA agent (review) · QA and SECURITY stages, and pre-RELEASE review.
• Evidence output: line-level review comments and defect reports per diff/commit/branch range.
• Limitations/risks: LLM reviews have false positives/negatives; findings must be validated before advancing a gate — never treated as automatic approval.

5. TesterArmy e2e INVESTIGATE

• Link: github.com/tester-army/e2e
• What it does: end-to-end testing framework for web and mobile apps where tests are written as natural-language user goals ("upgrade the workspace to Pro"), recorded once with AI, then replayed deterministically with zero model calls.
• What changed & when: hit #1 on GitHub daily trending in early October (e2e 0.18 release; ~9.5k stars by Oct 12 per a third-party trending digest — star figures UNVERIFIED and conflicting between sources, check the repo).
• Why it matters: AI-authored tests that replay without AI cost directly serve Agency OS's UI-E2E QA layer for client web apps — and running costs stay near zero.
• License & costs: Apache-2.0 (VERIFIED via third-party report citing the repo). Free and local; a model is needed only at recording time.
• Windows: TypeScript/Node-based — plausible on Windows; UNTESTED.
• Agency OS fit: the UI-E2E slice of the QA stage for client web apps. Compatibility: plausible, untested.
• Agent/stage: QA agent · QA stage.
• Evidence output: replayable test suites and pass/fail reports per run.
• Limitations/risks: young, fast-moving framework — API churn risk; verify maturity and CI integration before promising it in client contracts.

Disclosure: no verified new discovery this window in category 5 (reusable business components — dashboards, API starters, desktop/Android kits). Nothing worth fabricating a slot for; five strong beats five weak. Watch this category again tomorrow.

Test Today: NVIDIA SkillSpector

• Why this one: fully local, free, no API keys, no subscription, Python-based — realistic on a Windows PC with an AI IDE in under an hour. It also addresses the riskiest habit a vibe coder has: installing third-party skills.
• Prerequisites: Python 3.10+ and pip (or uv) on Windows; git.
• Official install steps (from the repo): create and activate a virtualenv first (the Makefile assumes it); install with uv or pip; then run skillspector scan against a skill directory. Follow the repo README exactly — it is the authoritative source; if a Makefile step fails on Windows, run the underlying commands manually rather than improvising.
• Practical test: pick one skill you already use or are considering installing; run the scanner on its folder; read the report.
• Expected results: a risk report with a 0–100 score, severity rating, per-file findings (e.g., env-variable harvesting), and a recommendation such as DO NOT INSTALL for dangerous patterns.
• Safety precautions: run inside a virtualenv; never pipe install scripts straight into a shell without reading them; scan copies, not live installs; a HIGH score means stop — do not install to review it. Never give any scanner unrestricted computer access.

Agency OS Integration Proposal: SkillSpector as SECURITY-stage evidence

• Problem solved: REUSABLE_COMPONENT onboarding and skill adoption currently have no automated security evidence — the security agent would have to rely on claims (SIMULATED/ASSUMPTION) instead of artifacts.
• Workflow improved: security agent's component/skill intake check.
• Lifecycle location: SECURITY stage, plus the REUSABLE_COMPONENT stage when a skill is promoted for reuse.
• Inputs: a skill/MCP package directory (local copy).
• Outputs: a SkillSpector risk report stored as project evidence with REAL classification and the scan date/version.
• Validation: evidence must be validated before advancing: report exists, scanner version recorded, findings triaged by the security agent; findings marked VERIFIED only after a human reads the flagged lines.
• On failure: any HIGH finding marks the component BLOCKED; the security agent records BLOCKED in project-state.json with the report as evidence. BLOCKED components cannot advance without an explicit human waiver.
• Gate blocking / approval: it feeds the existing SECURITY gate as evidence — it does not replace the gate engine. A waiver (accepting a flagged component) is a scope/safety decision → human approval required, per governance rules.
• Smallest safe experiment: in a virtualenv, scan one existing skill you already use; save the report; have the security agent classify findings REAL/SIMULATED/UNVERIFIED. No production change, no gate modification. Cost: ~30 minutes.

Business Opportunity (Ethiopia)

• Customer problem: small businesses in Addis Ababa adopting AI assistants (shops, agencies, clinics, importers) increasingly install third-party plugins and "skills" — with no way to know whether one silently sends their customer data or API keys abroad.
• Minimum viable service: a fixed-price "AI Tool Safety Audit": inventory the client's AI tools and plugins, run SkillSpector-style scans on every installed skill/MCP, deliver a plain-Amharic/English risk report with remove/keep/replace recommendations, plus a one-page plugin policy for staff.
• Delivery workflow: LEAD → DISCOVERY (interview + tool inventory) → SECURITY (local scans, evidence reports) → CEO_REVIEW (human signs off) → RELEASE (report + policy handover) → MAINTENANCE (re-scan quarterly).
• Main risks: clients undervaluing prevention (sell after a scare story or bundle with setup); false positives damaging trust (always human-review flagged lines before reporting); scanner gaps (clean ≠ safe — say so in the report).
• No revenue projections — price against local freelance audit rates once you have delivered one paid pilot.

Daily Learning: Skill supply-chain risk (prompt injection & exfiltration)

• Concept: an AI "skill" or plugin is code plus instructions that your agent will follow. A malicious one can embed instructions that redirect the agent ("send this .env file to help debug") — the agent follows them because instructions look authoritative. That's prompt injection; shipping the data out is exfiltration.
• How an AI agent gets it wrong: it treats a skill's README as trusted documentation and recommends installing it based on its own description — reasoning from the attacker's own words.
• How you recognize the failure: an adoption recommendation with no scan artifact behind it; evidence classed as VERIFIED when it's actually UNVERIFIED.
• Which tool/test detects it: SkillSpector scan before install, plus your own rule: no skill enters an Agency OS environment without a stored scan report.

Final Summary

• Best discovery today: NVIDIA SkillSpector — turns your riskiest habit (installing stranger-written skills) into a gated, evidence-backed process.
• Best security/reliability improvement: SkillSpector scans + Open Code Review on every diff — one guards what comes in, the other guards what goes out.
• Best Agency OS integration: SkillSpector as SECURITY-stage evidence producer (REAL artifacts, BLOCKED on HIGH findings, human waiver only) — without touching the gate engine.
• One practical action today: set up a virtualenv, scan one skill you already use, and save the report as your first REAL security evidence artifact.