Agency OS Radar

Start Here (plain English)

The problem: you run AI coding agents on the same PC that holds your clients' code, API keys, cloud logins, and payment credentials. If an agent is ever hijacked (for example by a malicious skill or prompt injection), everything it can reach is stolen in one go — and you have no list of what it can actually reach.

The solution in today's radar: today's best finds are measurement tools — one lists every credential your agent could grab, one checks AI infrastructure for known weaknesses, one turns real traffic into API tests, and a big-vendor framework release shows how mature agent orchestration is evolving. They turn "I think I'm safe" into a saved, checkable report.

Window: last 24h–7 days. Yesterday's report (Oct 17: SkillSpector, MS Agent Governance Toolkit, Codex CLI, Alibaba open-code-review, TesterArmy e2e) was retrieved and checked — none are repeated today because none had a verified meaningful update in this window. Findings are marked VERIFIED (seen in the repo, release, or license file) or UNVERIFIED. Category 5 (reusable business components) again had no verified new discovery — disclosed, not padded; five strong beats five weak.

Top 5 Discoveries

1. heldkeys — which keys is your agent holding? TEST NOW

• Link: github.com/hugomn/heldkeys
• What it does: a small Rust CLI that answers one question: "if a prompt injection hijacked the AI agent running here, which credentials could it use?" It lists the credentials an agent can reach — environment variables, cloud CLI logins (AWS, gcloud, Azure), AI tool logins (Claude Code, Codex, Gemini CLI, Copilot, Hugging Face), .env files and their editor backups, SSH keys, package/git tokens, and tokens pasted into MCP server configs — without ever printing a single secret. It also checks agent risks in GitHub Actions and has a --strict mode that fails like a gate.
• What changed & when: brand-new project — repo created October 15, 2026; actively updated through October 18 (VERIFIED via GitHub).
• Why it matters to a vibe coder: your IDE, terminals, and agents sit next to every client login you own. heldkeys turns that invisible exposure into a concrete list — the input for real least-privilege cleanup.
• License & costs: MIT (VERIFIED from repo). Free, open source, runs fully locally, no API keys, no subscription.
• Windows: NOT supported yet — built and tested on Linux and macOS only; on Windows use the Linux binary under WSL (VERIFIED from README FAQ). Native Windows support is on their roadmap.
• Agency OS fit: a preflight check before any agent is allowed near a client project. Agency OS compatibility: plausible, untested.
• Agent/stage: Security agent · SECURITY stage, and as a preflight gate before DEVELOPMENT and RELEASE.
• Evidence/output: a credential-exposure report (severity-graded findings, e.g. "HIGH: Docker registry credentials, plaintext") — REAL evidence, no secrets leaked into the report itself.
• Limitations/risks: it can't see browser sessions, IDE tokens, OS keychains, or unlocked password managers, and it can't tell what a token is allowed to do — a clean report means "no known exposure", not "no exposure" (README is explicit about this). Human review still decides what to revoke or move.

2. Tencent AI-Infra-Guard v4.6.6 — AI red-teaming & security scanner INVESTIGATE

• Link: github.com/Tencent/AI-Infra-Guard
• What it does: a full-stack AI security platform from Tencent's Zhuque Lab: agent scan, skills scan, MCP scan, and AI infrastructure scan. It has a web UI with one-click scanning, a Swagger-documented API, and plug-and-play agent skills for AI agent workflows.
• What changed & when: release v4.6.6 on October 15, 2026 (VERIFIED via GitHub releases). ~6.9k stars per GitHub search on Oct 18.
• Why it matters to a vibe coder: it complements yesterday's SkillSpector: it scans not just skills but whole AI deployments (MCP servers, agent endpoints, AI infrastructure) — the wider blast radius a one-person agency never has time to audit by hand.
• License & costs: Apache-2.0 with attribution requirements (VERIFIED from README): you must credit "Tencent Zhuque Lab AI-Infra-Guard" in docs/UI if you embed it, and repackaging it as your own product is prohibited. Open source ≠ unrestricted commercial use — read the LICENSE/NOTICE terms before client work. Free to run; Docker images on Docker Hub.
• Windows: cross-platform via Docker-based deployment, including Windows (VERIFIED from README).
• Agency OS fit: security agent's platform-level scanner for client deployments. Agency OS compatibility: plausible, untested.
• Agent/stage: Security agent · SECURITY stage and MAINTENANCE re-scans.
• Evidence/output: scan reports with findings per target (agent/skill/MCP/infra).
• Limitations/risks: scanner findings include false positives and must be validated before anything is marked VERIFIED; attribution clause limits white-label use; large platform — heavier than a single-purpose tool.

3. Keploy v3.6.141 — API tests from real traffic TEST NOW

• Link: github.com/keploy/keploy
• What it does: a developer-centric API and integration testing tool that auto-generates test cases (with mocks) from real application traffic instead of hand-written test code, then replays them; it also creates safe, isolated production sandboxes for API, integration, and E2E testing. It pitches "API tests faster than unit tests, from user traffic."
• What changed & when: release v3.6.141 published TODAY, October 18, 2026 (VERIFIED via GitHub releases) — very high release cadence. ~18.6k stars per GitHub search on Oct 18.
• Why it matters to a vibe coder: AI agents write features fast but regressions in API behavior are what quietly break client systems. Recording real calls and replaying them as tests gives Agency OS's API-QA layer evidence without writing hundreds of test stubs by hand.
• License & costs: Apache-2.0 (VERIFIED from LICENSE file). The OSS tool is free; Keploy also sells a cloud/enterprise offering — not required for local use.
• Windows: the keploy binary is Linux-native; on Windows use WSL2 (per Keploy's docs) — UNVERIFIED by me on Windows, check the official docs before installing.
• Agency OS fit: API/integration slice of the QA stage. Agency OS compatibility: plausible, untested.
• Agent/stage: QA agent · QA stage, plus regression evidence for RELEASE.
• Evidence/output: generated test case files, mocks, and pass/fail test-run reports per replay.
• Limitations/risks: recorded tests prove "same behavior as before", not "correct behavior" — someone must define what correct is; recordings can contain sensitive data that must be scrubbed before being stored as evidence.

4. Microsoft Agent Framework 1.22.0 (Python) — multi-agent orchestration INVESTIGATE

• Link: github.com/microsoft/agent-framework
• What it does: Microsoft's open-source framework for building, orchestrating, and deploying AI agents and multi-agent workflows (Python and .NET): agents with tools, graph/sequential/concurrent workflows, checkpoints, memory, and human-in-the-loop support.
• What changed & when: python-1.22.0 released October 15, 2026 (VERIFIED via GitHub releases): Python 3.15 support (partial — some packages/extras lag), workflow checkpoint and session improvements, and an important compatibility warning: pre-1.22 runtimes cannot read records written by 1.22 — do not mix old and new workers against the same persisted store.
• Why it matters to a vibe coder: Agency OS's orchestrator coordinates specialized agents with state and gates. This release shows the exact patterns (checkpoints, persisted state, human-in-the-loop) a production multi-agent system needs — useful as a reference design even if you never adopt it.
• License & costs: MIT (VERIFIED from LICENSE file). Free; model access costs depend on which LLM provider you point it at (a local model via Ollama is possible but unverified here).
• Windows: Python/.NET — runs on Windows; UNTESTED by me.
• Agency OS fit: reference for the orchestrator's workflow and checkpoint design. Do NOT swap your gate engine or project-state.json for it without evidence.
• Agent/stage: Orchestrator · all stages (workflow coordination).
• Evidence/output: checkpointed workflow state, execution traces, human-in-the-loop approval records.
• Limitations/risks: the persisted-state breaking change in this very release is the risk in miniature — agent state formats churn; pin versions, isolate state stores, and test upgrades. Model API costs apply.

5. browser-use 0.13.13 — AI browser agent INVESTIGATE

• Link: github.com/browser-use/browser-use
• What it does: open-source Python library that lets an AI agent control a real browser (click, type, navigate) to complete tasks — usable for automated web QA and UI-E2E checks.
• What changed & when: release 0.13.13 on October 15, 2026, named "OpenAI browser tools and Cloud fixes" (VERIFIED via GitHub releases). ~118k stars per GitHub search on Oct 18.
• Why it matters to a vibe coder: a QA agent that can actually operate a client's web app like a user is the missing UI-E2E pair to Keploy's API tests.
• License & costs: MIT (VERIFIED from LICENSE file). The library is free; you supply the model — Browser Use Cloud and some providers are paid, but the README documents running with a local browser and a local model via Ollama (subject to your hardware).
• Windows: Python library — works on Windows in principle; UNTESTED by me.
• Agency OS fit: UI-E2E slice of QA. Agency OS compatibility: plausible, untested.
• Agent/stage: QA agent · QA stage (UI-E2E).
• Evidence/output: step-by-step agent traces and pass/fail outcomes per web task.
• Limitations/risks: an agent with browser control is powerful — run it in a dedicated browser profile, never on accounts holding client credentials or payments; a passing run is not a full E2E suite; model quality governs reliability; recurring model costs unless run locally.

Disclosure: no verified new discovery this window in category 5 (reusable business-app components — dashboards, API starters, desktop/Android kits) for the second day running. New "template" repos found in the window were either replicas of closed SaaS products (license risk) or too immature to recommend. Watch this category again tomorrow.

Test Today: heldkeys (under WSL)

• Why this one: fully local, free, no API keys, no subscription, single small binary — and it measures the single biggest silent risk in your setup. It is the realistic one-hour test on a Windows PC via WSL.
• Prerequisites: WSL2 with Ubuntu (or any Linux/macOS machine). Rust toolchain if installing from crates.io, or grab the release binary from the repo's releases page.
• Official install (from the repo): it is published on crates.io, so cargo install heldkeys — or download the prebuilt binary from the repo's releases page. Follow the README exactly; it is the authoritative source.
• Practical test: inside WSL, in the same environment your coding agent/IDE shells run (same user, not root — the README warns root sees more than the agent can), run heldkeys, then heldkeys --details to see the first findings.
• Expected results: a summary table of credential types found (env vars, cloud CLIs, AI tool logins, .env files, SSH keys) with severities like "HIGH: Docker registry credentials (plaintext)" — and zero actual secret values printed.
• Safety precautions: run as the agent's user, not root; never run untrusted scanners outside a controlled environment; treat the report itself as sensitive metadata; do not paste full reports into chats or repos.

Agency OS Integration Proposal: heldkeys as a preflight security gate

• Problem solved: nothing currently measures what credentials an Agency OS agent could reach — the security agent would be asserting safety from an ASSUMPTION, not evidence.
• Workflow improved: security agent's environment preflight check, run before an agent session starts on a client project.
• Lifecycle location: SECURITY stage; also run as a preflight before DEVELOPMENT sessions and before RELEASE.
• Inputs: none beyond the environment itself — it runs in the agent's shell context.
• Outputs: a credential-exposure report stored as project evidence, classified REAL, with scanner version and date.
• Validation: evidence validated before advancing: report exists, findings triaged; each finding marked VERIFIED only after a human confirms the credential is really exposed and decides revoke/move/accept.
• On failure: with --strict, unreadable-but-present files fail the gate; HIGH findings mark the environment BLOCKED in project-state.json with the report as evidence. BLOCKED requires an explicit human waiver to proceed.
• Gate blocking / approval: it feeds the existing gate engine as evidence — it does not replace it. Accepting an exposure (waiver) is a safety decision → human approval required, per governance rules.
• Smallest safe experiment: in WSL, run heldkeys once in your dev environment today; save the report; have the security agent classify each finding REAL/UNVERIFIED/BLOCKED. No production change, no gate modification, ~30 minutes.

Business Opportunity (Ethiopia)

• Customer problem: small businesses in Addis Ababa — agencies, shops, clinics, travel firms — are adopting AI assistants and automation tools on office PCs that also hold payment API keys, Telebirch/Chapa credentials, email accounts, and customer data. Nobody has ever listed what their AI tools could touch if hijacked.
• Minimum viable service: a fixed-price "AI Credential Exposure Audit": inventory their AI tools, run a heldkeys-style scan in each working environment, and deliver a plain-Amharic/English report — what is exposed, what to revoke, what to move into a credential store — plus a one-page staff policy (no pasting tokens into chats, .env files out of backups).
• Delivery workflow: LEAD → DISCOVERY (interview + tool inventory) → SECURITY (local scans, evidence reports) → CEO_REVIEW (human signs off) → RELEASE (report + policy handover) → MAINTENANCE (re-scan after each new tool install).
• Main risks: clients see prevention as invisible value (bundle with a concrete fix: "audit + we revoke/move the top 5 exposures same day"); trust risk if you overstate findings (clean report ≠ safe — say so); WSL/technical setup friction on client machines.
• No revenue projections — price against local freelance IT-audit rates after one paid pilot.

Daily Learning: Credential blast radius

• Concept: "blast radius" is the damage possible if something goes wrong. For an AI agent, its credential blast radius is every key, login, and token it can reach from where it runs — often far more than you consciously gave it, because CLIs, .env files, and editors quietly store logins on your machine.
• How an AI agent might get it wrong: it assumes it only has the credentials it was explicitly handed ("I only have the API key in my config"), and reports the environment as safe based on that assumption.
• How you recognize the failure: a security claim with no scan artifact behind it; an evidence item classified VERIFIED that is actually an ASSUMPTION.
• Which tool/test detects it: heldkeys (today's test) enumerates the real blast radius without printing secrets; your rule: no agent environment is declared safe without a stored, dated scan report.

Final Summary

• Best discovery today: heldkeys — new, free, MIT, and it answers the most important unasked question in AI-assisted development: what could a hijacked agent actually steal?
• Best security/reliability improvement: a heldkeys preflight (what the agent can reach) + Tencent AI-Infra-Guard scans (what your AI infrastructure exposes) — incoming exposure and outgoing deployment, both measurable.
• Best Agency OS integration: heldkeys as a SECURITY-stage evidence producer with --strict, BLOCKED on HIGH findings, human waivers only — without touching the gate engine.
• One practical action today: open WSL, install heldkeys, run it in your dev environment, and save the report as your first REAL credential-exposure evidence artifact.