The skill that documents the threat is the skill that gets blocked

§1 The verdict

On 4 November 2026 Skillverdict checked anthropics/skills · skills/claude-api at commit ca45fc49. The page came back red: every static engine at its blocking verdict, the language-aware LLM review at CAUTION, the hidden-character control at SAFE.

The skill is written in English, so the language artefacts of the scorecard play no part here. This is the other failure mode, and it is language-independent: the engine does not tell a description of a threat from the threat.

§2 The skill

claude-api is Anthropic’s reference skill for agents that write code against the Claude API: a SKILL.md of about 600 lines, per-language guides (python/, typescript/, go/, java/, curl/), shared guides under shared/ (model migration, prompt caching, Managed Agents, the ant CLI) and a few helper scripts for evaluation reports. 96 text files.

Nothing in it downloads or runs anything by itself: it is text an agent reads to call the API correctly. The text talks about security a great deal — how to handle tokens, what never to do with a fetched web page, how a permission system denies a destructive command. Those are the paragraphs that collected most of the findings.

The check took 789 s against a median of 55 s that day; almost all of it was SkillSpector. The other three engines finished inside two minutes. Reproduced locally at the same commit the run takes 3 min 27 s of pure CPU: the engine’s static pattern analyzers walk 2.3 MB of Markdown, and on 17 files they stop at a bounded parser’s span limit (static_parse_limit in its own ledger). Those are the 20 “partially inspected” files, and they are what the 53 AE1 findings point at.

§3 What fired

288 findings across the four scanners; five rules account for two thirds of them. The middle column is the engine’s own wording.

One paragraph — the one that forbids running curl | sh from a fetched page — produced a HIGH or CRITICAL finding in all four engines.

§4 Five findings, read in context

The mechanism is the same each time: a static scanner looks for signatures of dangerous actions and finds them in text that describes, forbids or shows the blocking of those actions.

1. A cross-reference is an “uninspected artifact” (SkillSpector AE1, 53 × HIGH). SKILL.md refers to its own guides hundreds of times — “see shared/model-migration.md”, “read shared/prompt-caching.md”. The engine skipped 20 of 96 files (79 % coverage) and then reported every reference to a skipped file as a separate HIGH finding. Two thirds of its HIGH count is one rule about its own incomplete read.
2. The word “Caution” is an instruction to omit warnings (SkillSpector AR2, HIGH). A paragraph opens with “Caution — the runner does not auto-resume pause_turn” and warns about a silently truncated answer. A rule tuned for phrases like “without warnings” fired on the warning itself.
3. An example of a denied command is “parameter abuse” (SkillSpector TM1, 3 × HIGH). The permission-system documentation shows an event: the agent requested rm -rf /workspace/reports, the system answered evaluated_permission: deny. The engine saw rm -rf, did not see deny, and reported the same line three times.
4. Forbidding curl | sh is remote code execution (SkillSpector SC2, Cisco PIPELINE_TAINT_FLOW, Swarms DC002, skill-detector SD-009). The onboarding guide says a fetched page is data, not instructions: never run its commands or its curl | sh lines. It is the most protective sentence in the skill, and the only place where all four scanners agree.
5. A defence against key theft is key theft (SkillSpector PE3, Swarms CA001 and PS001, skill-detector SD-013). A comment in build-report-lite.mjs explains why file reads refuse symbolic links: a prompt-injected agent could plant summary.json → ~/.ssh/id_rsa. The mention of ~/.ssh/id_rsa in that comment became “credential access” for three engines and a “persistence mechanism” for two.

The remaining large groups follow the same logic: 53 “data sent to an external URL” are example calls to the Claude API in a skill about the Claude API; 17 “Base64 obfuscations” describe the field through which the API accepts a PDF; the YARA “hack tool” match is a table of changes between model versions.

§5 The LLM review

The language-aware review (Claude Haiku 5.5, prompt v2) read the same files and returned three findings, none above MEDIUM:

All three are observations about what the skill makes the agent do, not about words in it, and the first is worth a conversation with the authors. Nothing fired on “Caution”, on curl | sh or on ~/.ssh/id_rsa: the model told a prohibition from an action and an example from a command.

§6 What it adds to the scorecard

The scorecard currently names one class of known artefacts: language (Cyrillic, CJK, German). This case adds a second, security documentation. The more carefully a skill explains what the agent must not do, the more it resembles a threat to a signature scanner. Vendor and platform skills are written exactly that way, so they will come back red systematically.

• The benchmark corpus gets a sub-class of legitimate skills that document security — claude-api and a handful of other official platform skills (license checked before any copy; otherwise a URL + SHA pointer).
• The scorecard gets a column for false positives on documentation, per engine, next to the language columns.
• Skill pages mark AE1 (reference to a file inside the skill) and AR2 (the word Caution) as known artefacts, where the cause is unambiguous.

A scanner that cannot read “never run this” without reporting “runs this” will block every skill that teaches an agent to be careful.

§7 Reproduce

Every version is recorded inside the verdict itself; the engines are pinned in benchmark/engines.lock.yaml. The engines’ full outputs are kept with the verdict and quoted on the page as they came.