Agent Skills Security Scanner: SkillSpector vs Cisco vs Snyk

The star counts say SkillSpector has the attention. The accuracy row says Cisco has the evidence. Those are different things, and a gate should be chosen on the second.

SkillSpector's README lists 71 patterns across 17 categories, from prompt injection and data exfiltration to MCP tool poisoning and YARA signatures. It is also part of NVIDIA's Verified Skills pipeline, which scans and signs skills before they reach NVIDIA's own catalog. Cisco describes its tool as "a best-effort security scanner" and says plainly in its Scope and Limitations section that no findings does not equal no risk and that human review remains essential. Snyk Agent Scan is a different shape of product altogether: it auto-discovers 14 agent clients (Cursor, VS Code, GitHub Copilot, Claude Code, Gemini CLI, Codex and others) across macOS, Linux and Windows, then scores 14 risk indicators from 0 to 1000, four for MCP servers and ten for skills.

This is the table a regulated security team should read first.

Three findings fall out of it.

SkillSpector's default sends your skill to NVIDIA. The README's environment table says SKILLSPECTOR_PROVIDER defaults to nv_build, and the provider code says the same. A developer who sets NVIDIA_INFERENCE_KEY and scans an internal skill with the LLM stage on has sent that skill's analyzer-eligible files to a hosted endpoint. Switching to ollama or openai_compatible fixes that. One detail in the provider list is worth noticing for anyone who reads tool sandboxes for a living: the codex_cli provider is registered but disabled, because its read-only sandbox still permits reads of host files.

SkillSpector is not fully local even with --no-llm. The SC4 supply-chain check queries api.osv.dev with the package names and versions a skill declares. It runs in the static stage, so it runs under --no-llm, and there is no flag to disable it. It sends coordinates, not file contents, and if OSV is unreachable the tool degrades to a built-in fallback list. That is a defensible design (dependency risk in skills is the same surface as hallucinated and squatted packages), but an egress allowlist has to name the host or accept thinner dependency findings.

Snyk Agent Scan is a SaaS scanner with an open-source client. The client is Apache 2.0. The analysis is not something you can host: the README says secrets are redacted before transmission and that the tool does not store or log the contents and results of your MCP tool calls, but skill content itself is transmitted. The --analysis-url flag changes the endpoint, and no local analysis service is documented to point it at. The inspect subcommand lists tools, prompts, resources and skills without requesting analysis, which makes it an inventory, not a scanner, and it still contacts the MCP servers it finds.

Only Cisco publishes numbers you can plan a review queue around. These come from its MaliciousSkillBench source-disjoint test split, measured on the scanner as it shipped in 2.2.0 and still the published figures at 2.3.1.

Cisco's README gives F1 of 75.5% for balanced, 73.5% for low-noise and 64.9% for quiet, and its measured-results doc says never to run quiet without the judge. The practical reading: block at HIGH, review everything at MEDIUM+, and pick balanced unless the reviewer cannot keep up.

Read the judge rows carefully, because the model that produced them is not the one a fresh install uses. The judge was Gemma 4 26B through a hosted endpoint (Bedrock), not the default Claude Sonnet model. So the published recall describes an open-weights model you could serve yourself, and says nothing about the default. Cisco also served Gemma 4 26B-A4B locally on H200s and reproduced the hosted single-pass result: F1 64.2%, precision 81.6%, recall 52.9% locally against 61.2%, 81.5% and 49.0% hosted, using an earlier prompt. For an on-prem buyer, that line settles the deployment question: on Cisco's own numbers, a local judge did not cost recall.

Three more figures shape the configuration:

Cisco also publishes static-only flag rates on about 12,500 unlabelled public skills: 2.46% at HIGH+ and 3.76% at MEDIUM+ for its own scanner against 12.90% and 26.42% for SkillSpector on a 4,379-skill subset. Treat that as what it is: a vendor measuring a rival, on an unstated SkillSpector version, with flag rates that are an upper bound on false positives rather than an error rate. It suggests SkillSpector's static stage is noisy on its own, which is consistent with SkillSpector's own README describing that stage as high recall and moderate precision. It does not tell you which tool catches more malware.

SkillSpector's README claims the LLM stage "improves precision to ~87%". No corpus, method or recall figure accompanies it, so it cannot sit beside Cisco's numbers as a comparable result. Snyk publishes no detection figures at all.

Each scanner has a path where the gate passes something it should not have. A gate is only as good as its handling of that path.

SkillSpector collapses SAFE and CAUTION. Its README says so directly: by default, the exit code collapses SAFE and CAUTION into 0. Under its scoring, a single HIGH finding adds 25 points, which lands in the 21 to 50 CAUTION band and exits 0. That is why the strict flags exist. The second trap is quieter: if the requested LLM is unavailable at preflight, the report says results reflect static analysis only, and the scan carries on. The JSON exposes llm_requested and llm_available (the MCP tool reports llm_used and scan_mode), so a CI step can refuse a static-only result. Check those fields as well as passing --fail-on-incomplete; the docs do not state that the flag alone catches an unavailable provider.

Cisco fails closed on a broken judge, open on a large skill. If --use-llm is set and the judge cannot start, the scan stops with exit code 2, so a CI job with an expired secret fails rather than quietly passing on rules. That is the right default. The gap is per skill: across the 1,384-record split, the judge could not read 92 records (6.6%) on a Haiku arm and 51 (3.7%) on Gemma 4, mostly malicious ones, and 51 and 47 of them respectively still passed the gate. Cisco's own doc notes that a skill can evade the semantic stage by being large enough to exceed the context budget, and points to SkillSpector's completeness check as the better pattern.

Snyk's output is not a stable contract. The README warns against building production workflows on specific output fields or risk names, and separately states that using the standard Agent Scan API for large-scale scanning is considered abuse and will get the account blocked. Both matter if the plan was to loop it over a registry.

If the judge decides recall and the default judge is hosted, the on-prem configuration is the one that matters. Both blocks below keep skill content on infrastructure you control. Neither closes every gap above on flags alone: SkillSpector still needs the llm_available check, and Cisco's judge-unreadable INFO results still need routing to review.

SkillSpector with Ollama. This overrides the nv_build default and fails on any active finding and on partial analysis, not only on a score above 50. The model shown is the README's own Ollama default; substitute the model your organisation has approved. Pair --fail-on-findings with a reviewed suppression file created by skillspector baseline, or the first legitimate finding blocks every build.

# Keep the semantic stage on the host: override the nv_build default
export SKILLSPECTOR_PROVIDER=ollama
export OLLAMA_BASE_URL=http://localhost:11434/v1
export SKILLSPECTOR_MODEL=llama3.1:8b   # or your approved local model

# Fail on any active finding AND on partial analysis, not only on score > 50
skillspector scan ./incoming/some-skill/ \
  --format sarif --output some-skill.sarif \
  --fail-on-incomplete --fail-on-findings

The OSV call still leaves the host with dependency names and versions. Allowlist api.osv.dev for outbound HTTPS, or accept the fallback list and say so in the risk register. Keep the Ollama endpoint on loopback; an Ollama server reachable from the network is its own incident.

Cisco Skill Scanner with a local vLLM server as the judge, gating CI over an internal skills directory. The vLLM server runs separately with gemma-4-26b-a4b as its served model name, matching Cisco's own example. The policy is balanced, per the table above; Cisco's README example uses low-noise, which is the preset to switch to if the review queue outgrows the reviewer.

# Judge on a local OpenAI-compatible server (vLLM), not the hosted default
export SKILL_SCANNER_LLM_PROVIDER="openai-compatible"
export SKILL_SCANNER_LLM_BASE_URL="http://127.0.0.1:8000/v1"
export SKILL_SCANNER_LLM_MODEL="gemma-4-26b-a4b"
export SKILL_SCANNER_LLM_API_KEY="unused"   # required by the route, ignored by vLLM

skill-scanner scan-all ./skills --recursive --use-llm \
  --policy balanced --fail-on-severity high \
  --format sarif --output results.sarif

One server-side setting decides whether this works. Cisco's provider docs report that without disable_any_whitespace set to true in vLLM's structured-outputs configuration, the model emitted a complete JSON object and then padded whitespace to the token limit, losing 46.5% of analyses. With it, the loss was 0.6%. Set it before trusting any recall figure from this setup. If you prefer Ollama, Cisco's ollama/<model> route only accepts a loopback http:// endpoint, so skill content cannot be routed off the host by mistake. For a perimeter with no internet at all, the serving side is covered in our air-gapped vLLM deployment guide.

This is where the regulated answer diverges from the default answer. A rules-only scan is the easy air-gapped option and, at 7.7% at HIGH, close to useless. The real cost of an on-prem skill gate is a GPU that can serve a 26B-class judge, and that line belongs in the policy budget alongside the scanner itself.

The three tools are not substitutes. They answer different questions at different points in a skill's life.

The pre-install gate matters more than it looks, because a project skill's permissions apply whenever it is invoked, regardless of workspace trust. That mechanism is explained in our post on coding agents and auto-loaded repo files; the consequence here is that the scan has to happen before the skill directory exists on the machine, not after.

Two other entrants belong in the picture. JFrog's Skill Scanning scans uploaded .zip skill packages with a SKILL.md at the root and maps malicious to Critical and suspicious to High; it is in beta, requires the AI Catalog Security bundle, and needs Xray 3.145.0 and Artifactory 7.144.2 or later. Mondoo's skillcheck (v0.4.0, Apache 2.0) looks up a content hash of each installed skill in a database the README puts at more than 1,200 analysed skills. It is a useful known-bad check, and by its own README it fails open: a skill the database has not analysed, which includes any modified one, shows as clean.

In June 2026 the Cloud Security Alliance published a research note that summarised work in which five scanning products across three platforms were bypassed, including Cisco's skill-scanner, ClawHub's pipeline and the scanning services on a public skills marketplace, one of which is Snyk's. Four techniques did it:

Three of the four took under an hour to develop. The fourth is the uncomfortable one: the judge that lifts recall from 7.7% to 66.7% is itself an injection target, because it reads attacker-written text by design.

The note does not name SkillSpector among the products bypassed, and SkillSpector ships rules aimed at exactly these shapes: P9 for whitespace padding, SC8 for shipped Python bytecode, SC9 for an executable concealed in a document container. No independent test of those rules is published, so do not read their existence as resistance. The note's own summary is the right policy stance: do not outsource trust to a scanner.

What that means in an install policy:

The scan is one control in a stack, and the rest of that stack is laid out in the AI security pillar.

Pick five developer machines and inventory the skill directories on each, by hand if you have to. Choose one scanner: Cisco if you want published recall and a judge that fails closed, SkillSpector if you want a per-skill gate with SARIF and an MCP entry point. Point its judge at a model on your own hardware using one of the two blocks above, and confirm nothing reaches the hosted default by watching outbound traffic during one scan. Then run it against those five machines' skills and route every MEDIUM+ finding, and every result where the judge did not finish, to one named reviewer. The count of incomplete results in that first run tells you more about your gate than the count of findings.