Suggest agent skills with Jev: rank 182, verify 3, load at most 1

Agents with big skill rosters choose on almost no information: the roster arrives as an index – one truncated line per skill (Hermes cuts descriptions to 60 characters) – so the skill that edits .pptx files reads nearly the same as the one that authors them, and on turns where nothing fits, a list of names still invites a guess. TypeSafe's skill-suggestion cookbook fixes it with progressive disclosure: one cheap request ranks all 182 skills and asks whether the turn needs one at all; a second request re-reads only the top three in full detail and may reject all of them. The winner's name lands in a single line of the system prompt. This guide summarises the recipe and its measured results.

Why the baseline is bad

Measured over 488 requests against claude-haiku-4-5 (315 covered by exactly one skill, 173 covered by none, including 85 everyday requests and 46 near-misses written to punish guessing like "post this to Mastodon" on a roster covering X but not Mastodon), the agent alone with just its roster loads the wrong skill 16.8% of the time and loads one when nothing fits 9.8%. Of 36 wrong first picks, 10 came from the right skill's own category – the agent looks in roughly the right place, and the hard part is telling lookalikes apart.

Call 1 – skim all 182

One request carries two question kinds: a Choice over all 182 skill names (index description as each option's criteria – the same text the agent gets) whose probabilities are the ranking, plus three Nouls asking different ways whether the turn wants action rather than explanation (act on the user's system? consult a documented procedure? or does prose suffice – inverted). Their mean decides whether anything is suggested at all; under 0.30, stay quiet. Both go out in one round trip.

Call 2 – read the top 3 properly

Three options leave room for each skill's full description plus the opening of its SKILL.md, so the same question goes to better evidence: a Choice over the shortlist, plus one absolute fits::{name} Noul per candidate – does this skill do the specific thing asked? All can come back low, and a shortlist whose best fits score lands under 0.30 is dropped entirely.

The .pptx pair separates exactly here: the wide ranking puts the editing skill first (0.700 vs 0.300) for a deck-authoring request; re-read in full, the Choice flips to the authoring skill. Note the Choice and the Nouls can disagree (here the Nouls still score the editing skill higher) – they answer different questions: the Choice settles which, the Nouls settle whether to say anything at all. This is the same Choice-vs-Noul split as failure mode 8 in the Jev 1.13 jaggedness guide.

The whole recipe is two requests and two thresholds

def suggest(request: str) -> tuple[str, ...]:
    """At most one skill name for a request, or () for "nothing here applies"."""
    wide = rank_wide(request)
    if wide["gate"] < GATE_THRESHOLD:  # 0.30
        return ()
    shortlist = tuple(name for name, _ in wide["ranked"][:SHORTLIST])  # top 3
    result = rerank(request, shortlist, EXCERPT_CHARS)
    if max(result["fits"].values()) < FITS_THRESHOLD:  # 0.30
        return ()
    return (result["winner"],)

The suggestion goes into its own block after the roster, never inside it – so the roster text stays identical every turn and prefix caching still holds:

<skill_relevance>
Relevant to the current request: pptx-author. Ignore this if it does not
fit what the user actually asked for.
</skill_relevance>

The wording does two jobs: it says the suggestion can be ignored (pushing harder wins compliance on wrong suggestions too, and a wrong one is worse than none), and even an empty turn still sends a "nothing appears relevant" sentence, leaving the roster's own "err on the side of loading" instruction opposed rather than unopposed.

Measured results

That is 2.3× fewer wrong loads and 2.4× fewer needless ones – most of the gap between guessing from a truncated index and being handed the answer. Of 315 covered requests, the suggestion fixed 37 and broke 7 the agent had right on its own: a confident wrong suggestion persuades, which is the price of putting one in front of the turn. (The ceiling row is not zero either – an agent handed the right skill still does not always load it.)

Bottom line

Copy the shape for any agent carrying a large roster: cheap ranking over everything, then a close look at two or three – and let either step come back empty-handed. Full source: TypeSafe's skill-suggestion cookbook (runs used jev-1.12); see also intent routing, confidence-gated routing and speculative fan-out in the TypeSafe patterns docs for the same shape elsewhere.