Anatomy of an Agent Skill

Most skill failures are silent.

The skill doesnt crash, and the instructions inside it are solid. The problem is simpler and more frustrating: the agent never picked it.

That is the mental model shift. A skill is not just instructions, it is also a routing artifact in the agentic loop.

1. It needs to get selected.
2. Then it needs to execute well.

A Skill Has Three Jobs

At a high level, every skill has three jobs:

1. It advertises itself so the agent can decide whether to load it.
2. It guides execution once loaded.
3. It brings tools and assets the agent can use to finish the task.

Those jobs map to three parts:

• description in frontmatter (selection)
• SKILL.md body (workflow and constraints)
• supporting files in references/, assets/, and scripts/ (execution power)

When one part is weak, the whole skill feels unreliable.

The Description Is the Routing Layer

If you only improve one thing, improve the description.

In most setups, agents see skill names and descriptions first. They do not preload every SKILL.md. So your description is the gatekeeper for whether the real instructions are even loaded.

The description optimization guide makes the tradeoff clear: if the description is too narrow, the skill gets missed. If it is too broad, it fires when it should not.

What strong descriptions do

• Use imperative phrasing: “Use this skill when…”
• Describe user intent: what the user is trying to do
• Include adjacent phrasing: prompts that imply the domain without naming it
• Define boundaries: where this skill should not trigger

Weak descriptions read like labels. Strong descriptions read like trigger rules.

| Weak description | Better description | | --- | --- | | Process CSV files. | Use this skill when a user wants to clean, analyze, or visualize tabular data from CSV, TSV, or Excel files, including requests that do not explicitly say “analysis.” Do not use this for ETL jobs that write to databases. |

The second version does two important things: it says when to load and when not to load. That second part is where most skills fail.

Measure Selection, Then Improve It

For critical skills, you should test selection the same way you test code paths.

You need evals that answer two basic questions:

• Did this prompt trigger the skill when it should?
• Did this prompt avoid triggering the skill when it should not?

The optimization workflow recommends realistic eval sets with both positives and negatives, especially near misses.

I built Skills Dojo for this. It is a CLI for evaluating both skill selection and skill effectiveness.

To use it, install the CLI and all you need to do is add a selection.yaml under your skill folder:

evals:
  - name: should-select-code-review
    prompt: "Review this pull request for potential security issues and suggest improvements."

  - name: should-not-select-code-review
    prompt: "Write a Python function that calculates the Fibonacci sequence."
    assert: none

To evaluate, simply run:

dojo run

For description tuning, Dojo supports variants and decoys so you can compare wording changes and test whether the model can distinguish your skill from similar options.

Two details matter a lot in practice:

1. Run each query multiple times. Selection is probabilistic, so use trigger rate, not one-shot pass/fail.
2. Use train and validation splits. Tune on train, choose the best description by validation performance to avoid overfitting.

Dojo handles execution and reporting. Train and validation splits, plus repeated runs, are methodology you apply on top. Keep separate eval files for each set, run them repeatedly, and compare report outcomes.

This is the loop I keep coming back to:

1. Measure current trigger behavior
2. Inspect misses and false triggers
3. Rewrite description boundaries
4. Re-run evals
5. Keep the variant that generalizes best

Measure first, improve second.

Build SKILL.md Around Reusable Procedures

Once a skill is selected, it has to be usable.

The best practices guide pushes for procedures over declarations, and that matches real-world results.

This is weak:

Handle errors carefully and follow security best practices.

This is usable:

1. Validate input schema against `references/schema.json`.
2. Run `scripts/check-permissions.sh` before write operations.
3. If validation fails, return errors and stop.
4. Only continue after all checks pass.

The second version is executable. The first version is just advice.

Keep scope coherent

A skill should represent one coherent unit of work.

• Good: “Analyze tabular data and produce summary outputs.”
• Too broad: “Analyze data, manage database infrastructure, and handle incident response.”

If a skill does too many unrelated things, selection quality drops and maintenance gets messy.

Treat Gotchas as a Living Memory Layer

One of the highest-leverage sections in any skill is ## Gotchas.

Gotchas are not generic reminders. They are specific corrections for mistakes the agent will make in your environment if you do not warn it.

## Gotchas

- The `/health` endpoint only checks process liveness. Use `/ready` for dependency health.
- `account_id` in the DB maps to `customerId` in the billing API.
- Soft deletes are enabled. Add `deleted_at IS NULL` unless historical rows are required.

The best practices doc calls this out directly, and it is worth turning into a habit: when you correct the agent, add that correction to Gotchas.

That is how a skill gets better over time instead of fossilizing.

Right Size Your Skill

A good skill is not tiny, and it is not a full handbook.

You need enough detail to prevent common failures, but not so much detail that the agent drags irrelevant context into every run.

Practical rule of thumb:

• Keep SKILL.md focused on always-needed instructions
• Move detailed material into on-demand files
• Tell the agent exactly when to open those files

This is where progressive disclosure pays off.

Use the Skill Folder as a System, Not a Text File

Skills are directories for a reason.

my-skill/
  SKILL.md
  references/
    api-errors.md
    schema.md
  assets/
    report-template.md
  scripts/
    validate.py
    summarize.ts
  evals/
    selection.yaml

Each folder has a clear purpose:

• references/: detailed docs loaded only when needed
• assets/: reusable templates and output shapes
• scripts/: tested executable logic

You get a stronger skill without front-loading everything into the initial context window.

Be explicit about disclosure triggers

Avoid vague instructions like “see references for more.”

Write instructions like:

If any API response is non-200, read `references/api-errors.md` and map the code before retrying.
If output formatting is requested, load `assets/report-template.md` and follow that structure.

This keeps context loading targeted and predictable.

CLI + Skills Is a Strong Pattern

The scripts guide is especially useful because it turns skills into operational systems, not just text instructions.

A reliable pattern is:

1. Skill defines decision logic and workflow
2. Scripts handle repeatable, brittle operations
3. Agent orchestrates using both

This is usually better than asking the model to recreate parsing or validation logic from scratch every run.

Script design rules that prevent pain

• No interactive prompts
• Clear --help usage
• Structured output (JSON/CSV)
• Data on stdout, diagnostics on stderr
• Idempotent behavior where possible
• Safe defaults and dry-run for risky actions

If a command is easy to mess up, hide it behind a tested script and call that script from the skill.

Common Failure Modes to Watch For

The description says what, but not when

If the description reads like a noun phrase, selection quality will be unstable.

Missing negative boundaries

Without “do not use this for X,” skills over-trigger in adjacent domains.

Generic advice instead of executable procedure

“Follow best practices” sounds helpful but offers no operational control.

Gotchas never get updated

A stale Gotchas section means you keep paying the same mistake tax.

Everything stuffed into SKILL.md

Large unstructured skill bodies inflate context and reduce focus.

No eval loop for critical skills

If selection matters and you are not measuring it, reliability is a guess.

A Practical Skill Skeleton

When you start a new skill, this baseline structure is a solid default:

---
name: my-skill
description: >
  Use this skill when ...
  Do not use this skill when ...
---

## When to use
- ...

## When not to use
- ...

## Workflow
1. ...
2. ...

## Gotchas
- ...

## Validation loop
1. Run ...
2. If failure, fix and retry ...

## Scripts
- `scripts/...`

## Progressive disclosure triggers
- If X, read `references/...`

Then add selection evals right away, not six weeks later when reliability issues show up.

Final Thought

An agent skill is not just prompt text. It is a small system with routing logic, execution guidance, and reusable assets.

If you remember one thing, make it this: the description controls whether the system activates. That is why description quality and selection evals deserve real engineering effort, especially for skills in critical workflows.

Build the skill, measure how it triggers, and update it using real misses and false positives. That is how you move from an interesting artifact to a reliable tool.