Your team finds a useful-looking agent skill online. Before anyone installs it, run a short audit. A skill is a folder of instructions, scripts and resources that an agent loads when a task matches it. That convenience is also the risk: a skill can tell an agent to read files, run commands, call outside services or follow a workflow your business would never approve. Auditing does not require you to be a developer. It requires a checklist, a test workspace and the discipline to say no to a skill you cannot explain.
Before you start
You need three things. First, a copy of the skill folder itself, including its SKILL.md file and any scripts or reference files it bundles. Second, a test environment that holds no real customer data, no live passwords and no access to production systems. Third, one named owner on your team who can approve or reject the skill. If no one owns the decision, the skill should not go live.
You also need a simple record. Note the skill name, where you got it, its version or download date, and who reviewed it. That record matters later, when a skill is updated and you must decide whether to audit it again. Treat every update as a new audit, at least for the checks below.
Step 1: Read the SKILL.md file first
Start with the main file. According to Anthropic, a skill is a directory containing a SKILL.md file that starts with YAML frontmatter holding a name and a description. That description is what tells the agent when to use the skill, so read it critically. Does it match what the skill claims to do? Does it trigger on broad phrases that could fire on unrelated work?
Then read the body of the file. Look for instructions about tools, files and outside services. Flag anything that asks the agent to delete files, change settings, send data somewhere new or skip a human check. A good skill states its limits plainly. A vague skill that promises to handle everything is a warning sign, not a selling point.
Step 2: List every permission the skill needs
Make a plain list: which files it reads, which files it writes, which commands it runs and which outside services it contacts. If the skill bundles scripts, open them and check what they actually do. A script that only formats a document is very different from one that uploads files or changes account settings.
Reject or restrict any permission you cannot justify for the job. If a skill for drafting invoices also wants access to your email inbox, ask why. The answer should be in the skill itself. If it is not, contact the author or choose another skill. Never grant a skill broader access just to make a test pass.
Step 3: Check where your data goes
Follow the data, not the marketing. Does the skill send your documents, customer names or internal notes to an outside service? Does it store copies somewhere you do not control? Does it log content that should stay private? For a small business, this check matters more than speed or clever features.
If the skill works only with files you already have open, the risk is lower. If it connects to a third-party service, check that service separately: who runs it, what it stores and how you can remove your data. When you cannot answer those questions, keep the skill in the test environment and do not use it on real work.
Step 4: Test it on fake work, not real work
Create a small test case that looks like real work but contains invented names, numbers and documents. Ask the agent to complete the task with the skill installed, then check three things: did it do the task correctly, did it stay inside the permissions you listed, and did it produce output a human can verify?
Deliberately include one edge case, such as a missing file or an unclear instruction. A good skill asks for clarification or stops safely. A risky skill guesses, fills in invented details or proceeds without telling you. Record what happened, including any surprise actions, even if the final output looked fine.
Step 5: Review the output like a manager, not a fan
Check the output against your own source documents. Confirm names, numbers, dates and recommendations. Confirm the skill did not change wording in a contract, quote or customer message without flagging it. For customer-facing work, add a human approval step before anything is sent, no matter how good the test results look.
Also check for silent side effects. Did any file change that should not have changed? Did the agent create accounts, send messages or update records? If you cannot tell from the test, your logging is not good enough yet, and the skill is not ready for team use.
Step 6: Decide, document and schedule a re-check
Make one of three decisions: approve, approve with limits, or reject. Limits might mean one team only, one document type only, or human approval on every output. Write the decision down with the date and the reviewer name, and tell the team what is and is not allowed.
Set a re-check date. Re-audit when the skill updates, when your tools change or when the skill touches a new kind of data. A skill that was safe for internal drafts may not be safe for customer records, and a skill that was safe last quarter may have changed since.
Tips for small teams
Keep the number of installed skills small. Each skill is something to maintain, so retire skills your team no longer uses. Prefer skills from authors you can identify, with clear documentation and a visible update history. Keep a short list of banned actions for your team, such as sending customer data to unapproved services or running commands that change production systems, and check every new skill against that list.
Frequently asked questions
Do I need to audit a skill from a well-known author?
Yes, but the audit can be shorter. A known author lowers the risk; it does not remove it. Check permissions, data flow and output quality even for trusted sources, because skills change and your use case may differ from the author’s.
How long should an audit take?
For a simple skill, under an hour including a test run. Complex skills with scripts or outside services take longer. If the audit is taking days, that itself is useful information: the skill may be too complex for the value it adds.
What should make me reject a skill outright?
Reject it if you cannot read its instructions or scripts, if it hides what it does, if it sends data somewhere you cannot identify, or if it performs actions you cannot undo. Also reject it if it asks for broad access without explaining why.
Can one person audit and approve a skill alone?
For low-risk internal work, one careful reviewer can be enough. For anything touching customer data, money or public messages, use two reviewers or add a manager approval step. The goal is not bureaucracy; it is a second pair of eyes on irreversible actions.
Sources
Anthropic describes Agent Skills as folders of instructions, scripts and resources that agents discover and load dynamically, using progressive disclosure so only relevant detail loads into context. Read the official explanation before you build or install a skill.



