Home / Skills / huggingface-community-evals
huggingface-community-evals
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs,
- License
- Apache-2.0
- SKILL.md tokens
- 1,638
- Bundled files
- 5
- Bundled tokens (estimated)
- 5,887
Cost per 1,000 loads
Estimated input cost per 1,000 loads of SKILL.md only (characters ÷ 4). Bundled files, output tokens and execution are excluded.
| Model | Input / 1M tokens | 1,000 loads |
|---|---|---|
| Claude Opus 5 | $5.00 | $8.19 |
| Claude Sonnet 5 | $2.00 | $3.28 |
| Claude Haiku 4.5 | $1.00 | $1.64 |
Safety flags
Detected automatically from files; not a security audit.
- Runs scripts
- Yes
- Network access
- No
- Shell commands
- Yes
Works with
Open SKILL.md format
- Claude Code
- Codex
- Cursor
- GitHub Copilot
- Gemini CLI
- OpenCode
- Goose
Install
npx skills add huggingface/skills --skill huggingface-community-evalsLast verified:
huggingface-community-evals FAQ
What does the huggingface-community-evals skill do?
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, Source: https://github.com/huggingface/skills/blob/main/skills/huggingface-community-evals/SKILL.md
What does the huggingface-community-evals skill cost to load?
SKILL.md is about 1,638 tokens. Estimated input cost per 1,000 loads: Claude Opus 5 $8.19; Claude Sonnet 5 $3.28; Claude Haiku 4.5 $1.64. Estimated input cost per 1,000 loads of SKILL.md only (characters ÷ 4). Bundled files, output tokens and execution are excluded. Prices: https://docs.anthropic.com/en/docs/about-claude/pricing
How do I install the huggingface-community-evals skill?
Skills CLI: npx skills add huggingface/skills --skill huggingface-community-evals. Claude Code: /plugin marketplace add huggingface/skills, then install the plugin from that marketplace. Download the huggingface-community-evals skill folder as a ZIP and upload it in Claude.ai skills settings.
Is the huggingface-community-evals skill safe?
Automatically detected flags: runs scripts, shell commands. Flags are detected automatically from the skill's files and are not a security audit; review the SKILL.md before installing.