Home / Open source / lmms-eval
lmms-eval
EvolvingLMMs-Lab/lmms-eval · Enterprise stack · Enterprise
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
Verified via GitHub API
- agi
- evaluation
- large-language-models
- multimodal
- audio-evaluation
- benchmark
- llm-evaluation
- video-understanding
- vision-language-model
- vlm
- multimodal-evaluation
Data from GitHub API, refreshed Oct 17, 2026
- Stars
- 4453
- Forks
- 676
- License
- Other
- Language
- Python
- Last commit
- Oct 17, 2026
- Latest release
- v0.7.3
- Release date
- Aug 28, 2026
- Contributors
- 247
- Open issues
- 28
Works with — from README
- Models
- Claude, GPT-4o, Gemini, OpenAI, Qwen
Source: README.md keyword match (stated in README)
lmms-eval FAQ
Is lmms-eval free?
lmms-eval is open source under the Other license. Hosted services and infrastructure may have separate costs.
How popular is lmms-eval?
lmms-eval has 4453 GitHub stars as of Oct 17, 2026.
Is there a hosted version of lmms-eval?
A hosted alternative for lmms-eval is unknown (—) in this data.
What does lmms-eval work with?
From README: Claude, GPT-4o, Gemini, OpenAI, Qwen.