Can foundation models label data like humans?

This blog post examines the reliability of using large language models (LLMs) for data labeling and evaluation, comparing them to human labelers. The Hugging Face team expands the Open LLM Leaderboard to include automated academic benchmarks, professional human labels, and GPT-4 evaluations. They analyze where LLM-based labels can be trusted and where they fall short, providing insights into the calibration of automated evaluation methods for instruction-tuned models.