MusicTrivia#

Overview#

MusicTrivia is a curated multiple-choice benchmark for evaluating AI models on music knowledge. It covers both classical and modern music topics including composers, musical periods, instruments, and popular artists.

Task Description#

Task Type: Multiple-Choice Question Answering
Input: Music-related trivia question with multiple choice options
Output: Selected correct answer
Domains: Classical music, modern music, music history

Key Features#

Comprehensive coverage of music domains
Questions about composers, periods, and artists
Tests factual recall and domain knowledge
Curated for quality and accuracy
Balanced difficulty across topics

Evaluation Notes#

Default configuration uses 0-shot evaluation
Primary metric: Accuracy
Evaluates on test split
Uses standard single-answer multiple-choice template

Properties#

Property	Value
Benchmark Name	`music_trivia`
Dataset ID	extraordinarylab/music-trivia
Paper	N/A
Tags	`Knowledge`, `MCQ`
Metrics	`acc`
Default Shots	0-shot
Evaluation Split	`test`

Data Statistics#

Metric	Value
Total Samples	512
Prompt Length (Mean)	412.5 chars
Prompt Length (Min/Max)	256 / 1005 chars

Sample Example#

Subset: default

{
  "input": [
    {
      "id": "d16392be",
      "content": "Answer the following multiple choice question. The entire content of your response should be of the following format: 'ANSWER: [LETTER]' (without quotes) where [LETTER] is one of A,B,C,D.\n\nBeethoven's third period was distinct from his second on account of all the following factors EXCEPT\n\nA) The influence of Italian opera on his compositions\nB) The occupation of Vienna by French troops\nC) The introduction of new instruments in the orchestra\nD) The emergence of Romanticism in music"
    }
  ],
  "choices": [
    "The influence of Italian opera on his compositions",
    "The occupation of Vienna by French troops",
    "The introduction of new instruments in the orchestra",
    "The emergence of Romanticism in music"
  ],
  "target": "B",
  "id": 0,
  "group_id": 0,
  "metadata": {}
}

Prompt Template#

Prompt Template:

Answer the following multiple choice question. The entire content of your response should be of the following format: 'ANSWER: [LETTER]' (without quotes) where [LETTER] is one of {letters}.

{question}

{choices}

Usage#

Using CLI#

evalscope eval \
    --model YOUR_MODEL \
    --api-url OPENAI_API_COMPAT_URL \
    --api-key EMPTY_TOKEN \
    --datasets music_trivia \
    --limit 10  # Remove this line for formal evaluation

Using Python#

from evalscope import run_task
from evalscope.config import TaskConfig

task_cfg = TaskConfig(
    model='YOUR_MODEL',
    api_url='OPENAI_API_COMPAT_URL',
    api_key='EMPTY_TOKEN',
    datasets=['music_trivia'],
    limit=10,  # Remove this line for formal evaluation
)

run_task(task_cfg=task_cfg)