Custom Datasets

The following section introduces how to use EvalScope to customize evaluation datasets, including large model evaluation datasets, multimodal evaluation datasets, embedding models, and CLIP model evaluations.

• Large Language Model
  • Multiple-Choice Question Format (MCQ)
  • Question-Answering Format (QA)
  • (Optional) Custom Evaluation Using the ms-swift Framework
• Multimodal Large Model
  • Multiple-Choice Question Format (MCQ)
  • Custom QA Question Format (VQA)
• Embedding Model
  • Custom Text Retrieval Evaluation
• CLIP Model
  • Custom Image-Text Retrieval Dataset
  • Convert Image-Text Retrieval Data to Text Retrieval Data