Installation#

Method 1: Install Using pip#

We recommend using conda to manage your environment and installing dependencies with pip:

  1. Create a conda environment (optional)

# It is recommended to use Python 3.10
conda create -n evalscope python=3.10
# Activate the conda environment
conda activate evalscope

  1. Install dependencies using pip

pip install evalscope                # Install Native backend (default)
# Additional options
pip install 'evalscope[opencompass]'   # Install OpenCompass backend
pip install 'evalscope[vlmeval]'       # Install VLMEvalKit backend
pip install 'evalscope[rag]'           # Install RAGEval backend
pip install 'evalscope[perf]'          # Install dependencies for the model performance testing module
pip install 'evalscope[app]'           # Install dependencies for visualization
pip install 'evalscope[all]'           # Install all backends (Native, OpenCompass, VLMEvalKit, RAGEval)

Warning

As the project has been renamed to evalscope, for versions v0.4.3 or earlier, you can install using the following command:

pip install llmuses<=0.4.3

To import relevant dependencies using llmuses:

from llmuses import ...

Method 2: Install from Source#

  1. Download the source code

git clone https://github.com/modelscope/evalscope.git

  1. Install dependencies

cd evalscope/
pip install -e .                  # Install Native backend
# Additional options
pip install -e '.[opencompass]'   # Install OpenCompass backend
pip install -e '.[vlmeval]'       # Install VLMEvalKit backend
pip install -e '.[rag]'           # Install RAGEval backend
pip install -e '.[perf]'          # Install Perf dependencies
pip install -e '.[app]'           # Install visualization dependencies
pip install -e '.[all]'           # Install all backends (Native, OpenCompass, VLMEvalKit, RAGEval)

Docker Image#

You can use the official ModelScope Docker image, which includes the EvalScope library. For more information, refer to here.