安装指南¶

npu-smi info

    === "A2"

        #### 拉取镜像 {#pull-the-image}

        ??? tip "如果镜像下载缓慢"

            vLLM Ascend 镜像默认从 `quay.io` 下载。如果直接访问速度较慢，请使用以下任一镜像仓库来加速下载。

            例如，原始镜像地址为：

            ```text
            quay.io/ascend/vllm-ascend:<TAG>
            ```

            您可以将其替换为：

            ```text
            # Replace with tag you want to pull
            TAG=v0.23.0
            # use
            docker pull m.daocloud.io/quay.io/ascend/vllm-ascend:$TAG
            # or
            docker pull quay.nju.edu.cn/ascend/vllm-ascend:$TAG
            ```

            仅替换镜像仓库前缀，并保留完整的原始镜像标签，包括 `-a3`、`-310p`、`-950dt` 和 `-openeuler` 等后缀。


        === "Ubuntu"

            <span id="quick-start-atlas-a2-ubuntu"></span>

            <!-- doctest: quickstart-image-a2-ubuntu -->
            ```bash
            export IMAGE=quay.io/ascend/vllm-ascend:v0.23.0
            docker pull "$IMAGE"
            ```

        === "openEuler"

            <span id="quick-start-atlas-a2-openeuler"></span>

            <!-- doctest: quickstart-image-a2-openeuler -->
            ```bash
            export IMAGE=quay.io/ascend/vllm-ascend:v0.23.0-openeuler
            docker pull "$IMAGE"
            ```

        #### 启动容器 {: #quick-start-atlas-a2-container #start-the-container }

        === "Ubuntu"

            ```bash
            export DEVICE=/dev/davinci0
            export MODEL_CACHE="${HOME}/.cache"

            mkdir -p "$MODEL_CACHE"

            docker run --rm \
                --name vllm-ascend \
                --shm-size=1g \
                --device "$DEVICE" \
                --device /dev/davinci_manager \
                --device /dev/devmm_svm \
                --device /dev/hisi_hdc \
                -v /usr/local/dcmi:/usr/local/dcmi \
                -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
                -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
                -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
                -v /etc/ascend_install.info:/etc/ascend_install.info \
                -v "$MODEL_CACHE:/root/.cache" \
                -p 8000:8000 \
                -it "$IMAGE" bash
            ```

        === "openEuler"

            ```bash
            export DEVICE=/dev/davinci0
            export MODEL_CACHE="${HOME}/.cache"

            mkdir -p "$MODEL_CACHE"

            docker run --rm \
                --name vllm-ascend \
                --shm-size=1g \
                --device "$DEVICE" \
                --device /dev/davinci_manager \
                --device /dev/devmm_svm \
                --device /dev/hisi_hdc \
                -v /usr/local/dcmi:/usr/local/dcmi \
                -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
                -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
                -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
                -v /etc/ascend_install.info:/etc/ascend_install.info \
                -v "$MODEL_CACHE:/root/.cache" \
                -p 8000:8000 \
                -it "$IMAGE" bash
            ```


        #### 验证容器环境 {#verify-the-container-environment}

        在容器中运行以下命令。当输出包含 `vLLM Ascend environment: OK` 时，表示容器已就绪。

        <!-- doctest: quickstart-container-verify -->
        ```bash
        npu-smi info

        python3 - <<'PY'
        import torch
        import vllm
        import vllm_ascend

        assert torch.npu.is_available(), "No available Ascend NPU detected in the container"
        print("vLLM Ascend environment: OK")
        PY
        ```


    === "A3"

        #### 拉取镜像 {#pull-the-image}

        ??? tip "如果镜像下载缓慢"

            vLLM Ascend 镜像默认从 `quay.io` 下载。如果直接访问速度较慢，请使用以下任一镜像仓库来加速下载。

            例如，原始镜像地址为：

            ```text
            quay.io/ascend/vllm-ascend:<TAG>
            ```

            您可以将其替换为：

            ```text
            # Replace with tag you want to pull
            TAG=v0.23.0
            # use
            docker pull m.daocloud.io/quay.io/ascend/vllm-ascend:$TAG
            # or
            docker pull quay.nju.edu.cn/ascend/vllm-ascend:$TAG
            ```

            仅替换镜像仓库前缀，并保留完整的原始镜像标签，包括 `-a3`、`-310p`、`-950dt` 和 `-openeuler` 等后缀。


        === "Ubuntu"

            <span id="quick-start-atlas-a3-ubuntu"></span>

            ```bash
            export IMAGE=quay.io/ascend/vllm-ascend:v0.23.0-a3
            docker pull "$IMAGE"
            ```

        === "openEuler"

            <span id="quick-start-atlas-a3-openeuler"></span>

            ```bash
            export IMAGE=quay.io/ascend/vllm-ascend:v0.23.0-a3-openeuler
            docker pull "$IMAGE"
            ```

        ???+ warning "A3 容器启动要求"

            A3 采用双 DIE 设计，需要两个 Ascend 设备节点，例如 `/dev/davinci0` 和 `/dev/davinci1`。

        === "Ubuntu"

            ```bash
            export DEVICE0=/dev/davinci0
            export DEVICE1=/dev/davinci1
            export MODEL_CACHE="${HOME}/.cache"

            mkdir -p "$MODEL_CACHE"

            docker run --rm \
                --name vllm-ascend \
                --shm-size=1g \
                --device "$DEVICE0" \
                --device "$DEVICE1" \
                --device /dev/davinci_manager \
                --device /dev/devmm_svm \
                --device /dev/hisi_hdc \
                -v /usr/local/dcmi:/usr/local/dcmi \
                -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
                -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
                -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
                -v /etc/ascend_install.info:/etc/ascend_install.info \
                -v "$MODEL_CACHE:/root/.cache" \
                -p 8000:8000 \
                -it "$IMAGE" bash
            ```

        === "openEuler"

            ```bash
            export DEVICE0=/dev/davinci0
            export DEVICE1=/dev/davinci1
            export MODEL_CACHE="${HOME}/.cache"

            mkdir -p "$MODEL_CACHE"

            docker run --rm \
                --name vllm-ascend \
                --shm-size=1g \
                --device "$DEVICE0" \
                --device "$DEVICE1" \
                --device /dev/davinci_manager \
                --device /dev/devmm_svm \
                --device /dev/hisi_hdc \
                -v /usr/local/dcmi:/usr/local/dcmi \
                -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
                -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
                -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
                -v /etc/ascend_install.info:/etc/ascend_install.info \
                -v "$MODEL_CACHE:/root/.cache" \
                -p 8000:8000 \
                -it "$IMAGE" bash
            ```


        #### 验证容器环境 {#verify-the-container-environment}

        在容器中运行以下命令。当输出包含 `vLLM Ascend environment: OK` 时，表示容器已就绪。

        <!-- doctest: quickstart-container-verify -->
        ```bash
        npu-smi info

        python3 - <<'PY'
        import torch
        import vllm
        import vllm_ascend

        assert torch.npu.is_available(), "No available Ascend NPU detected in the container"
        print("vLLM Ascend environment: OK")
        PY
        ```


    === "Atlas 300I DUO"

        #### 拉取镜像 {#pull-the-image}

        ??? tip "如果镜像下载速度较慢"

            vLLM Ascend 镜像默认从 `quay.io` 下载。如果直接访问速度较慢，请使用以下任一镜像仓库来加速下载。

            例如，原始镜像地址为：

            ```text
            quay.io/ascend/vllm-ascend:<TAG>
            ```

            您可以将其替换为：

            ```text
            # Replace with tag you want to pull
            TAG=v0.23.0
            # use
            docker pull m.daocloud.io/quay.io/ascend/vllm-ascend:$TAG
            # or
            docker pull quay.nju.edu.cn/ascend/vllm-ascend:$TAG
            ```

            仅替换注册表前缀，并保留完整的原始镜像标签，包括 `-a3`、`-310p`、`-950dt` 和 `-openeuler` 等后缀。


        === "Ubuntu"

            <span id="quick-start-atlas-300i-duo-ubuntu"></span>

            <!-- doctest: quickstart-image-300i-duo-ubuntu -->
            ```bash
            export IMAGE=quay.io/ascend/vllm-ascend:v0.23.0-310p
            docker pull "$IMAGE"
            ```

        === "openEuler"

            <span id="quick-start-atlas-300i-duo-openeuler"></span>

            <!-- doctest: quickstart-image-300i-duo-openeuler -->
            ```bash
            export IMAGE=quay.io/ascend/vllm-ascend:v0.23.0-310p-openeuler
            docker pull "$IMAGE"
            ```

        #### 启动容器 {: #quick-start-atlas-300i-duo-container }

        === "Ubuntu"

            ```bash
            export DEVICE=/dev/davinci0
            export MODEL_CACHE="${HOME}/.cache"

            mkdir -p "$MODEL_CACHE"

            docker run --rm \
                --name vllm-ascend \
                --shm-size=1g \
                --device "$DEVICE" \
                --device /dev/davinci_manager \
                --device /dev/devmm_svm \
                --device /dev/hisi_hdc \
                -v /usr/local/dcmi:/usr/local/dcmi \
                -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
                -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
                -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
                -v /etc/ascend_install.info:/etc/ascend_install.info \
                -v "$MODEL_CACHE:/root/.cache" \
                -p 8000:8000 \
                -it "$IMAGE" bash
            ```

        === "openEuler"

            ```bash
            export DEVICE=/dev/davinci0
            export MODEL_CACHE="${HOME}/.cache"

            mkdir -p "$MODEL_CACHE"

            docker run --rm \
                --name vllm-ascend \
                --shm-size=1g \
                --device "$DEVICE" \
                --device /dev/davinci_manager \
                --device /dev/devmm_svm \
                --device /dev/hisi_hdc \
                -v /usr/local/dcmi:/usr/local/dcmi \
                -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
                -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
                -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
                -v /etc/ascend_install.info:/etc/ascend_install.info \
                -v "$MODEL_CACHE:/root/.cache" \
                -p 8000:8000 \
                -it "$IMAGE" bash
            ```

        在容器中运行以下命令。当输出包含 `vLLM Ascend environment: OK` 时，容器已准备就绪。

        <!-- doctest: quickstart-container-verify -->
        ```bash
        npu-smi info

        python3 - <<'PY'
        import torch
        import vllm
        import vllm_ascend

        assert torch.npu.is_available(), "No available Ascend NPU detected in the container"
        print("vLLM Ascend environment: OK")
        PY
        ```


    === "Atlas 200I Pro"

        #### 拉取镜像 {#pull-the-image}

        ??? tip "如果镜像下载缓慢"

            vLLM Ascend 镜像默认从 `quay.io` 下载。如果直接访问缓慢，请使用以下任一镜像仓库以加速下载。

            例如，原始镜像地址为：

            ```text
            quay.io/ascend/vllm-ascend:<TAG>
            ```

            您可以将其替换为：

            ```text
            # Replace with tag you want to pull
            TAG=v0.23.0
            # use
            docker pull m.daocloud.io/quay.io/ascend/vllm-ascend:$TAG
            # or
            docker pull quay.nju.edu.cn/ascend/vllm-ascend:$TAG
            ```

            仅替换镜像仓库前缀，并保留完整的原始镜像标签，包括诸如 `-a3`、`-310p`、`-950dt` 和 `-openeuler` 等后缀。


        === "Ubuntu"

            <span id="quick-start-atlas-200i-pro-ubuntu"></span>

            ```bash
            export IMAGE=quay.io/ascend/vllm-ascend:v0.23.0-310p
            docker pull "$IMAGE"
            ```

        === "openEuler"

            <span id="quick-start-atlas-200i-pro-openeuler"></span>

            ```bash
            export IMAGE=quay.io/ascend/vllm-ascend:v0.23.0-310p-openeuler
            docker pull "$IMAGE"
            ```

        #### 启动容器 {: #quick-start-atlas-200i-pro-container #start-the-container }

        ???+ warning "Atlas 200I Pro 容器启动要求"

            Atlas 200I Pro 需要额外的设备节点、驱动库和主机配置文件。在启动容器之前，请确保以下命令挂载的所有主机路径均存在。

        === "Ubuntu"

            ```bash
            export MODEL_CACHE="${HOME}/.cache"

            mkdir -p "$MODEL_CACHE"

            docker run --rm \
                --privileged \
                --name vllm-ascend \
                --shm-size=10g \
                --device=/dev/davinci0:/dev/davinci0 \
                --device=/dev/davinci_manager \
                --device=/dev/ascend_manager \
                --device=/dev/user_config \
                -v /etc/sys_version.conf:/etc/sys_version.conf \
                -v /etc/ld.so.conf.d/mind_so.conf:/etc/ld.so.conf.d/mind_so.conf \
                -v /etc/hdcBasic.cfg:/etc/hdcBasic.cfg \
                -v /var/dmp_daemon:/var/dmp_daemon \
                -v /usr/lib64/libmmpa.so:/usr/lib64/libmmpa.so \
                -v /usr/lib64/libcrypto.so.1.1:/usr/lib64/libcrypto.so.1.1 \
                -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi \
                -v /usr/lib64/libstackcore.so:/usr/lib64/libstackcore.so \
                -v /usr/lib/aarch64-linux-gnu/libyaml-0.so.2:/usr/lib64/libyaml-0.so.2 \
                -v /etc/slog.conf:/etc/slog.conf \
                -v /var/slogd:/var/slogd \
                -v /usr/local/Ascend/driver/lib64:/usr/local/Ascend/driver/lib64 \
                -v /usr/lib64/libtensorflow.so:/usr/lib64/libtensorflow.so \
                -v "$MODEL_CACHE:/root/.cache" \
                -p 8000:8000 \
                -it "$IMAGE" bash
            ```

        === "openEuler"

            ```bash
            export MODEL_CACHE="${HOME}/.cache"

            mkdir -p "$MODEL_CACHE"

            docker run --rm \
                --privileged \
                --name vllm-ascend \
                --shm-size=10g \
                --device=/dev/davinci0:/dev/davinci0 \
                --device=/dev/davinci_manager \
                --device=/dev/ascend_manager \
                --device=/dev/user_config \
                -v /etc/sys_version.conf:/etc/sys_version.conf \
                -v /etc/ld.so.conf.d/mind_so.conf:/etc/ld.so.conf.d/mind_so.conf \
                -v /etc/hdcBasic.cfg:/etc/hdcBasic.cfg \
                -v /var/dmp_daemon:/var/dmp_daemon \
                -v /usr/lib64/libsemanage.so.2:/usr/lib64/libsemanage.so.2 \
                -v /usr/lib64/libmmpa.so:/usr/lib64/libmmpa.so \
                -v /usr/lib64/libcrypto.so.1.1:/usr/lib64/libcrypto.so.1.1 \
                -v /usr/lib64/libyaml-0.so.2.0.9:/usr/lib64/libyaml-0.so.2 \
                -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi \
                -v /usr/lib64/libstackcore.so:/usr/lib64/libstackcore.so \
                -v /etc/slog.conf:/etc/slog.conf \
                -v /var/slogd:/var/slogd \
                -v /usr/local/Ascend/driver/lib64:/usr/local/Ascend/driver/lib64 \
                -v /usr/lib64/libtensorflow.so:/usr/lib64/libtensorflow.so \
                -v "$MODEL_CACHE:/root/.cache" \
                -p 8000:8000 \
                -it "$IMAGE" bash
            ```


        #### 验证容器环境 {#verify-the-container-environment}

        在容器中运行以下命令。当输出包含 `vLLM Ascend environment: OK` 时，容器已准备就绪。

        <!-- doctest: quickstart-container-verify -->
        ```bash
        npu-smi info

        python3 - <<'PY'
        import torch
        import vllm
        import vllm_ascend

        assert torch.npu.is_available(), "No available Ascend NPU detected in the container"
        print("vLLM Ascend environment: OK")
        PY
        ```

    安装其他版本的 `vllm-ascend` 时，请查看该仓库的 `requirements.txt`，以确定对应的 `triton-ascend` 版本。

=== "Atlas 300I DUO / Atlas 200I Pro"

    Atlas 300I DUO 和 Atlas 200I Pro 不支持 `triton` 或 `triton-ascend`。如果环境中已安装其中任一包，请将其卸载，并且不要重新安装 `triton-ascend`：

    ```bash
    pip uninstall -y triton triton-ascend
    ```

        此选项会告诉 pip 即使 HTTPS 验证失败也信任该主机，这会削弱对中间人攻击的防护。仅在你信任网络且无法修复证书配置时使用。

    检查设备构建类型：

    <!-- doctest: installation-pip-device-check -->
    ```bash
    python - <<'PY'
    from vllm_ascend._build_info import __device_type__

    print("vLLM Ascend wheel device type:", __device_type__)
    assert __device_type__ == "A2", __device_type__
    PY
    ```

=== "uv 安装"

    WheelNext 会从 variant 索引中选择与硬件匹配的 vLLM Ascend wheel。首先，安装并验证 `uv`：

    <!-- doctest: installation-uv-bootstrap -->
    ```bash
    # install uv-wheelnext
    curl -LsSf https://astral.sh/uv/install.sh | sed 's/verify_checksum "$_file"/true/' | INSTALLER_DOWNLOAD_URL=https://wheelnext.astral.sh sh
    source $HOME/.local/bin/env
    ```

    <!-- doctest: installation-uv-install -->
    ```bash
    # Install vllm-project/vllm. The newest supported version is v0.23.0.
    pip install "vllm==0.23.0"

    # Install vllm-project/vllm-ascend from wheelnext index.
    uv pip install --system \
        --extra-index-url https://mirrors.huaweicloud.com/ascend/repos/pypi/variant \
        --index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple \
        --find-links https://mirrors.huaweicloud.com/ascend/repos/pypi/triton-ascend/ \
        "vllm-ascend==0.23.0"
    ```

    ??? tip "如果 uv 安装失败，请清除缓存"

        如果 `uv pip install` 因缓存损坏或过期的包数据而失败，请在重新运行安装命令之前清除 uv 缓存：

            uv cache clean

    检查设备构建类型：

    <!-- doctest: installation-uv-device-check -->
    ```bash
    python - <<'PY'
    from vllm_ascend._build_info import __device_type__
    print("vLLM Ascend wheel device type:", __device_type__)
    PY
    ```

=== "源码安装"

    ???+ warning "A3 源码安装"

        当为 A3 从源码安装 vLLM Ascend 时，请手动运行 `git submodule update --init --recursive`，或确保环境可以访问互联网，以便自动下载所需的子模块。

    先安装 vLLM，然后安装 vLLM Ascend：

    <!-- doctest: installation-source-install -->
    ```bash
    # Install vLLM.
    git clone --depth 1 --branch v0.23.0 https://github.com/vllm-project/vllm
    cd vllm
    VLLM_TARGET_DEVICE=empty pip install -e . \
        --extra-index-url https://download.pytorch.org/whl/cpu/
    cd ..

    # Install vLLM Ascend.
    git clone --depth 1 --branch v0.23.0 https://github.com/vllm-project/vllm-ascend.git
    cd vllm-ascend
    # git submodule update --init --recursive
    pip install -e . \
        --extra-index-url https://download.pytorch.org/whl/cpu/ \
        --extra-index-url https://mirrors.huaweicloud.com/ascend/repos/pypi
    cd ..
    ```

最后，根据硬件处理 `triton` 和 `triton-ascend`：

=== "A2 / A3 / 950DT 产品"

    为确保 Triton Ascend 与当前 vLLM Ascend 版本匹配，在安装相应版本的 `triton-ascend` 之前，请卸载任何已存在的 `triton` 和 `triton-ascend` 包：

    <!-- doctest: installation-post-standard -->
    ```bash
    pip uninstall -y triton triton-ascend

    pip install \
        --extra-index-url https://mirrors.huaweicloud.com/ascend/repos/pypi \
        "triton-ascend==3.2.2"
    ```

    ??? note "社区 Triton 和 Triton Ascend 能否共存？"

        对于 Triton Ascend 3.2.0 及更早版本，在安装 Triton Ascend 之前，请卸载社区 Triton。

        对于 Triton Ascend 3.2.1 及更高版本，安装 Triton Ascend 会先安装所需的社区 Triton 包，然后通过共享安装路径提供其 Ascend 特定实现。如果之后重新安装或升级 Triton，它可能会覆盖 Triton Ascend 文件。在这种情况下，请卸载这两个包并重新安装 Triton Ascend。

        有关更多 Triton Ascend 安装指导，请参阅 [Triton-Ascend > 常见问题](https://github.com/Ascend/triton-ascend/blob/main/docs/en/FAQ.md#1-installation-and-environment-configuration)。

    安装其他版本的 `vllm-ascend` 时，请检查仓库中的 `requirements.txt` 以获取对应的 `triton-ascend` 版本。

=== "Atlas 300I DUO / Atlas 200I Pro"

    Atlas 300I DUO 和 Atlas 200I Pro 不支持 `triton` 或 `triton-ascend`。如果环境中已安装其中任一包，请卸载它，并且不要重新安装 `triton-ascend`：

    ```bash
    pip uninstall -y triton triton-ascend
    ```

python -m pip install --upgrade \
    pip "setuptools>=64" "setuptools-scm>=8" wheel \
    attrs googleapis-common-protos \
    "cmake>=3.26" ninja

python -m pip install \
    --index-url https://download.pytorch.org/whl/cpu/ \
    torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0
python -m pip install \
    --extra-index-url https://mirrors.huaweicloud.com/ascend/repos/pypi \
    torch-npu==2.10.0.post4 triton-ascend==3.2.2
python -m pip install \
    --extra-index-url https://mirrors.huaweicloud.com/ascend/repos/pypi \
    -r requirements.txt

export ASCEND_TOOLKIT_HOME="${ASCEND_TOOLKIT_HOME:-/usr/local/Ascend/ascend-toolkit/latest}"
export TORCH_DEVICE_BACKEND_AUTOLOAD=0
export COMPILE_CUSTOM_KERNELS=0
export SOC_VERSION=ascend910b1  # A2
python -m pip install \
    --no-build-isolation \
    --no-deps \
    --extra-index-url https://mirrors.huaweicloud.com/ascend/repos/pypi \
    -e .

# Execute on the target node (replace with actual IP)
hccn_tool -i 0 -ping -g address x.x.x.x