`bigdl-llm` Migration Guide

This guide helps you migrate your bigdl-llm application to use ipex-llm.

Upgrade `bigdl-llm` package to `ipex-llm`

Note

This step assumes you have already installed bigdl-llm.

You need to uninstall bigdl-llm and install ipex-llmWith your bigdl-llm conda environment activated, execute the following command according to your device type and location:

For CPU

pip uninstall -y bigdl-llm
pip install --pre --upgrade ipex-llm[all] # for cpu

For GPU

Choose either US or CN website for extra-index-url:

For US:

pip uninstall -y bigdl-llm
pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/

For CN:

pip uninstall -y bigdl-llm
pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/cn/

Migrate `bigdl-llm` code to `ipex-llm`

There are two options to migrate bigdl-llm code to ipex-llm.

1. Upgrade `bigdl-llm` code to `ipex-llm`

To upgrade bigdl-llm code to ipex-llm, simply replace all bigdl.llm with ipex_llm:

#from bigdl.llm.transformers import AutoModelForCausalLM # Original line
from ipex_llm.transformers import AutoModelForCausalLM #Updated line
model = AutoModelForCausalLM.from_pretrained(model_path,
                                             load_in_4bit=True,
                                             trust_remote_code=True)

2. Run `bigdl-llm` code in compatible mode (experimental)

To run in the compatible mode, simply add import ipex_llm at the beginning of the existing bigdl-llm code:

import ipex_llm # Add this line before any bigdl.llm imports
from bigdl.llm.transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(model_path,
                                             load_in_4bit=True,
                                             trust_remote_code=True)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!