Key Changes

• LlamaCPP Support. You can use DJL to run supported LLMs using the LlamaCPP engine. See the Chatbot example here to learn more.
• Manual Engine Initialization. You can configure DJL to not load any engines at startup, and query/register engines programmatically at runtime
• Engine Updates:
  • PyTorch 2.1.1
  • Huggingface Tokenizers 0.15.0
  • OnnxRuntime 1.16.3
  • XGBoost 2.0.3

Enhancement

• Add erf and atan2 by @TalGrbr in #2842
• Add FFT2 and FFT2 inverse by @TalGrbr in #2845
• [tokenizer] Update import script for huggingface_hub api change by @frankfliu in #2850
• [tokenizer] Not returns overflow tokens by default by @frankfliu in #2857
• [pytorch] Updates PyTorch engine to 2.1.1 by @frankfliu in #2864
• Adds Device.getDevices() for all Device by @zachgk in #2820
• Creates DJL manual engine initialization by @zachgk in #2885
• [pytorch] Allows to load libstdc++.so.6 form different location by @frankfliu in #2929
• Add Evaluator support to update multiple accumulators by @petebankhead in #2894
• Adds llama.cpp engine by @bryanktliu in #2904
• Yelov8 Translator optimization by @gevant in #2908
• [pytorch] Adds Yolov8n model to pytorch model zoo. by @frankfliu in #2910
• [onnx] Adds yolov8n to model zoo by @frankfliu in #2909
• [llama.cpp] Adds unit-test and standardize input parameters by @frankfliu in #2905
• [llama.cpp] Adds llama.cpp huggingface model zoo by @frankfliu in #2911
• [XGBoost] Updates XGBoost to 2.0.3 by @frankfliu in #2915
• [pytorch] Upgrade pytorch andorid to 2.1.1 by @frankfliu in #2914
• add awscurl release by @lanking520 in #2917
• [awscurl] change build to jar by @lanking520 in #2918
• [bom] Adds llama engine to BOM by @frankfliu in #2916
• [api] Adds ModelZooResolver interface by @frankfliu in #2922
• [api] Use folk java process to avoid jvm consume GPU memory by @frankfliu in #2882
• [onnxruntime] Updates OnnxRuntime to 1.16.3 by @frankfliu in #2888
• Tokenizers: Updated huggingface_models.py to support Safetensors models as well as pytorch by @dameikle in #2880
• [tokenizer] Uses fp32 for TextembeddingTranslator clip() by @frankfliu in #2881
• [tokenizer] Updates huggingface tokenizer to 0.15.0 by @frankfliu in #2867

Bug Fixes

• [tokenizer] Fixes tokenizer bug by @frankfliu in #2843
• Fixes archiveBaseName in native builds by @zachgk in #2859
• [pytorch] Ensure shared library loading order for aarch64 by @frankfliu in #2892
• [api] Handles both JNA conflict and missing case by @frankfliu in #2896
• Minor fixes to improve Apple Silicon MPS support by @petebankhead in #2873
• [tokenizer] Handles import huggingface model zoo exception case by @frankfliu in #2872
• [api] Update offline property name to avoid conflict with other app. by @frankfliu in #2877
• [tensorflow] Revert InstanceHolder for TensorFlow engine by @frankfliu in #2884
• [pytorch] Revert InstanceHolder for PyTorch engine by @frankfliu in #2876
• [pytorch] Fixes windows load nvfuser_codegen bug by @frankfliu in #2868

Documentation

• [docs] Update serving configuration nav by @zachgk in #2853
• Updates DJL version to 0.25.0 by @frankfliu in #2860
• Bump up DJL version to 0.26.0 by @frankfliu in #2861
• [docs] Move jupyter notebooks to DJL Demo by @zachgk in #2854
• [docs] Include LMI documents by @sindhuvahinis in #2870
• [docs] Updates documents to use JDK 17 by @frankfliu in #2898
• Updates DJL version to 0.26.0 by @siddvenk in #2930
• update master branch on the website to have large model inference guide by @lanking520 in #2865

CI/CD

• [ci] Allows build project with JDK 21 by @frankfliu in #2903
• [ci] Fixes pytorch android build by @frankfliu in #2921
• [ci] Fix build failure for build-pytorch-jni-linux by @maaquib in #2920
• [ci] Fixes native ci build failure by @frankfliu in #2924
• [CI] Fixes flaky early stopping test by @zachgk in #2866
• [ci] Fixes flaky early stopping training test by @frankfliu in #2879
• [ci] Use JDK 17 for github actions workflow by @frankfliu in #2897
• [ci] Fixes github action for centos and amazonlinux by @frankfliu in #2913
• [ci] Use macos-13 to avoid flaky test by @frankfliu in #2927
• [test] Fixes EarlyStopping flaky test by @frankfliu in #2926
• [api] Updates dependencies to latest version by @frankfliu in #2928
• [api] Updates common-compress version to address CVE issues by @frankfliu in #2871
• only build triton binaries by @lanking520 in #2847

New Contributors

• @TalGrbr made their first contribution in #2842
• @petebankhead made their first contribution in #2873
• @dameikle made their first contribution in #2880
• @gevant made their first contribution in #2908
• @maaquib made their first contribution in #2920

Full Changelog: v0.25.0...v0.26.0