Key Changes

• Upgrades for engines

  • Upgrades PyTorch engine to 2.3.1
  • Upgrades TensorFlow engine to 2.16.1
  • Introduces Rust engine CUDA support
  • Upgrades OnnxRuntime version to 1.18.0 and added CUDA 12.4 support
  • Upgrades javacpp version to 1.5.10
  • Upgrades HuggingFace tokenizer to 0.19.1
  • Fixes several issues for LightGBM engine
  • Deprecated llamacpp engine

• Enhancements for engines and API

  • Adds Yolov8 segmentation and pose detection support
  • Adds metric type to Metic class
  • Improves drawJoints and drawMask behavior for CV model
  • Improves HuggingFace model importing and conversion tool
  • Improves HuggingFace NLP model batch inference performance
  • Adds built-in ONNX extension support
  • Adds several NDArray operators in PyTorch engine
  • Adds fp16 and bf16 support for OnnxRuntime engine
  • Adds CrossEncoder support for NLP models

Enhancements

• Adds metric type to Metic class by @frankfliu in #3244
• Improves drawJoints behavior by @frankfliu in #3305
• [api] Allows to control json pretty print with env var by @frankfliu in #3288
• [api] Avoid null dimensions for Metric by @frankfliu in #3246
• [api] Improve NDArray.toDebugString() output by @frankfliu in #3290
• [api] Loads native engine in deterministic order by @frankfliu in #3300
• [api] Refactor drawMask() for instance segmentation by @frankfliu in #3304
• [api] Refactor nms for yolo translator by @frankfliu in #3297
• add close method to all nd manager by @lanking520 in #3225
• ported tools/stats.gradle by @elect86 in #3219
• use standard GSON output by @lanking520 in #3284
• [enhancement] Optimize memory copy overhead to enhance performance. by @ewan0x79 in #3289
• Gradle Kotlin script plus other stuff by @elect86 in #3167
• Improved incremental build by @benjie332 in #3231
• Refactored Identifiers by @congyuluo in #3276
• Refactored Identifiers by @congyuluo in #3282
• [gradle] Remove unused gradle files by @frankfliu in #3280
• [jacoco] exclude spark extension since it doesnot contain test by @frankfliu in #3230
• [Lgbm] support multi classification by @ewan0x79 in #3234
• [Lgbm] support multi type prediction by @ewan0x79 in #3237
• [llamacpp] Removing llamacpp support in DJL by @frankfliu in #3312
• [mxnet-model-zoo] Adds missing translatorFactory in metadata by @frankfliu in #3279
• [onnx] Adds fp16 and bfp16 support for OnnxRuntime by @frankfliu in #3281
• [onnxruntime] Add debug message for OnnxRuntime by @xyang16 in #3217
• [onnxruntime] Adds yolov8n pose model for OnnxRuntime by @frankfliu in #3309
• [onnxruntime] Adds yolov8n-seg model to onnxruntime model zoo by @frankfliu in #3310
• [onnxruntime] Load onnx extenstion if available by @frankfliu in #3333
• [pytorch] Adds Yolov8n-seg model to model zoo by @frankfliu in #3308
• [pytorch] Adds back PyTorch 2.1.2 support by @frankfliu in #3285
• [pytorch] Adds yolov8n pose estimation model by @frankfliu in #3298
• [pytorch] Implements gammaln operator for PyTorch by @frankfliu in #3262
• [pytorch] Split maven publish into two parts by @frankfliu in #3273
• [rust] Add tokenizer cuda build workflow by @xyang16 in #3322
• [rust] Allows -2 as dims for sum() by @frankfliu in #3221
• [rust] Change loging level to debug by @xyang16 in #3336
• [rust] Download cu124 jni library for cuda by @xyang16 in #3327
• [rust] Remove 0-dimension tensor compare in NDArrayTests by @xyang16 in #3320
• [rust] Update gpu build pipeline to cu122 by @xyang16 in #3334
• [rust] Upgrade candle version by @xyang16 in #3248
• [rust] Use fused layer by @xyang16 in #3260
• [spark] Do not support model_url by @xyang16 in #3224
• [spark] Update dependency versions by @xyang16 in #3241
• [spark] Updates spark version to 3.5.1 by @frankfliu in #3240
• [spark] Use batch predict API by @xyang16 in #3242
• [text-embedding] Remove CrossEncoderTranslatorFactory in favor of TextEmbeddingTranslatorFactory by @frankfliu in #3239
• [tokenizer] Adds maxos-13 support back by @frankfliu in #3328
• [tokenizer] Ensure GPU is used in TextEmbeddingTranslator by @david-sitsky in #3212
• [tokenizer] Process text embedding input and output in stacked NDArray by @xyang16 in #3213
• [tokenizer] Recover accidentally deleted file by @frankfliu in #3311
• [tokenizer] Supports cross encoder for text classification model by @frankfliu in #3338
• [tokenizers] Download jni lib files for cuda by @xyang16 in #3326

Bug Fixes

• [api] Fix unitest in GPU docker running on CPU case by @frankfliu in #3228
• [api] Fixes IdEmbedding memory leak by @frankfliu in #3257
• [api] Fixes nightly tests on GPU machine by @frankfliu in #3302
• [api] Fixes unitest by @frankfliu in #3210
• [fix] fix lgbm bytebuffer native order by @ewan0x79 in #3258
• Fix Application.of missing some applications by @tadayosi in #3277
• [mxnet] Fixes GloveWordEmbeddingTranslator bug by @frankfliu in #3287
• [pytorch-model-zoo]: fix PtSsdTranslator.Builder.self() by @eversnarf in #3204
• [pytorch] Fixes PyTorch 2.3.1 windows dependencies by @frankfliu in #3269
• [pytorch] Fixes PyTorch 2.3.1 windows dependencies by @frankfliu in #3270
• [pytorch] Fixes uploadS3 gradle task by @frankfliu in #3263
• [rust] Fix NDArrayTests failure on cuda by @xyang16 in #3319
• [rust] Fix deleteModel error by @xyang16 in #3229
• [rust] Fix output tensor dtype by @xyang16 in #3249
• [rust] Fix tokenizer cuda pipeline name by @frankfliu in #3325
• [rust] Fixes test failure on GPU by @frankfliu in #3301
• [timeseries] Fixes contentLength issue for inference by @frankfliu in #3306
• [timeseries] Fixes duration format issue by @frankfliu in #3307
• [tensorrt] Fixes gradle biuld script by @frankfliu in #3253
• [tokenizer] Fixes detect include token type logic by @frankfliu in #3318
• [tokenizer] Fixes tokenizer build workflow by @frankfliu in #3323
• [tokenizers] Fixes huggingface build for Windows by @frankfliu in #3330
• [tokenizers] Fixes memory leak when there is overflowing tokens by @baldersheim in #3317
• [xgb] Fixes gradle build script by @frankfliu in #3254

Documentation

• [doc] add output formatter schema to LMI docs.djl.ai by @sindhuvahinis in #3268
• [doc] add release notes to docs.djl.ai by @sindhuvahinis in #3266
• [docs] Bump up DJL version to 0.28.0 by @frankfliu in #3247
• [docs] Update example reference by @emmanuel-ferdman in #3275
• [docs] add dark theme and fixed broken link by @Varun-Dutta in #3295
• [example] Adds PyTorch action recognition model to model zoo by @frankfliu in #3292
• [examples] Enabled training unit tests on macOS M1 by @frankfliu in #3256
• [examples] Fixes ObjectDetection example for macOS m1 by @frankfliu in #3206
• [examples] Fixes nightly build failure on Windows by @frankfliu in #3267
• [examples] Remove symbolic training for MXNet by @frankfliu in #3299
• Update README.md by @bradh in #3200

CI/CD

• [android] Updates android with PyTorch 2.2.2 by @frankfliu in #3236
• [api] Updates slf4j version to 2.0.13 by @frankfliu in #3329
• [bom] Uses release version for tensorflow by @frankfliu in #3313
• [ci] Disable github actions runner for non-djl repo by @frankfliu in #3220
• [ci] Fixes nightly publish for nodejs20 issue by @frankfliu in #3314
• [ci] Fixes publish maven native package by @frankfliu in #3264
• [ci] Fixes pytorch JNI build by @frankfliu in #3339
• [ci] Fixes rust jni build for nodejs20 issue by @frankfliu in #3315
• [ci] Fixes serving nightly publish by @frankfliu in #3245
• [ci] Fixes windows pytoch jni build by @frankfliu in #3209
• [ci] Minor github action workflow changes by @frankfliu in #3331
• [ci] Remove fastertransformer build workflow by @frankfliu in #3291
• [ci] Update to amazon-ecr-login@v2 by @frankfliu in #3250
• [ci] Updates OnnxRuntime to 1.18.0 by @frankfliu in #3235
• [ci] Updates dependency versions to latest by @frankfliu in #3332
• [ci] Updates spotbugs to 6.0.15 by @frankfliu in #3233
• [djl] update djl version and readmes by @tosterberg in #3202
• [MCM 0.29.0] Remove -SNAPSHOT for release v0.29.0 by @ydm-amazon in #3345
• Add more test logging by @xyang16 in #3321
• [pytorch] Updates PyTorch to 2.3.1 by @frankfliu in #3265
• [release] Bump up versions to 0.29.0 in documents to point to new url by @ydm-amazon in #3344
• [tensorflow] Updates tensorflow to 2.16.1 by @frankfliu in #3283

New Contributors

• @bradh made their first contribution in #3200
• @eversnarf made their first contribution in #3204
• @benjie332 made their first contribution in #3231
• @emmanuel-ferdman made their first contribution in #3275
• @tadayosi made their first contribution in #3277
• @congyuluo made their first contribution in #3276
• @Varun-Dutta made their first contribution in #3295
• @baldersheim made their first contribution in #3317

Full Changelog: v0.28.0...v0.29.0