DJL v0.15.0 updates the engines PyTorch to 1.10.0, ONNXRuntime to 1.10.0, TensorFlow to 2.7.0, TensorFlowLite to 2.6.2 and introduces several new features:

Key Features

• Introduces Huggingface tokenizers extension which allows user to leverage high performance fast tokenizer in Java
• Upgrades PyTorch engine to 1.10.0 with CUDA 11.3 support
• Upgrades TensorFlow to 2.7.0 with CUDA 11.3 support
• Upgrades ONNXRuntime engine to 1.10.0
• Upgrades TensorFlowLite to 2.6.2
• Provides better PyTorch engine backward compatibility support
• Adds load model from InputStream support
• Adds Windows support for SentencePiece
• Removes -auto packages to simplify DJL dependencies
• Fixes log4j CVEs

Enhancement

• Improves PyTorch engine:
  • Adds support to load custom build PyTorch native library from specified location
  • Adds support to use precxx11 version of PyTorch on GPU
  • Provides offline native package for older version of PyTorch to run with latest DJL (#1385)
  • Report correct engine version for custom build PyTorch native library
  • Adds Tuple support to IValue (#1436)
• Adds support to load model from InputStream for some of the engines (#1400):
  • Adds load from InputStream for PyTorch model
  • Adds load from InputStream for TensorFlowLite model (#1402)
  • Adds load from InputStream for ONNXRuntime model (#1402)
  • Adds load from InputStream for SentencePiece (#1139)
• Introduces several new features in djl-serving:
  • Automatically detect model’s engine
  • Reduces netty dependencies to minimize package size
  • Installs engine dependency on-demand for non-commonly used engines
  • Adds support for nested folder in model archive file
  • Released djl-serving to homebrew
• Introduces server new features in djl-bench
  • Release djl-bench to homebrew and snapcraft
• Improve opencv extension to expose API from opencv to end users
• Introduces several API improvements:
  • Improves TrainingResult with toString() print out (#1369)
  • Allows to register engine at runtime (#1386)
  • Allows to dynamic add model zoo (#1397)
  • Creates IndexEvaluator and IndexLoss (#1414)
  • Introduces Huggingface Tokenizer (#1406)
• Publishes several new docker images to dockerhub (https://hub.docker.com/u/deepjavalibrary)
  • djl-serving docker image
  • djl-serving for inferentia docker image
  • DJL windows docker image with jdk11

Documentation and examples

• Updates inferentia demo to support neuron runtime 2.x
• Updates jupyter notebooks with NoBatchifyTranslator to simplify example code (#1370)
• Updates benchmark README (#1410)
• Fixes rank classification jupyter notebook (#1368)
• Updates packaging model document (#1471)
• Fixes d2l book (https://d2l.djl.ai/) image display issue (#183)
• Translates d2l Chinese book (https://d2l-zh.djl.ai/) chapter 9 to chapter 14 to Chinese

Breaking change

• N/A

Bug Fixes

• Fixes several ci issues on Github Actions (#1358, #1362, #1363, #1364)
• Fixes crash when IValue list input is empty (#1440)
• Fixes NullPointException bug in getContextClassLoader() (#1445)
• Fixes model name detection issue in loading model from jar (#1446)
• Fixes unset numEmbeddings in TrainableWordEmbedding.java (#1450)
• Fixes protobuf-java CVEs

Contributors

This release is thanks to the following contributors:

• Andrey Zakharov (@zakhio)
• daofaziran1 (@daofaziran1)
• enpasos (@enpasos)
• Frank Liu (@frankfliu)
• Hampus Londögård (@Lundez)
• Jake Lee (@stu1130)
• James Zow(@jzow)
• Qing Lan (@lanking520)
• Zach Kimberg (@zachgk)
• Viet Yen Nguyen (@nguyenvietyen)

New Contributors

• @Lundez made their first contribution in #1412
• @zakhio made their first contribution in #1436
• @daofaziran1 made their first contribution in #1450

Full Changelog

0855108...v0.15.0