DJL v0.12.0 added GPU support to PaddlePaddle and ONNXRuntime, and introduces several new features:

Key Features

• Updates PaddlePaddle engine with GPU support.
• Updates ONNXRuntime engine with GPU support.
• Upgrades ONNXRuntime engine to 1.8.0.
• Upgrades XGBoost engine to 1.4.1.
• Introduces AWS Inferentia support, see our example for detail.
• Adds FLOAT16 datatype support in NDArray.
• Support UTF16 surrogate characters in NLP tokenization.
• Makes benchmark as a standalone tool.
• Releases djl-serving docker image to docker hub.

Enhancement

• DJL Benchmark now can benchmark any datatype as input.
• Makes Grayscale image processing match openCV’s behavior (#965)
• Improves PyTorch engine to load extra shared library for custom operators (#983)
• Improves djl-serving REST API to support load model on specified engine (#977)
• Improves djl-serving to support load multiple version of a model on the same endpoint (#1052)
• Improves djl-serving to support auto-scale workers based on traffic (#986)
• Implements several operators:
  • Adds the truncated normal operator (#1005)
  • Adds the one hot operator for PyTorch (#1014)
  • Adds the LayerNorm operator in PyTorch (#1069)
• Introduces several API improvements
  • Improves Criteria.loadModel() API (#1018)
  • Refactors ModleLoader and TranslatorFactory (#712)
  • Improves BlockFactory API (#1045)
  • Makes SpProcessor public API (#1060)

Documentation and examples

• Adds Low cost inference with AWS Inferentia demo.
• Adds BigGAN demo in examples (#1038)
• Adds Super-resolution demo in examples (#1049)

Breaking change

• Direct access ModelZoo ModelLoader is no longer supported, use Criteria API instead.
• Deprecates ModelZoo.loadModel() API in favor of using Criteria.loadModel().

Bug Fixes

• Fixes missing softmax in action_recognition model zoo model (#969)
• Fixes saveModel NPE bug (#989)
• Fixes NPE bug in block.toString() function (#1076)
• Adds back String tensor support to TensorFlow engine (lost in 0.11.0 during refactor) (#1040)
• Sets ai.djl.pytorch.num_interop_threads default value for djl-serving (#1059)

Known issues

• The TensorFlow engine has a known memory leak issue due to the JavaCPP dependency. The memory leak issue has been fixed in javacpp 1.5.6-SNAPSHOT. You have to manually include javacpp 1.5.6-SNAPSHOT to avoid the memory leak. See: https://github.com/deepjavalibrary/djl/tree/master/tensorflow/tensorflow-engine#installation for more details.

Contributors

This release is thanks to the following contributors:

• Akshay Rajvanshi(@aksrajvanshi (https://github.com/ghost))
• Aziz Zayed(@AzizZayed (https://github.com/AzizZayed))
• Erik Bamberg(@ebamberg (https://github.com/ebamberg))
• Frank Liu(@frankfliu (https://github.com/frankfliu))
• Hodovo(@hodovo (https://github.com/Hodovo))
• Jake Lee(@stu1130 (https://github.com/stu1130))
• Qing Lan(@lanking520 (https://github.com/lanking520))
• Tibor Mezei (@zemei (https://github.com/zemei))
• Zach Kimberg(@zachgk (https://github.com/zachgk))