DJL 0.9.0 brings MXNet inference optimization, abundant PyTorch new feature support, TensorFlow windows GPU support and experimental DLR engine that support TVM models.

Key Features

• Add experimental DLR engine support. Now you can run TVM model with DJL

MXNet

• Improve MXNet JNA layer by reusing String, String[] and PointerArray with object pool which reduce the GC time significantly

PyTorch

• you can easily create COO Sparse Tensor with following code snippet

long[][] indices = {{0, 1, 1}, {2, 0, 2}};
float[] values = {3, 4, 5};
FloatBuffer buf = FloatBuffer.wrap(values);
manager.createCoo(FloatBuffer.wrap(values), indices, new Shape(2, 4));

• If the input of your TorchScript model need List or Dict type, we now add simple one dimension support for you.

// assum your torchscript model takes model({'input': input_tensor})
// you tell us this kind of information by setting the name
NDArray array = manager.ones(new Shape(2, 2));
array.setName("input1.input");

• we support loading ExtraFilesMap

// saving ExtraFilesMap
Criteria<Image, Classifications> criteria = Criteria.builder()
  ...
  .optOption("extraFiles.dataOpts", "your value")  // <- pass in here 
  ... 

TensorFlow

• Windows GPU is now supported

Several Engines upgrade

Enhancement

• Add docker file for serving
• Add Deconvolution support for MXNet engine
• Support PyTorch COO Sparse tensor
• Add CSVDataset, you can find a sample usage here
• Upgrade TensorFlow to 2.3.1
• Upgrade PyTorch to 1.7.0
• Add randomInteger operator support for MXNet and PyTorch engine
• Add PyTorch Profiler
• Add TensorFlow Windows GPU support
• Support loading the model from jar file
• Support 1-D list and dict input for TorchScript
• Remove the Pointer class being used for JNI to relieve Garbage Collector pressure
• Combine several BertVocabulary into one Vocabulary
• Add loading the model from Path class
• Support ExtraFilesMap for PyTorch model inference
• Allow both int32 & int64 for prediction & labels in TopKAccuracy
• Refactor MXNet JNA binding to reduce GC time
• Improve PtNDArray set method to use ByteBuffer directly and avoid copy during tensor creation
• Support experimental MXNet optimizeFor method for accelerator plugin.

Documentation and examples

• Add Amazon Review Ranking Classification
• Add Scala Spark example code on Jupyter Notebook
• Add Amazon SageMaker Notebook and EMR 6.2.0 examples
• Add DJL benchmark instruction

Bug Fixes

• Fix PyTorch Android NDIndex issue
• Fix Apache NiFi issue when loading multiple native in the same Java process
• Fix TrainTicTacToe not training issue
• Fix Sentiment Analysis training example and FixedBucketSampler
• Fix NDArray from DataIterable not being attaching to NDManager properly
• Fix WordPieceTokenizer infinite loop
• Fix randomSplit dataset bug
• Fix convolution and deconvolution output shape calculations

Contributors

Thank you to the following community members for contributing to this release:

Frank Liu(@frankfliu)
Lanking(@lanking520)
Kimi MA(@kimim)
Lai Wei(@roywei)
Jake Lee(@stu1130)
Zach Kimberg(@zachgk)
0xflotus(@0xflotus)
Joshua(@euromutt)
mpskowron(@mpskowron)
Thomas(@thhart)
DocRozza(@docrozza)
Wai Wang(@waicool20)
Trijeet Modak(@uniquetrij)