Key Features

• Upgrades and enhancements for Engines
  • Upgrades PyTorch to 1.13.1 (#2245)
  • Upgrades ONNXRuntime to 1.14.0 (#2393)
  • Upgrades HuggingFace tokenizer version to 0.13.2 (#2369)
  • Upgrades XGBoost to 1.7.3 (#2371)
  • Removes Neo-DLR engine from DJL #2373
• Introduces several improvements for extensions:
  • Adds batch support huggingface tokenizer
  • Adds API improvement for Spark extensions
  • Add a few image processing methods in OpenCV extension (#2320)
  • Adds stft and fft forier transform for audio extension (#2259)
• Implements NDScope to automatically close NDArray in the scope (#2321)
• Allows MXNet runs on Ampere GPU (#2313)
• DJLServing release
  • Adds faster transformer support (#424)
  • Adds Deepspeed ahead of time partition script in DLC (#466)
  • Adds SageMaker MME support (#479)
  • Adds support for stable-diffusion-2-1-base model (#484)
  • Adds support for stable diffusion depth model (#488)
  • Adds out of memory protection for modle loading (#496)
  • Makes load_on_devices per model setting (#493)
  • Adds several per model settings
  • Improves management console model loading and inference UI (#431, #432)
  • Updates deepspeed to 0.8.0 (#465)

Enhancement

• Introduces several enhancements for timeseries extension:
  • Adds probability distribution support for timeseries (#2025)
  • Add time series dataset support for timeseries package (#2026)
  • Add some basic block and deepAR model (#2027)
  • Enable pytorch deepar model inference in time series package (#2149)
• Introduces several enhancement for HuggingFace tokenizer:
  • Adds batch encoding support(#2342, #2343, #2337, #2338)
  • Adds batchEncode for text pair (#2339)
  • Adds mean_sqrt_len and weightedmean pooling for TextEmbedding (#2272)
  • Adds more pooling mode form TextEmbedding (#2261)
  • Allows Huggingface model zoo list models in offline mode (#2322)
  • Update TextEmbedding pooling model name (#2314)
• Introduces a few new examples:
  • Adds clip model to examples (#2239)
  • Adds openai whisper model to examples (#2293)
  • Adds stable diffusion examples (#2246)
• Introduces several enhancement for Spark extension:
  • Add pyspark support (#2301)
  • Adds spark extension docker image (#2243)
  • Adds Numpy binary translator (#2399)
  • Adds huggingface tokenizer support for Spark (#2311)
  • Refactor Spark extension API (#2370)
• DJL API improvements:
  • Adds limit and callback for Metrics API (#2362)
  • Adds newBaseManager(String engineName) api (#2275)
  • Falls back to PassthroughNDManager if there is no engine (#2354)
  • Improves Criteria.build() error message (#2397)
  • Improves hybrid engine operators (#2279)
  • Improve NDArray encode/decode performance (#2361)
  • Refactors Engine class (#2303)
  • Implements gatherNd, partial flatten and enable BertOnCode training (#2216)
  • Makes TestDataset constructor protected (#2271)
  • Creates SimplePaddingStackBatchifier (#2384)
  • Creates standard for PreTrained behavior (#2360)
  • Creates the TabularTranslator (#2344)
  • Enables tuning distill_bert embedding layer (#2203)
  • Handle RuntimeException on ImageFactory::newInstance (#2241)
  • Implement AdamW on Pytorch and MXNet (#2206)
  • Improves TranslatorExpansions with pre-processing and post-processing (#2213)
  • Opens LayerNorm.Builder for inheritance (#2309)
  • Adds feature to identify NDArray if double closed (#2352)
  • Adds back support for getting managed Arrays (#2386)
  • Opens Conv2d block constructor for inheritance (#2231)
• PyTorch engine improvements:
  • Adds scatter function to PtNDArray (#2332)
  • Search for model.pt or model.onnx when loading the model (#2364)
  • Better handle String tensor operations (#2380)
  • Fixes typo in error message (#2355)
  • Makes JNI comptible with PyTorch 1.11.0 (#2263)
  • No longer search java.library.path (#2235)
  • Uses runMethod to replace forward function (#2234)
  • Workaround hann_window issue for PyTorch 1.12.1 (#2262)
  • Adds support for torch::cuda::empty_cache(). (#2305)
  • Fixes memory leak in PyTorch indexing fuction (#2300)
• LightGBM engine improvements:
  • Fixes atomic move issue (#2258)
  • Fixes byte order for fp32 and fp64 array creation (#2278)
• Updates library dependencies:
  • Reduces aws s3 extension dependencies (#2378)
  • Reduces hadoop extension dependencies (#2377)
  • Removes unnecessary dependency (#2329)
  • Upgrade dependencies versions (#2371)
• Adds aarch64 and macOS M1 support for SentencePiece (#2325, #2324)
• Adds centos 7 support for SentencePiece (#2402)
• Adds aarch64 support for audio extension (#2250)

Breaking change

• Removes Neo-DLR engine from DJL #2373
• Moves RawDataset from basicdata to api module (#2375)
• Changes Spark extension API (#2388)

Bug fixes

• Fixes nested model directory issue (#2214)
• Fixes NPE if block is cleard before training (#2365)
• Fixes toDebugString() IllegalStateException if NDArray is closed (#2347)
• Fixes typo in javadoc (#2286)
• Fixes unittest failure on GPU (#2255)
• Fixes inconsistencies using RANK in engine providers (#2244)
• Fixes the content format (#2229)
• Fixes the name passing bug (#2222)
• Fixes TrainAmazonReviewRanking example for PyTorch (#2173)
• Fixes performance issues from freezing MXNet (#2394)

Documentation and Examples

• Fixes ONNXRuntime Android doc (#2194)
• Updates readme of tranferFreshFruilt (#2236)
• Updates cache menagement adds ONNX and Huggingface cache directory (#2334)
• Updates FAQ (#2328)
• Fix typo in javadoc of Block and AbstractBlock (#2288)
• Adds extensions to docs web page (#2284)
• Adds missing DJLServing docs menu items (#2363)
• Improves documents (#2396)
• Updates PyTorch graph exector optimization document (#2374)
• Updates README for engines (#2282)
• Fixes whisper model translator (#2341)
• Documents workaround AudioGrabber bug for whisper model (#2330)
• Adds tensorflow dependency to pom.xml (#2268)
• Fixes PyTorch nightly test (#2264)
• Fixes Onnx version in README (#2253)
• Update onnx version in README file (#2223)
• Fix djl-zero doc (#2201)

CI improvements:

• Adds lost package for Codespaces Dockerfile (#2260)
• Uses ubuntu-latest as runner (#2199)
• Updates configure-aws-credentials to v1-node16 (#2196)
• Adds formatCpp exclusion to gradle plugin (#2215)
• Allows publish djl-serving from a branch (#2225)
• Build cu102 JNI for PyTorch 1.11.0 and 1.12.1 (#2254)
• Fixes jdk18 compile error (#2306)
• Replace jacoc root report with aggregation report plugin (#2280)
• Updates github action setup-python to v4 (#2202)
• Upgrade gradle to 7.6 (#2302)
• Upgrade testng to 7.7.0 (#2230)
• Fixes PMD 6.21.0 reported issues (#2296)
• Removes DLR github actions (#2398)
• Sets seed in TransferFreshFruitTest for consistent results (#2238)
• fastText should not depends on any engine #2353
• Avoid unittest polluting cache directory. (#2228)
• Minor refactor encoding class (#2336)
• Split TranslatorTest into smaller test classes. (#2335)

Contributors

• @dayo05
• @demq
• @enpasos
• @frankfliu
• @KexinFeng
• @lanking520
• @Noricks
• @siddvenk
• @SuperMaskv
• @xyang16
• @zachgk

New Contributors

• @dayo05 made their first contribution in #2241
• @Noricks made their first contribution in #2260
• @SuperMaskv made their first contribution in #2288

Full Changelog: v0.20.0...v0.21.0