Blog

TLDR: Efficient full-parameter fine-tuning of GPT-OSS-20B & Qwen3-14B models on a single NVIDIA GH200 and…

PyTorch now offers native quantized variants of Phi4-mini-instruct, Qwen3, SmolLM3-3B and gemma-3-270m-it through a collaboration…

The Challenge of PyTorch 2.0 Compilation Since the release of PyTorch 2.0 (PT2) and its…

PyTorch 2.8 has just been released with a set of exciting new features, including a…

Intel announces a major enhancement for distributed training in PyTorch 2.8: the native integration of…

As training jobs become larger, the likelihood of failures such as preemptions, crashes, or infrastructure…

In this blog post, we explore the kernel design details presented in the paper Fast…

Large Language Models (LLMs) have transformed tasks across numerous industries, including drafting emails, generating code,…

tldr: 1.22x - 1.28x training acceleration with MXFP8, equivalent convergence compared to BF16. We recently…

Large Language Models (LLMs) have revolutionized how we write and consume documents. In the past…

TL;DR NJTs (Nested Jagged Tensors) boost DRAMA model inference efficiency by 1.7x-2.3x, making it more…

Introduction ZenFlow is a new extension to DeepSpeed introduced in summer 2025, designed as a…

In this post, we present an optimized Triton BF16 Grouped GEMM kernel for running training…

charliemarsh’s tweet, creator of uv PyTorch is the leading machine learning framework for developing and…

Introduction We integrate mixed and low-precision training with Opacus to unlock increased throughput and training…

Key Takeaways: ExecuTorch 0.7 now enables KleidiAI by default, delivering automatic acceleration on Arm CPUs…