Artificial Intelligence (AI) is going to be the extension of our brains, in the same way as cars are the extension of our legs. It has already been an indispensable part of our life. Every day, AI navigates us to places, answers our queries, and recommends restaurants and movies. Overall, it amplifies what we do, augmenting our memory, giving you instant knowledge, allowing us to concentrate on doing things that are properly human.
However, designing new AI models is still reserved for experts; and the goal of my research is to democratize AI, making it accessible to everybody, such that any person regardless of their prior experiences, and any company regardless of size can deploy sophisticated AI solutions with only a few simple clicks.
Zhao, Yiyang, Linnan Wang, Kevin Yang, Tianjun Zhang, Tian Guo, Yuandong Tian Multi-objective Optimization by Learning Space Partitions ICLR-2022 · Acceptance Rate: 32% · International Conference on Learning Representations Paper I just love this work, the best extension to LA-MCTS so far.
Yang, Kevin, Tianjun Zhang, Chris Cummins, Brandon Cui, Benoit Steiner, Linnan Wang, Joseph E. Gonzalez, Dan Klein, Yuandong Tian Learning Space Partitions for Path Planning NeurIPS-2021 · Acceptance Rate: 26% · Advances in Neural Information Processing Systems Paper
Zhao, Yiyang, Linnan Wang (equally contributed), Yuandong Tian, Rodrigo Fonseca, Tian Guo ICML-2021 · Acceptance Rate: 3% · International Conference on Machine Learning Few-shot Neural Architecture Search Paper ·
Code ·
FB AI Blog Long Oral
Automated Machine Learning: building an AI that builds AI.
AlphaX: inspired by AlphaGo, we build the very first NAS search algorithm based on Monte Carlo Tree Search (MCTS). We showed Neural Networks designed by AlphaX improve the downstream applications such as detection, style transfer, image captioning, and many others.
LA-MCTS: we find that different action space used in MCTS significantly affects the search efficiency, which motivates the idea of learning action space for MCTS (LA-MCTS). This project contains 1) a distributed version that is scalable to hundreds of GPUs to push SoTA results, and 2) a one-shot version that let you get a working result within a few GPU days. You can find the entire pipeline (search and training) for doing NAS here.
With LA-MCTS, we have achieved SoTA results on many CV tasks including CIFAR-10, ImageNet and detection. Besides, LA-MCTS also achieves strong performance in general black-box optimization and reinforcement learning benchmarks, in particular for high-dimensional problems.
Machine Learning System: building efficient distributed systems for AI.
SuperNeurons: this project builds a C++ Deep Learning framework, which features a dynamic GPU memory scheduling run-time to enable the neural network training far beyond the GPU DRAM capacity, and a FFT based gradient compression protocol for the efficient distributed DNN training.
BLASX: this project builds a level-3 BLAS library for heterogeneous multiGPUs. Due to the novel tile-cache design to avoid unnecessarily data-swapping, BLASX is 30% faster than commercial cuBLAS-XT from NVIDIA.
