| CARVIEW |
Hao Zhou (周浩)
Research Associate Professor
Insititute for AI Industry Research (AIR), Tsinghua
University
Email: zhouhao@air.tsinghua.edu.cn & haozhou0806@gmail.com
Hello! I am a Research Associate Professor at the Institute for AI Industry Research (AIR), Tsinghua University. My work centers on building Generative AI for discrete symbols (text and proteins). I focus on Large Language Models (LLMs) and its applications to scientific discovery, with special interests in biological agents. Previously, I was a Research Scientist/Manager at Bytedance, where I led the ByteDance AI Lab research teams on text generation and AI drug design. Here is my bio in Chinese 中文 and my CV.
- I lead the GenSI research group at Tsinghua AIR
- I co-lead the SIA Lab, the joint laboratory of THU-AIR and Bytedance Seed on cutting-edge LLM research.
- Professional Service: Area Chair of EMNLP 2022; ACL 2023; NeurIPS 2022–2025; ICML 2023–2025; ICLR 2023–2026.
Recent Highlight
Seed Diffusion Preview
A large scale language model based on discrete-state diffusion, achieves an inference speed of 2,146 token/s, outperforming its Google Gemini Diffusion and Mercury Counterparts.
🚀 全面超越google的Gemini-diffusion,一秒钟生成2000+的词,性能不掉!
Page Paper Demo CitationsNEW
AMix-1: A Pathway to Test-Time Scalable Protein Foundation Model
We propose a systematic paradigm, including scaling laws, structural emergent ability, in-context learning, and test-time scaling, for crafting protein foundation models, emphasizing a road map towards scalable protein design. We design a high-activity AmeR variant with a 50× improvement over the wild type.
全新可推理、可Test-time-Scaling的蛋白质基座 AMix-1🔥!
我们提出了AMix-1,首次尝试以Scaling Law、Emergent Ability、In-Context Learning 和 Test-time Scaling
的系统化方法论来构建蛋白质基座模型,并和清华张数一老师团队合作在AmeR蛋白设计上提升其能力50倍!
Page Paper CodeNEW Model CitationsNEW
MemAgent: Reshaping Long-Context LLM with Multi-Conv RL based Memory Agent
MemAgent has demonstrated superb long-context capabilities, being able to extrapolate from an 8K context trained on 32K text to a 3.5M QA task with performance loss < 5% and achieves 95%+ accuracy in 512K RULER test.
🔥多轮对话RL + LLM外挂Notepad 🗒️ = Memory Agent.
🚀不需要改动任何网络架构(线性注意力/稀疏注意力),一切交给RL,LLM能自己学到在notepad里保留哪些关键信息并不断更新. 使用具有多轮独立对话的Agent处理长文并用DAPO算法进行了端到端训练。
🏆仅使用8K Context的14B模型可从训练时的32K长度无损外推至3.5M,并且在512K长度的RULER测试集中取得95%+的成绩,7B模型也在测试中超越了更大尺寸的专门长文模型.
Page Paper CodeNEW Data Model CitationsNEW
DAPO: an Open-Source LLM Reinforcement Learning System at Scale
We propose the Decoupled Clip and Dynamic sAmpling Policy Optimization (DAPO) algorithm. By making our work publicly available, we provide the broader research community and society with practical access to scalable reinforcement learning, enabling all to benefit from these advancements. Our system is based on the awesome verl framework. Thanks for their great work! Applying DAPO training to Qwen2.5-32B base model proves to outperform the previous state-of-the-art DeepSeek-R1-Zero-Qwen-32B on AIME 2024, achieving 50% accuracy with 50% less training steps.
🚀我们提出了DAPO,一个工业级别的,训练稳定高效率的,框架、数据、训练策略完全开源的大规模强化学习系统,RL from Pretrained (Qwen2.5-32B) 在 AIME 上跑到了50分,超越了之前 DeepSeek 在 R1 paper 里达到的的47分SOTA,欢迎试用!
Page Paper CodeNEW Data Model CitationsNEW
Honors and Awards
- Beijing Nova Program, 2024
- Ranked #1 on WMT EN-DE translation, 2021 🏆 using NAT
- CCF NLPCC Distinguished Young Scientist, 2021
- CAAI Doctoral Dissertation Award, 2019
- Outstanding Doctoral Dissertation of Nanjing University, 2019
- National Scholarship, 2011/2016
Talks
-
Generative AI for Scientific DiscoverySlides
- Tsinghua Young Scholars Monthly Salon (清华大学青年学者沙龙), Beijing, March, 2024
- CATL (宁德时代), Ningde, April, 2024
- NJU ChemBIC Research (南京大学化学与生物医药创新研究院学术年会), April, 2024
- NJNU Synthetic Biology Research Center (南京师范大学合成生物研究中心), Nanjing, May, 2024
- NJU AI School (南京大学人工智能学院), Nanjing, May, 2024
- Tsinghua FRCBS (清华大学北京生物结构前沿研究中心), Beijing, May, 2024
- Beijing Academy of AI Conference (智源大会), Beijing, June, 2024
- Beijing Hospital (北京医院), Beijing, June, 2024
- Annual Academic Conference of Shanghai AI Lab (上海人工智能实验室学术年会), Shanghai, December, 2024
-
Advances of Large Scaled Parallel Text Generationn
Slides
- NLGIW Tutorials / CIPS ATT 30 (第一届自然语言生成与智能写作讲习班), Online, April 2022. with Lihua Qian
- Invited Talk at NLG workshop of CNCC, Shenzhen, December, 2021
- Invited Talk at Artificial Intelligence Industry Summit Forat of CCF-AI (中国计算机学会人工智能会议产业高峰论坛), Yantai, July, 2021
- Invited talk at NEU (东北大学), THU (清华大学), 2021 - 2022.
-
Deep Generative Model for Text Generation
Slides
- NLPCC Tutorials /CCF Advanced Disciplines Lectures (ADL 107), Dunhuang, October 2019.with Prof. Lei Li
- CAAI-AIDL (中国人工智能学会前沿讲习班), Shanghai, December 2019.
- CCMT (第十六届全国机器翻译大会), Tutorial, Online, October 2020.
- Invited talk at FDU (复旦大学), NJU (南京大学), NJNU (南京师范大学), WestLake University (西湖大学), SJTU (上海交通大学), HIT (哈尔滨工业大学), 2019 - 2020.
-
Discreteness in Neural Natural Language Processing
Slides P1
Slides P2
Slides P3
- EMNLP-IJCNLP 2019, Tutorial, HongKong with Prof. Lili Mou and Lei Li