Publications

* indicates equal contribution, † indicates alphabetical order.

Default:

,,[Google Scholar]

Topics:

,,,,,,

CLLMs: Consistency Large Language Models
Siqi Kou*, Lanxiang Hu*, Zhezhi He, Zhijie Deng, Hao Zhang
Preprint 2024
code
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
Wei-Lin Chiang*, Lianmin Zheng*, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E. Gonzalez, Ion Stoica
Preprint 2024
code project
APIServe: Efficient API Support for Large-Language Model Inferencing
Reyna Abhyankar*, Zijian He*, Vikranth Srivatsa, Hao Zhang, Yiying Zhang
Preprint 2024
Break the Sequential Dependency of LLM Inference using Lookahead Decoding
Yichao Fu, Peter Bailis, Ion Stoica, Hao Zhang
Preprint 2024
code project
DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, Hao Zhang
Preprint 2024
Online Speculative Decoding
Xiaoxuan Liu, Lanxiang Hu, Peter Bailis, Ion Stoica, Zhijie Deng, Alvin Cheung, Hao Zhang
Preprint 2023
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
Lianmin Zheng*, Wei-Lin Chiang*, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric Xing, Joseph E Gonzalez, Ion Stoica, Hao Zhang
ICLR 2024
project
How Long Can Context Length of Open-Source LLMs truly Promise?
Dacheng Li*, Rulin Shao*, Anze Xie, Ying Sheng, Lianmin Zheng, Joseph Gonzalez, Ion Stoica, Xuezhe Ma, Hao Zhang
Instruction Tuning and Instruction Following Workshop @ NeurIPS 2023
project
Judging LLM-as-a-judge with MT-Bench and Chatbot Arena
Lianmin Zheng*, Wei-Lin Chiang*, Ying Sheng*, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E Gonzalez, Ion Stoica
NeurIPS 2023
code project
Efficient Memory Management for Large Language Model Serving with PagedAttention
Woosuk Kwon*, Zhuohan Li*, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Yu, Joey Gonzalez, Hao Zhang, Ion Stoica
SOSP 2023
code project
Vicuna: An Open-source Chatbot Impressing GPT-4 with 90%* Chatgpt Quality
Wei-Lin Chiang†, Zhuohan Li†, Zi Lin†, Ying Sheng†, Zhanghao Wu†, Hao Zhang†, Lianmin Zheng†, Siyuan Zhuang†, Yonghao Zhuang†, Joseph E Gonzalez†, Ion Stoica†, Eric P Xing†
Blogpost 2023
code project
AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving
Zhuohan Li*, Lianmin Zheng*, Yinmin Zhong*, Vincent Liu, Ying Sheng, Xin Jin, Yanping Huang, Zhifeng Chen, Hao Zhang, Joseph E Gonzalez, Ion Stoica
OSDI 2023
code project
MPCFormer: Fast, Performant and Private Transformer Inference with MPC
Dacheng Li*, Rulin Shao*, Hongyi Wang*, Han Guo, Eric P. Xing, Hao Zhang
ICLR 2023 (Notable-top-25%)
code
On Optimizing the Communication of Model Parallelism
Yonghao Zhuang*, Hexu Zhao*, Lianmin Zheng, Zhuohan Li, Eric P. Xing, Qirong Ho, Joseph E. Gonzalez, Ion Stoica, Hao Zhang
MLSYS 2023
code presentation
Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning
Lianmin Zheng*, Zhuohan Li*, Hao Zhang*, Yonghao Zhuang, Zhifeng Chen, Yanping Huang, Yida Wang, Yuanzhong Xu, Danyang Zhuo, Eric P. Xing, Joseph E. Gonzalez, Ion Stoica
OSDI 2022
project presentation
Terapipe: Token-level Pipeline Parallelism for Training Large-scale Language Models
Zhuohan Li, Siyuan Zhuang, Shiyuan Guo, Danyang Zhuo, Hao Zhang, Dawn Song, Ion Stoica
ICML 2021
code
Pollux: Co-adaptive Cluster Scheduling for Goodput-optimized Deep Learning
Aurick Qiao, Sang Keun Choe, Suhas Jayaram Subramanya, Willie Neiswanger, Qirong Ho, Hao Zhang, Gregory R Ganger, Eric P Xing
OSDI 2021 (Jay Lepreau Best Paper Award)
code presentation
Machine Learning Parallelism Could Be Adaptive, Composable, and Automated
Hao Zhang
CMU PhD Dissertation 2020
Autosync: Learning to Synchronize for Data-parallel Distributed Deep Learning
Hao Zhang*, Yuan Li*, Zhijie Deng, Xiaodan Liang, Lawrence Carin, Eric Xing
NeurIPS 2020
code
Toward Understanding the Impact of Staleness in Distributed Machine Learning
Wei Dai, Yi Zhou, Nanqing Dong, Hao Zhang, Eric P Xing
ICLR 2019
Cavs: An Efficient Runtime System for Dynamic Neural Networks
Shizhen Xu*, Hao Zhang*, Graham Neubig, Wei Dai, Jin Kyu Kim, Zhijie Deng, Qirong Ho, Guangwen Yang, Eric P Xing
ATC 2018
code
Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters
Hao Zhang, Zeyu Zheng, Shizhen Xu, Wei Dai, Qirong Ho, Xiaodan Liang, Zhiting Hu, Jinliang Wei, Pengtao Xie, Eric P Xing
ATC 2017
code project
Geeps: Scalable Deep Learning on Distributed GPUs with a GPU-specialized Parameter Server
Henggang Cui, Hao Zhang, Gregory R Ganger, Phillip B Gibbons, Eric P Xing
EUROSYS 2016
code
HD-CNN: Hierarchical Deep Convolutional Neural Networks for Large Scale Visual Recognition
Zhicheng Yan, Hao Zhang, Robinson Piramuthu, Vignesh Jagadeesh, Dennis DeCoste, Wei Di, Yizhou Yu
ICCV 2015
code
Automatic Photo Adjustment Using Deep Neural Networks
Zhicheng Yan, Hao Zhang, Baoyuan Wang, Sylvain Paris, Yizhou Yu
ACM Transactions on Graphics 2015
code