Research

Selected Publications

  1. deepconf.png
    Deep Think with Confidence
    Yichao Fu* , Xuewei Wang , Yuandong Tian , and Jiawei Zhao*
    In Preprint , 2025
    arXiv Code GitHub stars
  2. galore.png
    GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
    Jiawei Zhao, Zhenyu Zhang , Beidi Chen , Zhangyang Wang , Anima Anandkumar , and Yuandong Tian
    In International Conference on Machine Learning (ICML) , 2024
    Oral presentation (top 1.5%)
    arXiv Code GitHub stars
  3. actpay.png
    Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
    Haizhong Zheng , Yang Zhou , Brian R. Bartoldson , Bhavya Kailkhura , Fan Lai , Jiawei Zhao, and Beidi Chen
    In Advances in Neural Information Processing Systems (NeurIPS) , 2025
    arXiv
  4. galore2.png
    GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projection
    DiJia Su , Andrew Gu , Jane Xu , Yuandong Tian , and Jiawei Zhao
    In Preprint , 2025
    arXiv
  5. zero_init.png
    ZerO Initialization: Initializing Neural Networks with only Zeros and Ones
    Jiawei Zhao, Florian Tobias Schaefer , and Anima Anandkumar
    Transactions on Machine Learning Research (TMLR), 2022
    arXiv Code
  6. signsgd.jpg
    signSGD with Majority Vote is Communication Efficient and Fault Tolerant
    Jeremy Bernstein* , Jiawei Zhao*, Kamyar Azizzadenesheli , and Anima Anandkumar
    In International Conference on Learning Representations (ICLR) , 2019
    arXiv Code

Recent Publications

  1. paretoq.png
    ParetoQ: Scaling Laws in Extremely Low-bit LLM Quantization
    Zechun Liu , Changsheng Zhao , Hanxian Huang , Sijia Chen , Jing Zhang , Jiawei Zhao, Scott Roy , Lisa Jin , Yunyang Xiong , Yangyang Shi , Lin Xiao , Yuandong Tian , Bilge Soran , Raghuraman Krishnamoorthi , Tijmen Blankevoort , and Vikas Chandra
    In Advances in Neural Information Processing Systems (NeurIPS) , 2025
    arXiv
  2. welore.png
    From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications
    Ajay Jaiswal , Yifan Wang , Lu Yin , Shiwei Liu , Runjin Chen , Jiawei Zhao, Ananth Grama , Yuandong Tian , and Zhangyang Wang
    In International Conference on Machine Learning (ICML) , 2025
    arXiv
  3. s2ft.png
    S^2FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity
    Xinyu Yang , Jixuan Leng , Geyang Guo , Jiawei Zhao, Ryumei Nakada , Linjun Zhang , Huaxiu Yao , and Beidi Chen
    In Advances in Neural Information Processing Systems (NeurIPS) , 2024
    arXiv
  4. headinfer.png
    HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading
    Cheng Luo , Zefan Cai , Hanshi Sun , Jinqi Xiao , Bo Yuan , Wen Xiao , Junjie Hu , Jiawei Zhao, Beidi Chen , and Anima Anandkumar
    In Preprint , 2025
    arXiv
  5. minisequencetransformers.png
    Mini-Sequence Transformers: Optimizing Intermediate Memory for Long Sequences Training
    Cheng Luo , Jiawei Zhao, Zhuoming Chen , Beidi Chen , and Anima Anandkumar
    In Advances in Neural Information Processing Systems (NeurIPS) , 2024
    arXiv

Please refer to my Google Scholar for a complete list of publications.