Weili Xu

^* Equal contribution. ^† Project lead.

1. Evolving Infra

   

   Evolving Infra: Agent-Adaptive LLM Serving at Scale

   Hao Kang^*, Weili Xu^*, Ziyang Li^*, Zheng Du^*, Hanchen Li, Tian Xia, Song Bian, Tushar Krishna, Huizi Mao, Ion Stoica, and Song Han

   2026

   Preprint

   arXiv Bib Code Website

   @misc{kang2026evolvinginfra,
     title = {Evolving Infra: Agent-Adaptive LLM Serving at Scale},
     author = {Kang, Hao and Xu, Weili and Li, Ziyang and Du, Zheng and Li, Hanchen and Xia, Tian and Bian, Song and Krishna, Tushar and Mao, Huizi and Stoica, Ion and Han, Song},
     year = {2026},
     note = {Preprint},
   }

2. FlashStates

   

   FlashStates: Unlocking Flexible Hybrid State Management for Agentic LLM Inference

   Chenxi Li^*, Weili Xu^*†, Tianyi Zhang, Jue Wang, Zhongzhu Zhou, and Chenfeng Xu

   2026

   Preprint

   Bib Code Website

   @misc{li2026flashstates,
     title = {FlashStates: Unlocking Flexible Hybrid State Management for Agentic LLM Inference},
     author = {Li, Chenxi and Xu, Weili and Zhang, Tianyi and Wang, Jue and Zhou, Zhongzhu and Xu, Chenfeng},
     year = {2026},
     note = {Preprint},
   }

3. QATFactory

   

   QATFactory: A Versatile, Deployment-Aligned Framework for Quantization-aware Training and Distillation of LLMs

   Weili Xu^*, Jisen Li^*, Yuqing Jian^*, Chenxi Li, Zhizhou Sha, Yifan Yu, Qingyang Wu, Chenfeng Xu, Zhongzhu Zhou, Tianyi Zhang, and Ben Athiwaratkun

   2026

   Preprint

   arXiv Bib Code Website

   @misc{xu2026qatfactory,
     title = {QATFactory: A Versatile, Deployment-Aligned Framework for Quantization-aware Training and Distillation of LLMs},
     author = {Xu, Weili and Li, Jisen and Jian, Yuqing and Li, Chenxi and Sha, Zhizhou and Yu, Yifan and Wu, Qingyang and Xu, Chenfeng and Zhou, Zhongzhu and Zhang, Tianyi and Athiwaratkun, Ben},
     year = {2026},
     note = {Preprint},
   }

4. ThunderAgent

   

   ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System

   Hao Kang^*, Ziyang Li^*, Weili Xu^*, Xinyu Yang^*, Yinfang Chen, Junxiong Wang, Beidi Chen, Tushar Krishna, Chenfeng Xu, and Simran Arora

   In Proceedings of the International Conference on Machine Learning (ICML), Jul2026

   ICML 2026 (July) Spotlight. Integrated into NVIDIA Dynamo and SkyRL.

   Abs arXiv Bib PDF Forum Code Website

   ThunderAgent is a simple, fast, and program-aware system for agentic inference. It introduces a program-aware scheduler to improve KV-cache reuse and balance memory usage across nodes, together with tool-call lifecycle management for long-running agentic rollouts. ThunderAgent has been integrated into NVIDIA Dynamo and SkyRL.

   @inproceedings{kang2026thunderagent,
     title = {ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System},
     author = {Kang, Hao and Li, Ziyang and Xu, Weili and Yang, Xinyu and Chen, Yinfang and Wang, Junxiong and Chen, Beidi and Krishna, Tushar and Xu, Chenfeng and Arora, Simran},
     booktitle = {Proceedings of the International Conference on Machine Learning (ICML)},
     month = jul,
     year = {2026},
     url = {https://openreview.net/forum?id=kR4iOTaAOJ&referrer=%5Bthe%20profile%20of%20Beidi%20Chen%5D(%2Fprofile%3Fid%3D%7EBeidi_Chen1)},
     forum = {https://openreview.net/forum?id=kR4iOTaAOJ&referrer=%5Bthe%20profile%20of%20Beidi%20Chen%5D(%2Fprofile%3Fid%3D%7EBeidi_Chen1)},
     note = {ICML 2026 (July) <strong>Spotlight</strong>. Integrated into <a href="https://docs.nvidia.com/dynamo/latest/user-guides/agents/thunder-agent-program-scheduler">NVIDIA Dynamo</a> and <a href="https://github.com/NovaSky-AI/SkyRL/tree/main/examples/train/thunder_agent">SkyRL</a>.},
   }

5. AuroraLong

   

   AuroraLong: Bringing RNNs Back to Efficient Open-Ended Video Understanding

   Weili Xu, Enxin Song, Wenhao Chai, Xuexiang Wen, Tian Ye, and Gaoang Wang

   In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Oct2025

   Abs arXiv Bib PDF Supp Code

   The challenge of long video understanding lies in its high computational complexity and prohibitive memory cost, since the memory and computation required by transformer-based LLMs scale quadratically with input sequence length. We propose AuroraLong to address this challenge by replacing the LLM component in MLLMs with a linear RNN language model that handles input sequence of arbitrary length with constant-size hidden states. To further increase throughput and efficiency, we combine visual token merge with linear RNN models by reordering the visual tokens by their sizes in ascending order. Despite having only 2B parameters and being trained exclusively on public data, AuroraLong achieves performance comparable to Transformer-based models of similar size trained on private datasets across multiple video benchmarks. This demonstrates the potential of efficient, linear RNNs to democratize long video understanding by lowering its computational entry barrier. To our best knowledge, we are the first to use a linear RNN based LLM backbone in a LLaVA-like model for open-ended video understanding.

   @inproceedings{xu2025auroralong,
     title = {AuroraLong: Bringing RNNs Back to Efficient Open-Ended Video Understanding},
     author = {Xu, Weili and Song, Enxin and Chai, Wenhao and Wen, Xuexiang and Ye, Tian and Wang, Gaoang},
     booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
     month = oct,
     year = {2025},
     pages = {23453-23465},
   }

6. Video-MMLU

   

   Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark

   Enxin Song, Wenhao Chai, Weili Xu, Jianwen Xie, Yuxuan Liu, and Gaoang Wang

   In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, Oct2025

   Abs arXiv Bib PDF Supp Code Website

   Recent advancements in language multimodal models (LMMs) for video have demonstrated their potential for understanding video content, yet the task of comprehending multi-discipline lectures remains largely unexplored. We introduce Video-MMLU, a massive benchmark designed to evaluate the capabilities of LMMs in understanding Multi-Discipline Lectures. We evaluate over 90 open-source and proprietary models, ranging from 0.5B to 40B parameters. Our results highlight the limitations of current models in addressing the cognitive challenges presented by these lectures, especially in tasks requiring both perception and reasoning. Additionally, we explore how the number of visual tokens and the large language models influence performance, offering insights into the interplay between multimodal perception and reasoning in lecture comprehension.

   @inproceedings{song2025videommlu,
     author = {Song, Enxin and Chai, Wenhao and Xu, Weili and Xie, Jianwen and Liu, Yuxuan and Wang, Gaoang},
     title = {Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark},
     booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},
     month = oct,
     year = {2025},
     pages = {6099-6113},
   }