OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelism

Abstract: Embodied AI agents increasingly execute multiple tasks in parallel from shared observations, but existing VLA inference systems suffer from redundant computation and resource contention caused by isolated KV cache management. OxyGen treats the KV cache as a shared resource across tasks and time, enabling cross-task KV sharing and cross-frame continuous batching. Implemented for pi-0.5 and evaluated on NVIDIA RTX 4090 and Jetson AGX Thor platforms, OxyGen achieves up to 3.7x speedup while simultaneously delivering over 200 tokens/s language throughput and 70 Hz action frequency without degrading action quality.

@inproceedings{neuripsoxygen,
  title={OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelism},
  author={Xiangyu Li and Huaizhi Tang and Xin Ding and Weijun Wang and Ting Cao and Yunxin Liu},
  booktitle={Conference on Neural Information Processing Systems (NeurIPS)},

  year={2026}
}