Efficient VLM Inference System With LoRA Adapters At the Edge

Abstract: Vision Language Models enable complex vision tasks at the edge, while LoRA adapters provide lightweight domain specialization. Existing LoRA serving systems designed for text workloads, however, incur padding waste under heterogeneous adapter batching, cause request starvation during mode switching, and lack a principled way to generate accuracy-guaranteed adapters. VaLoRA addresses these limitations with an accuracy-aware adapter generator and hierarchical runtime router, an Adaptive-Tiling Matrix Multiplication operator, and a flexible orchestrator for mode switching and mixture inference. Across five vision tasks and three VLMs, VaLoRA improves accuracy by 24–62% over the original VLMs and reduces latency by 20–89% compared with S-LoRA, Punica, and dLoRA.

@article{tmcvalora,
  title={Efficient VLM Inference System With LoRA Adapters At the Edge},
  author={Weijun Wang and Liang Mi and Jinghan Chen and Tianze Yu and Ting Cao and Yuanchun Li and Meng Li and Haipeng Dai and Guihai Chen and Yunxin Liu},
  journal={IEEE Transactions on Mobile Computing (TMC)},

  year={2026}
}