TENET: An Efficient Sparsity-Aware LUT-Centric Architecture for Ternary LLM Inference On Edge

Abstract: Ternary quantization reduces the computational and memory footprint of large language models, but conventional inference platforms cannot fully exploit its benefits because they lack native ternary arithmetic and memory support and remain underutilized in low-batch scenarios. TENET is a sparsity-aware LUT-centric architecture that co-optimizes algorithms, compute, and memory for ternary LLM inference. It combines a Sparse Ternary LUT core, Dynamic Activation N:M Sparsity, LUT-based ternary weight decompression, and a programmable heterogeneous accelerator with a Linear-Projection-aware Sparse Attention dataflow. FPGA and ASIC prototypes improve energy efficiency by 4.3x and 21.1x over an NVIDIA A100 GPU, respectively, while TENET-ASIC achieves a 2.7x average end-to-end inference speedup.

@inproceedings{microtenet,
  title={TENET: An Efficient Sparsity-Aware LUT-Centric Architecture for Ternary LLM Inference On Edge},
  author={Zhirui Huang and Rui Ma and Shijie Cao and Ran Shu and Ian Wang and Ting Cao and Chixiao Chen and Yongqiang Xiong},
  booktitle={IEEE/ACM International Symposium on Microarchitecture (MICRO)},

  year={2026}
}