LLM Security · NPU Acceleration · Microarchitectural Side-Channels · RISC-V
Ongoing Research
2025–Present
Protecting Privacy of Mixture-of-Expert Large Language Model Inference Via Slotted Execution
This research explores the privacy implications of accelerating large language model inference on neural processing units (NPUs), using the open-source RISC-V Gemmini accelerator as an experimental platform. The project focuses on Mixture-of-Experts (MoE) models, whose input-dependent routing creates highly variable computational workloads across experts. We investigate how the previously proposed MoEcho attack, developed by my colleagues to recover information from MoE execution patterns, extends to NPU-accelerated inference. By instrumenting Gemmini with hardware performance counters and integrating it as an inference backend, we demonstrated that expert activity during the prefill phase remains observable and can be used to recover information about model inputs. We then explored runtime defenses, culminating in randomized time-slotted expert execution, which obscures expert workload and execution order while preserving the model's routing decisions. The current phase of the project broadens this question beyond MoE scheduling to examine whether emerging LLM performance optimizations introduce new input-dependent behavior that can compromise privacy when inference is offloaded to NPUs.
- LLM Acceleration on RISC-V NPUs: Uses Gemmini as an open-source NPU platform to study how LLM inference behavior changes when computationally intensive operations are moved from the CPU to specialized accelerator hardware.
- Extending MoEcho to NPU Execution: Investigates whether the input-dependent expert workloads exploited by the previously proposed MoEcho attack remain observable when Mixture-of-Experts inference is accelerated on Gemmini, including through hardware performance counters and accelerator execution behavior.
- Performance-Counter Leakage: Demonstrates that expert workload during the prefill phase produces distinguishable accelerator activity, allowing template-based attacks to infer properties of the input from Gemmini performance measurements.
- Randomized Slotted Execution: Develops a runtime mitigation that divides expert computation into regular-sized execution slots and randomizes their scheduling, reducing leakage from both expert workload and execution order without changing the model's routing decisions.
- Privacy of LLM Optimizations: Extends the research beyond MoE scheduling to study whether recently proposed NPU-oriented LLM optimizations create new input-dependent execution patterns, data movement, or heterogeneous workloads that can themselves become side channels.