Ke Wan

Hi! I’m a Software Engineer II on the HPC/AI team at Microsoft Azure Core, where I work on large-scale LLM inference and serving systems in production cloud environments. My research focuses on efficient transformer inference, system-aware LLM serving, and multimodal model evaluation. In particular, I study memory-efficient inference techniques such as KV-cache optimization, with the goal of improving scalability, concurrency, and throughput without requiring model retraining.

A central theme of my work is bridging research ideas with practical systems implementations. My research on KV-cache compression has been integrated and optimized across major inference frameworks, including vLLM and SGLang, and adapted to work with LMCache-based serving workflows. These implementations span algorithm design, GPU-kernel optimization, memory management, scheduling, and distributed inference across tensor and data parallel settings.

My research has received more than 100 citations, and related open-source projects have attracted over 1,200 GitHub stars. I have also completed approximately 45 manuscript reviews for leading journals, including IEEE Transactions on Image Processing and Neural Networks. I am broadly interested in developing reliable, scalable, and empirically grounded methods for evaluating and serving modern generative models.