We investigate scale-out context memory architectures for large language models, treating context as a first-class system resource rather than a static data object. Our research spans memory systems, storage, networking, runtime scheduling, and power management to enable efficient management of massive context workloads across heterogeneous resources. The project aims to establish the foundation for future hyperscale AI inference platforms supporting long-context reasoning, personalization, and agent-based AI services.
We are developing Ethernet-based GPU cluster network fabric system and optimization technologies to maximize network efficiency in large-scale GPU cluster environments. This three-year project is carried out in collaboration with Acryl Co., Ltd., Yonsei University, and Sungkyunkwan University.
Key research directions include:
Artificial intelligence services increasingly rely on large language models (LLMs) and heterogeneous AI accelerators distributed across cloud and data center environments. This project proposes Inference-over-Fabrics (IoF), a kernel-integrated architecture that enables remote AI inference resources to be accessed as first-class system resources, analogous to the way NVMe-over-Fabrics virtualizes remote storage. By moving inference management into the operating system kernel, IoF aims to eliminate unnecessary user-kernel transitions, reduce communication overhead, and provide a unified, low-latency interface for remote AI accelerators. The project investigates kernel-level resource abstraction, high-performance request/response communication over RDMA and TCP, and lightweight remote inference servers, ultimately establishing a scalable foundation for next-generation cloud and edge AI infrastructure.