We investigate scale-out context memory architectures for large language models, treating context as a first-class system resource rather than a static data object. Our research spans memory systems, storage, networking, runtime scheduling, and power management to enable efficient management of massive context workloads across heterogeneous resources. The project aims to establish the foundation for future hyperscale AI inference platforms supporting long-context reasoning, personalization, and agent-based AI services.
We are developing Ethernet-based GPU cluster network fabric system and optimization technologies to maximize network efficiency in large-scale GPU cluster environments. This three-year project is carried out in collaboration with Acryl Co., Ltd., Yonsei University, and Sungkyunkwan University.
Key research directions include:
Artificial intelligence services increasingly rely on large language models (LLMs) and heterogeneous AI accelerators distributed across cloud and data center environments. This project proposes Inference-over-Fabrics (IoF), a kernel-integrated architecture that enables remote AI inference resources to be accessed as first-class system resources, analogous to the way NVMe-over-Fabrics virtualizes remote storage. By moving inference management into the operating system kernel, IoF aims to eliminate unnecessary user-kernel transitions, reduce communication overhead, and provide a unified, low-latency interface for remote AI accelerators. The project investigates kernel-level resource abstraction, high-performance request/response communication over RDMA and TCP, and lightweight remote inference servers, ultimately establishing a scalable foundation for next-generation cloud and edge AI infrastructure.
Despite the emergence of 400 Gbps-class network hardware, the Linux kernel’s network subsystem serializes packet processing for a single connection on a small number of cores. This fails to utilize the full hardware performance. This project will develop parallelization and modularization techniques for the network subsystem together with load-driven adaptive scaling. These techniques will enable even a single network connection to be processed by multiple cores without the need for dedicated stacks, such as DPDK or RDMA. The techniques will also maintain compatibility with the existing Linux kernel.
Key research directions include:
Led by the Chungbuk Science & Technology Innovation Institute, this project brought together 12 institutions to develop big data and AI platform infrastructure for the wellcare industry spanning healthcare, beauty, and food. The principal investigator led the participating institution’s research. The team built AI computing infrastructure (GPU clusters) and developed standard AI models specialized for the wellcare industry based on federated learning, meta-learning, and model compression, and also enhanced the AI platform.