NYCU HPC PaaS Platform
A PaaS inference platform being built on NYCU's HPC cluster — 5 NVIDIA DGX nodes (H100 / H200) scheduled with Slurm, over InfiniBand and a WEKA parallel filesystem. Work in progress at the university's IT Service Center.
Work in progress. The cluster described below already existed when I joined — my part is the PaaS inference platform being built on top of it, plus operations work on the platform itself.
The platform
NYCU’s high-performance computing service runs GPU workloads for researchers across the university. The cluster is built around NVIDIA DGX nodes and managed as a single Slurm-scheduled resource pool:
- 5 DGX compute nodes — two with H100 GPUs, three with H200
- InfiniBand interconnect between compute nodes, so multi-node jobs are not bottlenecked by the general-purpose network
- WEKA parallel filesystem for shared storage, reached over the Ethernet fabric — the two networks carry different traffic
- Slurm for scheduling, with partitions separating the H100 and H200 pools, plus a MIG partition for jobs that only need a fraction of a GPU
- NVIDIA Base Command Manager (BCM) for cluster provisioning and management, with a REST API that lets other systems query and drive the cluster
- LDAP for identity, so accounts are consistent across the login and compute nodes
Users submit work from a login node; the scheduler places it on whichever compute node has the resources free. The architecture behind this — and why HPC clusters are shaped this way — is written up separately in HPC 叢集架構.
What I am building
A PaaS inference platform on top of the cluster.
The gap it fills: Slurm is built for batch work — you submit a job, it queues, it runs to completion, it releases the resources. That model fits training runs well, but it fits inference services badly. Someone who wants to expose a model as an endpoint does not want to write a batch script and wait in a queue; they want a service that stays up and answers requests.
The goal is to let researchers deploy and serve models as a platform service, while the underlying GPUs remain part of the same shared, accounted-for resource pool — rather than having inference workloads quietly squat on nodes outside the scheduler’s view.
Operations
Alongside the platform work, the day-to-day side of running a shared service:
- Responding to user problems — quota, connectivity, and billing questions
- Maintaining the account-application system that provisions cluster access
- Keeping the platform documentation current
Related
- HPC 叢集架構 — how the pieces fit together, and why
- Job Scheduling & Resource Management System — Slurm and PBS in depth