SK Hynix: The competition in AI data centers is shifting from single chips to overall infrastructure architecture

SK Hynix stated in a recent article that the competition in AI is shifting from the performance of individual chips to the design and operation of the entire infrastructure architecture. The competitiveness of AI data centers no longer depends on individual components, but on whether the five key elements of computing, memory, storage, networking, and power cooling can be seamlessly integrated.

The article points out that the continuous expansion of AI model scales has led to a surge in demand for computing power and data movement. Training requires repeatedly reading massive datasets, while inference relies on quickly retrieving user request information, both of which place higher demands on the system architecture of data centers. At the memory level, HBM, server DRAM, and others have formed a hierarchical system, each undertaking different bandwidth and capacity tasks. At the networking level, as large-scale training and inference rely on multi-server parallel processing, networking has become a key factor determining the scalability of data centers. System design is shifting from single-server to whole rack and cluster-level expansion.

According to Omdia's forecast, the AI data center chip market will grow from $123 billion in 2024 to $207 billion in 2025, reaching $286 billion by 2030. SK Hynix also mentioned that Microsoft's Fairwater data center in Wisconsin is about the length of five football fields, indicating that infrastructure is being deployed on a larger scale. SK Hynix emphasizes that memory is becoming a key layer connecting computing and data.