Artificial intelligence (AI) still cannot think like a human. However, thanks to advances in large language models (LLMs) and machine learning (ML), AI can learn. Because AI-powered systems can analyse vast data sets in real time and learn new ways to improve hyperscale resourcing efficiencies, the data center industry has entered an explorative era in innovation. This blog examines some of the standout AI and hyperscale trends making waves in 2024 and beyond.
Artificial intelligence development and investment
AI is reshaping hyperscale development and investment strategies. By leveraging ML to analyze data and extract meaningful insights, AI’s increasingly sophisticated real-world applications continue to revolutionize how various industries deliver personalized end user experiences. The trend towards faster access to tailored information is not without ethical considerations, including AI governance and transparency. Let’s cover the development and investment trends in AI accelerators and AI-as-a-Service (AIaaS).
AI accelerators
Accelerators may be specially designed software or hardware components deployed to improve the speed and efficiency of AI computations. There are many instances of AI accelerators in common usage today. For example, field-programmable gate arrays (FPGAs) enable graphics-intensive gaming, while developers use graphics processing units (GPUs) to train neural networks. Tailor-made neural processing units (NPUs) offer another example – many modern mobile devices contain NPUs, which accelerate the way AI algorithms interpret and process data.
AI accelerators enable multiple industries to drive innovation and efficiency. An example is the UK transport industry. AI adoption in this area is helping form future congestion alleviation plans, with a UK university receiving a patent for a responsive, AI-powered traffic light system. The traffic calming initiative will analyze travel patterns and adjust traffic light timings in response to real-time traffic flow challenges.
AI-as-a-Service
Hyperscale customers seeking to incorporate AI-powered tools into their systems may leverage selected services via a third-party vendor, an approach known as AlaaS. The global AIaaS market value stands at $10bn+, with a predicted value of around $100bn by 2030. The benefits of AIaaS range from fast deployment times and service stability to version control and automated security updates. Popular AIaaS services include:
Machine learning (ML) frameworks
To train and deploy an AI model, developers must use ML frameworks. However, the ML framework alone does not provide a complete machine learning operations (MLOps) pipeline. For the AI model to succeed, developers require additional tools and expertise. AIaaS provides end-to-end MLOps solutions, enabling seamless AI model deployment.
Application programming interface (API)
Using only a few lines of code, developers can enhance AI functionality by deploying natural language processing APIs (e.g., translation, sentiment analysis, entity extraction) and computer vision APIs (e.g., in-video search, face detection, object detection).
The growing trend in AIaaS adoption will help multiple industries identify inefficiencies and improve process optimization. Examples of industries that stand to benefit include healthcare (e.g., wearables, virtual assistants, and health record pattern detection), manufacturing (e.g., predictive analytics and quality control for consistent, error-free production) and hospitality (e.g., personalized recommendations, language translation, and fraud detection through real-time transaction pattern analysis). The customer service industry can also expect AlaaS to help deliver enhanced end user experiences—through greater sentiment analysis and contextually relevant interactions, one study predicts AI technology will automate 40% of customer interactions by 2030, freeing up customer service agents to handle more complex tasks.
Networking and bandwidth
AI-powered algorithms continue to grow in data-intensive complexity. As a result, data centers face increasing demand for high-speed networking and increased bandwidth capacity. The demand for—and trend toward—AI-compatible compute resources open new frontiers in distributed computing (i.e., simultaneous AI workload processing across multiple servers), edge computing (i.e., real-time decision making made possible via low-latency communication between edge devices such as smart phones and internet-of-things sensors), and cloud computing (i.e., scalable, flexible, AI services in the cloud, removing on-premises infrastructure requirements).
Let’s look at on board optics and chip on wafer on substrate technology, focusing on how the trend toward smaller and more concise component functionality will impact AI system development in 2024 and beyond.
On board optics (OBO)
In simplified terms, OBO technology places components close together on the central processing unit (CPU), minimizing the energy expenditure and transit times otherwise associated with processing digital data. Another way to imagine OBO is to consider a well-organized toolbox, conveniently shrinking the proximity between logically sequenced, task-specific tools.
The reduced latency and faster task completion times linked to OBO (OBO modules can achieve interface speeds from 400Gb/s to 1.6Tb/s) stand to significantly improve the performance and efficiency of AI systems. The trend toward shrinking the distance between electrical CPUs and optical components drives innovation in GPUs, TPUs, and FPGAs, meaning OBO will likely emerge as a main hyperscale technology within the next one-to-two generations of accelerator and network development.
Chip on Wafer on Substrate (CoWoS)
Continuing the theme of trending toward minimizing distances between components in AI powered systems, CoWoS technology enables semiconductor integrations across memory, processors, and specialized accelerators, all on a single silicon wafer.
By integrating High Bandwidth Memory (HBM) directly onto the silicon chip, CoWoS technology promotes high-speed, low-latency memory access for AI accelerators (reducing data request bottlenecks, which plays a crucial role in enabling AI model training tasks).
The wide-ranging benefits of CoWoS packaging technology (i.e., grouping system-on-chip and HBM components on the same silicon chip) include increased performance, greater scalability, and lowered power consumption.
Industry leading CoWoS manufacturers face challenges in supplying hyperscale data centers with the necessary computational infrastructure to process complex AI tasks. Challenges include maintaining high yields and performance, environmental concerns (linked to sustainable production practices), and scaling production capacity—due to demand, one leading manufacturer recently increased its estimated monthly output from around 35,000 to 40,000 wafers by the end of 2024.
Innovation in fiber technology
The concept of multicore fibers (MCF) first appeared several decades ago. The optical fiber and hyperscale industries have since become familiar with multiple cores contained within a single cable, enabling significantly enhanced data transmission capacity. The imperative to expand optical transmission systems and increase optical fiber transmission distances has grown progressively urgent. To meet demand, modern multicore cables typically range from two to twenty-four cores, although much higher counts are achievable depending on user requirements.
Reduced fiber strand diameters
Reducing fiber strand diameters allows manufactures to accommodate more fiber strands inside a single cable. More strands per cable brings the inherent benefit of increased transmission capacity. However, there are two more benefits aligned with reduced fiber strand diameters.
Firstly, innovation in reduced fiber strand diameters has seen increases in data transmission rates. Although the concept of increasing transmission capacity by narrowing the fiber strand may seem counterintuitive, finer cables minimize dispersion and signal attenuation, enhancing signal quality. Next, smaller diameter strands serve to enhance the cable’s bend radius, increasing the cable’s flexibility and facilitating easier installation within congested hyperscale data center environments.
In 2023, 160-micron optical fibers became the smallest diameter fiber strands, enabling triple the capacity of traditional 250-micron fiber strands.
Conclusion – the future of AL cluster scale
AI-powered technologies indicate a strong correlation between compute performance and the number of available servers/nodes. For this reason, the symbiotic rise of AI and hyperscale will undoubtedly result in larger hyperscale data centers powered by larger AI cluster sizes. How large? Today’s large cluster sizes average around 16,000 GPU accelerators. By 2026-2027, growth rates suggest this number will increase 10-fold. At that stage, we will likely see clusters of clusters designed to crunch AI data.
Expansion on this scale brings efficiency considerations around power, cooling, white space optimization, and cabling. For more information about these topics, and to learn more about artificial intelligence development and investment, networking and bandwidth, and innovation in fiber technology, view and download the AI trends e-book from AFL.