If we want AI to succeed and unlock the urgent solutions we need for the planet and humanity we have some big energy and waste challenges before us, says Mark Kidd in this thought provoking article.
The record-breaking uptake of ChatGPT has raised huge interest – and investment – in generative AI. The ability of ChatGPT and other large language models (LLMs) to bridge the linguistic gap between humans and machines has caught the popular imagination and raised awareness of the potential to automate and improve many aspects of our lives. Looking at this phenomenal new technology from the point of view of the digital infrastructure it will need to succeed, there are two key challenges – energy and e-waste – that will require particular attention from the federal government, industry and businesses.
Step change
Generative AI will power more and more applications over the coming decade. ChatGPT, DALL-E, GitHub Copilot and Stable Diffusion, are just the first generation, creating and sorting images, answering complex questions, creating websites, and making programming accessible to all.
Data centres will need to have the capability to provide the infrastructure for many high power computing, or HPC, configurations running generative AI, and will need to develop specialist facilities that meet their needs.
High density power, modular architecture, high bandwidth training (input) and inference (output) connectivity and advanced cooling are all critical factors. In order to adequately support these new applications that promise to accelerate innovation and even save lives, we need to see industry start to make a greater step change in infrastructure design.
Power surge
By far the greatest challenge in supporting generative AI is a huge surge in power loads. Generative AI models use graphics processing unit (GPU) chips which require 10–15 times the energy of a traditional central processing unit (CPU). Many models have billions of parameters and require fast and efficient data pipelines in their training phase, which can take months to complete. ChatGPT 3.5, for instance, has 175 billion parameters and was trained on over 500 billion words of text. To train a ChatGPT 3.5 model requires 300-500 MW of power. Currently, a typical data center requires 30-50 MW of power. While LLMs are definitely at the most power-hungry end of the generative AI boom, every generative model has processor and power needs which grow exponentially, either doubling or tripling each year.
Forecasting the power requirements of generative AI over time is hard to do with any accuracy, but most analysts would agree that it will ramp up current requirements hugely.
If one estimates current data centre compound growth at a relatively modest 15 per cent (it’s probably nearer 20 per cent), global capacity will double in five years and quadruple in 10. With generative AI in the mix, the compound annual growth rate could rise as high as 25 per cent, tripling capacity in five years and increasing it ninefold in a decade. Enterprises, AI startups and Cloud Service Providers are already racing to secure data centre capacity for their workloads, with the hyperscale clouds leading the pack.
This is happening fast. Analyst TD Cowen reported “a tsunami of AI demand,” with 2.1 GW of data centre leases signed in the US (a fifth of current total supply) in Q2 2023.
a wave of e-waste
The second AI-generated tsunami is at the back end; a stream of used equipment. AI is driving faster server innovation, particularly in chip design, and the latest AI chips, such as the Nvidia H100, have had so many billions advanced against their manufacture and are in such short supply that they are even being used as debt collateral and made available for rent.
While this refresh rate will be key to improving efficiency it will also increase the scale of e-waste, in tandem with the rise in capacity, -. E-waste is one of the fastest-growing waste streams in the world, and the fastest growing waste stream in Australia.
By 2030, annual e-waste production is on track to reach a staggering 75 million metric tons. Global e-waste is thought to hold roughly $60 billion-worth of raw materials such as gold, palladium, silver, and copper. However, just 17 per cent of global e-waste is documented to be collected and properly recycled each year.
Rising to the challenge
These waves at the front and back end of the data centre will take place as the climate crisis deepens and zero emission targets loom. There will be unprecedented pressure on power grids to provide new electrical power for industries that are weaning themselves off fossil fuels. It is fair to say that generative AI in particular will be under intense environmental scrutiny.
To address the twin challenges of capacity growth and e-waste the industry will have to be at the top of its game.
Decarbonisation & recycling
Low-to-no-carbon power sources will be the key to addressing power challenges. The power demands of generative AI will accelerate this focus and drive new innovations in microgrids and backup power sources such as battery, hydrogen and nuclear. Renewables will also be key. Most hyperscalers and a growing number of colocation providers have been growing the green grid and eliminating carbon to the point that today, hyperscalers are the biggest buyers of renewables in the world.
Data center owners will now need to take a further step. Following Google’s lead, two years ago Iron Mountain committed to provide not just 100 per cent renewables but 24/7 carbon-free power. In time, it is likely that this approach will in time replace the current year-by-year renewable power purchase agreement model.
Recycling & remarketing
New chips and superfast GPUs will drive the AI revolution, but what will happen to the old ones? For both efficient performance and impact reduction, AI providers will need to check that IT asset lifecycle optimisation and recycling, remarketing and secure disposal are available.
The industry has been fairly slow to integrate this, but this needs to accelerate and there is huge potential for this segment to increase to support AI customers over the coming years.
Generating new opportunities
In the same way that generative AI will revolutionise the industries that run its applications, ultimately it will revolutionise the infrastructure industry that supports it.
In order to leverage this new technology to its full extent and to enjoy the vast economic value, estimated by Goldman Sachs to be almost $7 trillion globally (7 per cent of global GDP), it will take commitment and collaboration from government entities, industries and businesses alike.