Bringing Power Directly to the AI Processor

­This column has focused many times on the upcoming challenges for data centres, whether that is meeting the data centre’s needs for power in the first instance, or for taking that power and converting it to feed the computing racks, but those two issues are not the only ones – taking the power supplied to the racks and feeding it to the processing chips is another area that is set to be just as challenging. Each Nvidia Grace Hopper IC can use up to 700W and the company’s next generation Blackwell GPUs can demand over 1,200W. These numbers will only continue to rise in the future. As the voltage levels that the chips use is normally under one volt, that means that each chip may need a thousand or more amps supplied. In some ways the challenge here is even larger. The power just not only has to be delivered, it has to be carefully sequenced to ensure the chip operates properly. The power supply also has to react quickly to changing loads, and, of course, do all of that at the highest efficiency levels.

Eric Pittana, Senior Director of Global Marketing and EMEA Sales at Empower Semiconductor explains, “Digital systems were essentially doubling their density frequently, but power management was behind, relying on architectures and technologies that were proven, but certainly not improving at the same pace. Now, the massive growth of AI processors have seen a tremendous increase in power demand. How do you get that massive amount of power through to the final stage of conversion at the processor? If you look at a typical NVIDIA board, on the top side there is a sea of power converters and inductors surrounding the chips, and on the back side, right underneath the processor, there is a massive amount of capacitors, which ensure the power integrity of the processor. Some companies have made strides on bringing power converters to the back side, but to an extent they still remain far out from the SOC.”

He continues, “Empower has developed a solution that provides true vertical delivery. The technique eliminates all of the capacitors under the IC and places converters in their space. These converters are only between 1 and 2mm tall, so can deal with pretty much any type of backside constraint in the industry. Placing the last stage conversion directly under the IC also gets significant performance benefits in terms of efficiency. The large current now only has to travel the thickness of the PCB and not a few inches. Our calculations show that this proximity can save between 5% and 20% of power distribution losses over traditional lateral current transmission systems.”

Empower calls its solution the Crescendo Platform, and it is based on the company’s Integrated Voltage Regulator (IVR), a 5mm x 5mm IC that combines a switching voltage regulator with all necessary control and filtering circuitry. The IVRs are digitally configurable and can be used together to handle up to 3000A of current.

Expanding, Pittana says, “The Crescendo Platform operates at speed that are a magnitude faster than traditional power management products. Multi-megahertz speeds, by extension, provide a multi-megahertz bandwidth. This means that the control loop has the ability to react to any type of load demand from the processor or transient. Other solutions rely on localized energy storage capacitors under the processor to accomplish this task. IVRs get benefits in terms of both efficiency and power density. We estimate that when you take everything into account, the results are a 5X smaller footprint, and a 2X better transient response. On performance, we have seen independent numbers from our customers that say there is a 5% to 10% increase in processing performance.”

https://www.empowersemi.com/