Resources: chips
How chip power drives rack density and cooling
The power one accelerator draws sets how many fit in a rack, how much power the rack needs and whether air can still cool it. This page reads every figure from a record in the IT equipment library, with the vendor's own conditions. Where a vendor does not state a figure, we say so.
Chip power, system power and rack power are different measures. They are not added up or compared across vendors here, and performance claims are not compared.
From chip power to rack power
Stated accelerator power now runs from a few hundred watts for PCIe cards to well over a kilowatt for module-based GPUs. NVIDIA lists the H100 and H200 SXM at up to 700 W TDP, configurable (NVIDIA (opens in a new tab)). AMD lists 1,000 W typical board power for the MI350X and 1,400 W for the MI355X (AMD (opens in a new tab)). Lenovo documents B200 GPUs as 1,000 W SXM6 parts in its SR680a V3 (Lenovo (opens in a new tab)). Microsoft gives Maia 200 a 750 W SoC TDP (Microsoft (opens in a new tab)). PCIe cards sit lower: 300 W for the Tenstorrent Blackhole p150b and 250 W for the Cambricon MLU370-X8.
Grouping more accelerators per rack multiplies this. An eight-GPU DGX B200 uses about 14.3 kW and is air cooled (NVIDIA (opens in a new tab)). A 72-GPU GB200 NVL72 rack draws approximately 120 kW and the GB300 NVL72 up to 142 kW (NVIDIA (opens in a new tab)). NVIDIA's DSX facility reference design plans compute cabinets at 198 to 330 kW, from MGX Gen 1.1 to Vera Rubin NVL72 (NVIDIA DSX (opens in a new tab)).
Not every AI rack is that dense. SambaNova states 10 kW typical and 16.4 kW maximum for an air-cooled SambaRack SN40-16 and a 20 kW average for the SambaRack SN50 (SambaNova (opens in a new tab)). A Cerebras Wafer-Scale Cluster rack is provisioned at 34 kW (Cerebras (opens in a new tab)).
What it does to cooling
At rack scale the vendors move heat to liquid. NVIDIA describes the GB300 NVL72 as a liquid-cooled rack, and the GB200 NVL72 as hybrid, with direct liquid cooling plus air for the remaining load (NVIDIA reference architecture (opens in a new tab)). AMD lists direct liquid cooling for the MI455X (AMD (opens in a new tab)). Google says its liquid cooling runs at gigawatt scale and shows third-generation CDUs serving an Ironwood superpod (Google Cloud (opens in a new tab)).
The DSX reference design sizes the technology cooling loop for at least 1.5 L/min per kW (NVIDIA DSX (opens in a new tab)). As a worked example of that design rule, not a vendor figure, a 142 kW rack would need about 213 L/min.
Some designs bring liquid to the chip without facility water at the rack. Meta's MTIA 400 rack design uses air-assisted liquid cooling so it can be deployed quickly in existing halls, and Meta says MTIA 400 also supports facility liquid cooling (Meta (opens in a new tab)). Microsoft cools Maia 200 with a second-generation closed-loop heat exchanger unit (Microsoft (opens in a new tab)). Cerebras says the CS-3 can use an external liquid loop or liquid-to-air cooling (Cerebras (opens in a new tab)). Air still serves PCIe cards and lower-density inference racks, such as the SambaNova and Tenstorrent products above.
To turn a rack figure into electrical and cooling loads, use the engineering pack. For cooling equipment, see CDUs and technology guides.
Power delivery inside the rack
NVIDIA plans its Kyber rack generation for 576 Rubin Ultra GPUs by 2027, with 800 V DC power delivery and liquid cooling. It says 800 V DC carries over 150% more power through the same copper, removing the need for 200 kg copper busbars to feed a single rack, and moves in-rack components from 54 V DC to 800 V DC (NVIDIA (opens in a new tab)). In July 2026 the research firm SemiAnalysis reported Kyber NVL144 delayed to 2028, and NVIDIA replied that its roadmap is intact (CNBC (opens in a new tab)). Both are shown, and neither is confirmed here.
Memory and packaging
HBM sits in the accelerator package, so its power is part of board power and has no separate facility figure. The three HBM makers claim efficiency gains for HBM4: more than 40% over the previous generation from SK hynix (SK hynix (opens in a new tab)), 40% from Samsung (Samsung (opens in a new tab)), and more than 20% over its HBM3E from Micron (Micron (opens in a new tab)). These are vendor claims, not independently verified. TSMC plans 9.5-reticle CoWoS packages with 12 or more HBM stacks in volume production in 2027 (TSMC (opens in a new tab)). Larger packages put more silicon in one module, and the module's power is set by the chip product.
Products by segment
Each row links to its full record with conditions, sources and accessed dates. In-house hyperscaler chips link to the operator profile, because they are not sold as hardware. Suppliers are listed in the AI and data centre chips category.
GPU and accelerator makers
| Product | Power as stated | Cooling | Rack power | Next on the vendor roadmap |
|---|---|---|---|---|
| GB300 NVL72 rack NVIDIA | Not stated | Liquid | up to 142 kW (Full rack power requirement) | None announced in the sources checked |
| DSX reference design compute cabinet range (MGX Gen 1.1 to Vera Rubin NVL72) NVIDIA | Not stated | Liquid | 198 to 330 kW (Cabinet TDP range, MGX Gen 1.1 to Vera Rubin NVL72) | None announced in the sources checked |
| DGX B200 NVIDIA | about 14.3 kW (DGX B200 system (8 GPUs), System power usage) 1,000 W (B200 GPU in Lenovo SR680a V3, SXM6 GPU rating as documented by Lenovo) | Air | Not stated | None announced in the sources checked |
| Instinct MI355X AMD | 1,400 W (MI355X, Typical board power) | Not stated | Not stated | None announced in the sources checked |
| Instinct MI350X AMD | 1,000 W (MI350X, Typical board power) | Not stated | Not stated | None announced in the sources checked |
| DGX GB200 NVL72 rack NVIDIA | Not stated | Liquid | approximately 120 kW (Rack power consumption) | None announced in the sources checked |
| H100 Tensor Core GPU (SXM and NVL) NVIDIA | 700 W (H100 SXM, Max TDP, configurable) 350 to 400 W (H100 NVL, Max TDP, configurable) | Not stated | Not stated | None announced in the sources checked |
| H200 GPU (SXM and NVL) NVIDIA | 700 W (H200 SXM, Max TDP, configurable) 600 W (H200 NVL, Max TDP, configurable) | Not stated | Not stated | None announced in the sources checked |
| Vera Rubin NVL72 rack NVIDIA | Not stated | Not stated | Not stated | 2027: Kyber rack generation with 576 Rubin Ultra GPUs, 800 V DC power delivery and liquid cooling |
| Instinct MI455X and Helios rack AMD | Not stated | Liquid | Not stated | 2028: MI600 Series GPUs and Helios 600 rack |
| Gaudi 3 AI accelerator OAM (HL-325L) Intel | 900 W (HL-325L, passive or liquid cooling, Total device power, technical paper of July 2025) 1,200 W (HL-325L, liquid cooling (earlier figure), Hot Chips 2024 presentation) | Air or liquid | Not stated | None announced in the sources checked |
| CS-3 system (WSE-3) Cerebras | 23,000 W (CS-3 system, Peak system power) | Liquid | 34 kW (Provisioned power per rack, Wafer-Scale Cluster example) | None announced in the sources checked |
| SambaRack SN40-16 (SN40L RDU) SambaNova | Not stated | Air | 10 typical, 16.4 max kW (Inference power consumption per rack) | None announced in the sources checked |
| SambaRack SN50 (SN50 RDU) SambaNova | Not stated | Air | 20 kW (Average power per rack) | Second half of 2026: SN50 starts shipping to customers |
| Blackhole p150b PCIe card Tenstorrent | 300 W (p150b, Total board power) | Air | Not stated | None announced in the sources checked |
Custom and hyperscaler silicon
| Product | Power as stated | Cooling | Rack power | Next on the vendor roadmap |
|---|---|---|---|---|
| Ironwood TPU (seventh generation) | Not stated | Liquid | Not stated | 22 Apr 2026: Eighth-generation TPU 8t (training) and TPU 8i (inference) announced, available to Cloud customers soon |
| Trainium3 and Trn3 UltraServer AWS | Not stated | Not stated | Not stated | 2 Dec 2025: Trainium4 to use NVIDIA NVLink Fusion (no date stated) |
| Inferentia2 AWS | Not stated | Not stated | Not stated | None announced in the sources checked |
| Maia 200 Microsoft | 750 W (Maia 200 SoC, SoC TDP envelope) 700 W (Maia 100 (previous generation), Design TDP) | Liquid | Not stated | 26 Jan 2026: Maia 200 announced |
| MTIA family (MTIA 200 to MTIA 500) Meta | 90 W (MTIA 200 (2024), TDP) | Liquid | Not stated | 2027: MTIA 500 mass deployment |
Chinese AI chips
| Product | Power as stated | Cooling | Rack power | Next on the vendor roadmap |
|---|---|---|---|---|
| Ascend 950 series and Atlas SuperPoDs Huawei | Not stated | Not stated | Not stated | Q4 2028: Ascend 970 |
| MLU370-X8 accelerator card Cambricon | 250 W (MLU370-X8, Maximum thermal power) | Air | Not stated | None announced in the sources checked |
Memory (HBM)
| Product | Power as stated | Cooling | Rack power | Next on the vendor roadmap |
|---|---|---|---|---|
| SK hynix HBM4 SK hynix | Not stated | Not stated | Not stated | 12 Sep 2025: HBM4 development completed and mass production readied |
| Samsung HBM4 Samsung | Not stated | Not stated | Not stated | 12 Feb 2026: HBM4 mass production started and commercial shipments made |
| Micron HBM4 Micron | Not stated | Not stated | Not stated | 10 Jun 2025: HBM4 36 GB 12-high samples shipped to key customers. Ramp planned in calendar 2026 |
Foundry and packaging
| Product | Power as stated | Cooling | Rack power | Next on the vendor roadmap |
|---|---|---|---|---|
| CoWoS advanced packaging TSMC | Not stated | Not stated | Not stated | 2028: A14 process enters production |
| Samsung Foundry SF2 (2 nm) Samsung | Not stated | Not stated | Not stated | 2026: 2 nm expanded to HPC (plan stated in 2023) |
| Intel Foundry 18A Intel | Not stated | Not stated | Not stated | 9 Oct 2025: Panther Lake on 18A to enter high-volume production at Intel's newest Arizona fab later in 2025 |
Networking and DPU silicon
| Product | Power as stated | Cooling | Rack power | Next on the vendor roadmap |
|---|---|---|---|---|
| Spectrum-4 SN5600 NVIDIA | 940 W (SN5600 switch system, Typical power) | Air | Not stated | None announced in the sources checked |
| BlueField-3 DPU NVIDIA | Not stated | Not stated | Not stated | 16 Mar 2026: BlueField-4 DPU announced as in full production with the Vera Rubin platform |
| Tomahawk 6 switch series Broadcom | Not stated | Not stated | Not stated | 3 Jun 2025: Tomahawk 6 shipping |
Gaps
Google, AWS and Huawei do not publish chip or rack power for the products listed, and Meta does not for MTIA 300 to 500. These fields show as not stated rather than estimated. Still in research:
- AI storage systems (no vendor data sheet checked so far states system power with its configuration)
- Rack power shelves, busbars and manifolds from rack integrators
- Dell rack-scale GB200 and GB300 systems (IR7000 rack, XE9712)
- Cisco Nexus 800G switches
- Groq GroqCard and GroqRack power (product brief host did not respond when re-checked on 19 Oct 2026)
- Hygon DCU accelerators (no vendor data sheet with power could be checked)
- Per-GPU TDP for NVIDIA Blackwell and Rubin GPUs from an NVIDIA data sheet
- Marvell custom AI products and networking silicon power
Updated 19 Oct 2026.
Sources
- AMD: AAI 2026: AMD delivers full-stack compute for the agentic AI era (press release) (opens in a new tab) (accessed 19 Oct 2026)
- AMD: AMD Instinct MI350X GPUs product page (opens in a new tab) (accessed 18 Oct 2026)
- AMD: AMD Instinct MI355X GPUs product page (opens in a new tab) (accessed 18 Oct 2026)
- AMD: AMD Instinct MI400 Series GPUs page (opens in a new tab) (accessed 19 Oct 2026)
- AMD: AMD Instinct MI455X GPU product page (opens in a new tab) (accessed 19 Oct 2026)
- AWS: Announcing Amazon EC2 Trn3 UltraServers (What's New) (opens in a new tab) (accessed 19 Oct 2026)
- AWS: AWS Inferentia page (opens in a new tab) (accessed 19 Oct 2026)
- Broadcom: Broadcom ships Tomahawk 6: world's first 102.4 Tbps switch (press release) (opens in a new tab) (accessed 19 Oct 2026)
- Cambricon: MLU370-X8 accelerator card product page (opens in a new tab) (accessed 19 Oct 2026)
- Cerebras: Cerebras CS-3 vs. Nvidia B200: 2024 AI accelerators compared (blog) (opens in a new tab) (accessed 19 Oct 2026)
- Cerebras: Cerebras Wafer-Scale Cluster data sheet (opens in a new tab) (accessed 19 Oct 2026)
- CNBC: Nvidia's next-gen AI rack system delayed to 2028 on manufacturing snags, SemiAnalysis says (opens in a new tab) (accessed 19 Oct 2026)
- Google: 3 things to know about Ironwood, Google's latest TPU (opens in a new tab) (accessed 19 Oct 2026)
- Google Cloud: AI infrastructure at Next '26 (blog) (opens in a new tab) (accessed 19 Oct 2026)
- Google Cloud: Ironwood TPUs and new Axion-based VMs for your AI workloads (blog) (opens in a new tab) (accessed 19 Oct 2026)
- Huawei: Groundbreaking SuperPoD interconnect: leading a new paradigm for AI infrastructure (HUAWEI CONNECT 2025 keynote) (opens in a new tab) (accessed 19 Oct 2026)
- Intel: Intel Gaudi 3 AI accelerator technical paper, July 2025 V1 Rev. 3 (opens in a new tab) (accessed 19 Oct 2026)
- Intel: Intel Gaudi 3 AI accelerator, Hot Chips 2024 presentation (opens in a new tab) (accessed 19 Oct 2026)
- Intel: Intel unveils Panther Lake architecture, first AI PC platform built on 18A (press release) (opens in a new tab) (accessed 19 Oct 2026)
- Lenovo: Lenovo ThinkSystem SR680a V3 with B200 (Lenovo Press) (opens in a new tab) (accessed 19 Oct 2026)
- Meta: Four MTIA chips in two years: scaling AI experiences for billions (blog) (opens in a new tab) (accessed 19 Oct 2026)
- Meta: Our next generation Meta Training and Inference Accelerator (blog) (opens in a new tab) (accessed 19 Oct 2026)
- Micron: Micron ships HBM4 to key customers to power next-gen AI platforms (press release) (opens in a new tab) (accessed 19 Oct 2026)
- Microsoft: Inside MAIA 100, Hot Chips 2024 presentation (opens in a new tab) (accessed 19 Oct 2026)
- Microsoft: Maia 200: the AI accelerator built for inference (Official Microsoft Blog) (opens in a new tab) (accessed 19 Oct 2026)
- NVIDIA: AWS integrates AI infrastructure with NVIDIA NVLink Fusion for Trainium4 deployment (technical blog) (opens in a new tab) (accessed 19 Oct 2026)
- NVIDIA: DGX GB Rack Scale Systems User Guide: Hardware (opens in a new tab) (accessed 19 Oct 2026)
- NVIDIA: DGX SuperPOD Reference Architecture featuring DGX GB200 (PDF) (opens in a new tab) (accessed 19 Oct 2026)
- NVIDIA: DSX Facilities Infrastructure Reference Design Overview (design guide v2.0, 19 August 2026) (opens in a new tab) (accessed 18 Oct 2026)
- NVIDIA: NVIDIA BlueField-3 DPU data sheet (opens in a new tab) (accessed 19 Oct 2026)
- NVIDIA: NVIDIA DGX B200 product page, specifications (opens in a new tab) (accessed 18 Oct 2026)
- NVIDIA: NVIDIA H100 Tensor Core GPU product page (opens in a new tab) (accessed 19 Oct 2026)
- NVIDIA: NVIDIA H200 GPU product page (opens in a new tab) (accessed 19 Oct 2026)
- NVIDIA: NVIDIA Vera Rubin Opens Agentic AI Frontier (press release) (opens in a new tab) (accessed 19 Oct 2026)
- NVIDIA: NVIDIA, partners drive next-gen efficient gigawatt AI factories in buildup for Vera Rubin (blog) (opens in a new tab) (accessed 19 Oct 2026)
- NVIDIA: NVL72 AI Factory Enterprise Reference Architecture: System Hardware and Components (opens in a new tab) (accessed 18 Oct 2026)
- NVIDIA: Spectrum-4 SN5000 2U Switch Systems Hardware User Manual: Specifications (updated 15 June 2026) (opens in a new tab) (accessed 19 Oct 2026)
- SambaNova: Introducing the SN50 RDU, purpose-built for agentic inference (blog) (opens in a new tab) (accessed 19 Oct 2026)
- SambaNova: SambaRack SN40-16 data sheet (opens in a new tab) (accessed 19 Oct 2026)
- SambaNova: SambaRack SN50 data sheet (opens in a new tab) (accessed 19 Oct 2026)
- Samsung Semiconductor: Samsung Electronics unveils foundry vision in the AI era at Samsung Foundry Forum 2023 (newsroom) (opens in a new tab) (accessed 19 Oct 2026)
- Samsung Semiconductor: Samsung ships industry-first commercial HBM4 (news) (opens in a new tab) (accessed 19 Oct 2026)
- SK hynix: SK hynix completes world's first HBM4 development and readies mass production (newsroom) (opens in a new tab) (accessed 19 Oct 2026)
- Tenstorrent: Blackhole PCIe cards, specifications and requirements (opens in a new tab) (accessed 19 Oct 2026)
- TSMC: TSMC unveils next-generation A14 process at North America Technology Symposium (press release) (opens in a new tab) (accessed 19 Oct 2026)